MATLAB Answers

When use "ones" initialization in DeepLearning?

조회 수: 1(최근 30일)
Claudia Borredon
Claudia Borredon 2021년 4월 21일
답변: Divya Gaddipati 2021년 5월 11일
Hello everyone
I wanted to know when (or if) it is useful to use the "ones" weights initialization option in the fullyConnectedLayer or if it should be avoided.
Moreover what is the point of that kind of initialization, is it usually used as a reference or is it actively used sometimes?
Thank you in advance.

채택된 답변

Divya Gaddipati
Divya Gaddipati 2021년 5월 11일
Intuitively, with a constant weight initialization, all the layer outputs during the initial forward pass of a network are essentially the same and this makes it very hard for the network to figure out which weights to be updated. And, so any constant initialization would produce a poor result and so better to avoid using it.
Having the weights initialized with values sampled from a random distribution instead of constant values like zeros and ones actually helps the network to train better and faster. Moreover, neural networks being very sensitive and prone to overfitting, having random weight initialization actually prevents the neurons from learning the same features. Also, this imposed randomness is highly suitable for gradient-based optimization techniques and helps a network to better guide which weights to update. Hence, random weight initialization is more actively used.

추가 답변(0개)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by