Question about fullyConnectedLayer's configurations/settings

I want to verify the following:

I am working in a situation, where within the fullyconnectedLayer, I have a part of the set of weights, that I do NOT want updated during the back propagation. (I want to be able to elect which rows I want updated and others I don't want updated) I want the weights to be used in the predict/forward calculation, but NOT to be updated in the back propagation.

In this situation, I have determined, that there exists no settings within MATLAB's fullyConnectedLayer, the ability to go and set any value, so that the layer only updates on the rows of the weights selected. As a result of this, I have gone off and created my own Custom Fully Connected Layer to accomplish this.

I wanted to ask somone at MATLAB, if my understanding of the configurations/settings of 'fullyConnectedLayer' is indeed true ? There is no way to get the fullyConnectedLayer to operate, per the description I have provided above ?

[Eva]

I wanted to ask to verify this , because I am having some suboptimal performance in my Custom Fully Connected Layer that I am having a difficult time to mitigate.

If I could go back and use your fullyConnectedLayer, but able to use it in the way I need to, this would be preferred.

Answer 1

[Siraj]

Hii! It is my understanding that you have a fullyConnectedLayer which has a weight matrix of dimensions Output size X Input Size. In the training phase of the network all the values of this weight matrix get updated, while you want some rows to remain unaltered.

I investigated but was not able to see any configuration with which we can freeze some rows of this weight matrix such that values in these rows remain unaltered during the training of the network.

Please refer to the documentation of fullyConnectedLayer for more details.

https://in.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html

[Eva]

Thank you Siraj, for taking the time to look into this question!