How to get the gradient with respect to the output of a specific layer in a deep learning model?

Hi,

I have a model myModel which is a simple 2 Layers model (i.e. input size: 3; Layer_1 size: 7; Layer_2 size: 4).

That is, the layer's output is a and the loss is .

From below code, now I have the whose dimention is 60 since there are 60 parameters in the model (i.e. (3+1)*7+(7+1)*4 = 60). Nis the number of data points (i.e. in my code)

But want I want is the gradient with respect to the layer 2's output, instead of the loss. That is, I want instead, whose dimention would be since .

How can I get ? Many thanks!

My code

rng(123); % seed

X_ori=[4,163,80;5,164,75]; % data; #(number) = 2; #(features) = 3;
X=permute(X_ori,[3,4,2,1]);
dlX = dlarray(X, 'SSCB');

Y_ori=[0, 0, 0, 1; 0, 1, 0, 0]; % data labels (i.e. one-hot vectors for 4 classes)

myModel = [
    imageInputLayer([1 1 3],'Normalization','none','Name','in')
    fullyConnectedLayer(7,'Name','Layer 1')
    fullyConnectedLayer(4,'Name','Layer 2')];

MyLGraph = layerGraph(myModel);
myDLnet = dlnetwork(MyLGraph);
gradients = dlfeval(@modelGradients, myDLnet, dlX, Y_ori);

function [gradients] = modelGradients(myModel, modelInput, CorrectLabels)

CorrectLabels_transpose=transpose(CorrectLabels);

[modelOutput,state] = forward(myModel,modelInput);

loss = -sum(sum(CorrectLabels_transpose.*log(sigmoid(modelOutput/100))));
gradients = dlgradient(loss, myModel.Learnables);

end

Thanks

Answer 1

[Athul Prakash]

You could try looking at the source code for the fullyConnectedLayer that you're using; and then customizing that code to return the gradients you're looking for.

Alternatively, try creating your own custom layer to achieve this. See the doc below for that:

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html

Hope it Helps!