error message: 'dlgradient' inputs must be traced dlarray objects or cell arrays, structures or tables containing traced dlarray objects. To enable tracing, use 'dlfeval'.

I want a small network that contains only a batch normalization layer followed by fully-connected layer, softmax and finally cross entropy loss. To do that, my model is implemented as follows:

function [dlY1,state] = model(parameters,dlX,doTraining,state)
% Batch normalization, ReLU
    offset = parameters.batchnorm1.Offset;
    scale = parameters.batchnorm1.Scale;
    trainedMean = state.batchnorm1.TrainedMean;
    trainedVariance = state.batchnorm1.TrainedVariance;
    
    if doTraining
        [dlY,trainedMean,trainedVariance] = batchnorm(dlX,offset,scale,trainedMean,trainedVariance);
        
        % Update state
        state.batchnorm1.TrainedMean = trainedMean;
        state.batchnorm1.TrainedVariance = trainedVariance;
    else
        dlY = batchnorm(dlX,offset,scale,trainedMean,trainedVariance);
    end
    
    dlY = relu(dlY);
    
    % Fully connect, softmax (labels)
    weights = parameters.fc1.Weights;
    bias = parameters.fc1.Bias;
    dlY1 = fullyconnect(dlY,weights,bias);
    dlY1 = softmax(dlY1);
end

TO calculate the gradients, I have this function here:

function [gradients,state,loss] = modelGradients(parameters,dlX,T1,state)

    doTraining = true;
    [dlY1,state] = model(parameters,dlX,doTraining,state);
    
    lossLabels = crossentropy(dlY1,T1);
    
    loss = lossLabels;
    
    gradients = dlgradient(loss,parameters,'EnableHigherDerivatives',true);

end

TO start the program with simple example to calculate the derivatives:

 dlX=dlarray(rand(10,5),'BC');
T1=[1;2];  T1=repelem(T1,5);
dlY=onehotencode(categorical(T1'),1);
sz=[2,5];
numOut=2;
numIn=5;
parameters.batchnorm1.Offset = zeros([5 1]);
parameters.batchnorm1.Scale = ones([5 1]);
state.batchnorm1.TrainedMean = zeros(5,1,'single');
state.batchnorm1.TrainedVariance = ones(5,1,'single');
parameters.fc1.Weights=initializeGlorot(sz,numOut,numIn);
parameters.fc1.Bias=zeros(2,1);

[gradients,state,loss] = dlfeval(@modelGradients, parameters, dlX, dlY, state);

So, the gradients are the thing I am looking for and the output of this process is this following error message:

Error using dlfeval (line 43)
'dlgradient' inputs must be traced dlarray objects or cell arrays,
structures or tables containing traced dlarray objects. To enable tracing,
use 'dlfeval'.

I tried to use dlfeval instead of dlgradient but another error message appears:

Error using dlfeval (line 43)
Nested dlfeval calls are not supported. To compute higher derivatives, set
the 'EnableHigherDerivatives' option of the dlgradient function to true.

Does anyone got any idea why I recieve these error messages? THanks.

[Kingshuk]

Hello sir, I got the same for to train my encoder decoder (contain LSTM layer) model.

How to solve the issue, if you help me please.

Error using dlarray/dlgradient

'dlgradient' inputs must be traced dlarray objects or cell arrays, structures or tables containing traced

dlarray objects. To enable tracing, use 'dlfeval'.

This is the error which is coming while training

Answer 1

[MA]

It seems I solve it. The parameters needs to be dlarry as well as follows:

parameters.batchnorm1.Offset = initializeZeros([5 1]);
parameters.batchnorm1.Scale = initializeOnes([5 1]);
state.batchnorm1.TrainedMean = zeros(5,1,'single');
state.batchnorm1.TrainedVariance = ones(5,1,'single');
numClasses=2; 
parameters.fc1.Weights = initializeGlorot(sz,numOut,numIn);
parameters.fc1.Bias = initializeZeros([numClasses 1]);
[gradients,state,loss] = dlfeval(@modelGradients, parameters, dlX, dlY, state);

This way the program works. THanks.