Mean and Standard Deviation of outputs on a neural network

I am trying to train a Bayesian neural network, and with 5 inputs and 4 outputs. In the end, I want to have a mean prediction for all the outputs and a estimate of the standard deviation. When I ran the following code, it says that the network must have an output layer. I am wondering whats incorrect. I have followed this example.

PS: I have updated the code and am using trainnet now. The error that comes now is that :Fixed sequence length training option is not supported: in Trainnet.

My utrain_load has a size 5*78739 and ytrain_load has a size 4*78739.

addpath('~\MATLAB\Examples\R2024a\nnet\TrainBayesianNeuralNetworkUsingBayesByBackpropExample')

Warning: Name is nonexistent or not a directory: /users/mss.system.wjdqm/~\MATLAB\Examples\R2024a\nnet\TrainBayesianNeuralNetworkUsingBayesByBackpropExample

numHiddenUnits1= 8;
numResponses = 4; % y1 y2 y3 y4
featureDimension = 5; % u1 u2 u3 u4 u5 % 
% featureDimension = 4; % u1 u2 u3 u4 u5
maxEpochs = 100; % IMPORTANT PARAMETER
miniBatchSize = 512; % IMPORTANT PARAMETER

% architecture
Networklayer_h2df = [...
    sequenceInputLayer(featureDimension)
    fullyConnectedLayer(4*numHiddenUnits1)
    reluLayer
    bayesFullyConnectedLayer(4*numHiddenUnits1,Sigma1=1,Sigma2=0.5)
    reluLayer
    fullyConnectedLayer(8*numHiddenUnits1)
    reluLayer
    gruLayer(LSTMStateNum,'OutputMode','sequence',InputWeightsInitializer='he',RecurrentWeightsInitializer='he')
    fullyConnectedLayer(8*numHiddenUnits1)
    reluLayer
    fullyConnectedLayer(4*numHiddenUnits1)
    reluLayer
    fullyConnectedLayer(numResponses)
    bayesFullyConnectedLayer(numResponses,Sigma1=1,Sigma2=0.5)
    ];

'bayesFullyConnectedLayer' is used in Train Bayesian Neural Network.

% training options
options_tr = trainingOptions('adam', ...
    'MaxEpochs',maxEpochs, ...
    'MiniBatchSize',miniBatchSize, ...
    'GradientThreshold',1, ...
    'SequenceLength',8192,...
    'Shuffle','once', ...
    'Plots','training-progress',...
    'Verbose',1, ...
    'VerboseFrequency',64,...
    'LearnRateSchedule','piecewise',...
    'LearnRateDropPeriod',250,...
    'LearnRateDropFactor',0.75,...
    'L2Regularization',0.1,...
    'ValidationFrequency',10,...
    'InitialLearnRate', 0.001,...
    'Verbose', false, ...
    'ExecutionEnvironment', 'cpu', ...
    'ValidationData',[{uval_load} {yval_load}],...
    'OutputNetwork','best-validation-loss');


%% training and Saving model data
savename = [sprintf('%04d',MP),'_',sprintf('%04d',numHiddenUnits1),'_',sprintf('%04d',LSTMStateNum),'_',sprintf('%04d',trainingrun),'.mat'];

if do_training == true
    tic
    [h2df_model, h2df_model_infor] = trainnet(utrain_load,ytrain_load,Networklayer_h2df,"mse",options_tr);
    toc
    ElapsedTime = toc;
    h2df_model_analysis = analyzeNetwork(h2df_model); % analysis including total number of learnable parameters
    h2df_model_infor.ElapsedTime = ElapsedTime;

    save(['../Results/h2df_model_',savename],"h2df_model")
    save(['../Results/h2df_model_info_',savename],"h2df_model_infor")
    save(['../Results/h2df_model_analysis_',savename],"h2df_model_analysis")
else
    load(['../Results/h2df_model_',savename])
    load(['../Results/h2df_model_info_',savename])
    load(['../Results/h2df_model_analysis_',savename])
end

[Aditya]

@Vasu Sharma Could you provide the exact error message and the code till the line where the error hits!

[Vasu Sharma]

@Aditya : Have updated the post with more info.

Answer 1

[Matt J]

You have shown the code for your network, but not how you have adapted the training code, nor the error messages. So, it calls for guesswork.

A fair guess however is that you are using the deprecated trainnetwork() function rather than trainnet() to do the training. With trainnetwork, the network must have an output layer, for example a regressionLayer or a classificationLayer which specifies the loss function to be used during training. The network in your code does not have such a layer.

[Vasu Sharma]

Hi @Matt J! Thanks for the response. I am still getting an error after updating to trainnet.:

Namely: Error using trainnet (line 46)

Fixed sequence length training option is not supported.

Could you kindly advice?

Best,

Vasu