Why the seed number of rng cause different results of accuracy in neural network?

Dear respected MATLAB users,

I am currently developing a regression model using neural network. To do that, I'm using a loop to select the best network with the lowest value of mse. While writing the code, I notice that the number of seed for rng plays an important role in determining the R2 value of trained and test data. As such when i put rng(0), the R2 value is lower compared to when i use rng(19). I already read the definition for rng but I still could not understand why the difference exist? Hope you can help me. Below is the part of coding I use in MATLAB.

x=input(1:35,:);
t=output(1:35,:);
[I N]=size(x) % I=4, N=35
[O N]=size(t) %O=1, N=35
trainFcn ='trainlm';
hiddenLayerSize=3;
net=fitnet(hiddenLayerSize,trainFcn);
net.performFcn='mse';
net.layers{1}.transferFcn ='tansig';
net.layers{2}.transferFcn='purelin';%change transfer function output
net.divideFcn='dividetrain';
rng(19)
numNN=20;
NN=cell(1,numNN);
perfs=zeros(1,numNN); % To make array with zero
for i=1:numNN
    fprintf('Training %d/%d\n', i, numNN);
    net = configure(net,x,t);
    [NN{i} tr] = train(net, x, t);
    y=NN{i}(x);
    perfs(i)=(mse(net,t,y));
end

Results

The best R2 from 20 trials as follows:

when rng(0), R2=0.74

when rng(19), R2= 0.93

[Bharath Attoti]

Interesting, shows how variable the performance can be with the random initial weights; not reliable, I suppose.

Answer 1

[Walter Roberson]

Neural Networks can only find "the" solution (best theoretical possible separation) under a restricted set of conditions. You should not think of them as being deterministic.

Neural Networks are function approximators . They are doing fitting of a function to the data, and the model they are using is very likely to have a lot of local minima. As such, the result is going to depend heavily on the initial conditions.

train() initializes the weights of the layers randomly, and then proceeds to (effectively) do a parameter fitting with that set of initial conditions. And then it initializes the weights differently (randomly) and proceeds to fit with those conditions. And so on, with a number of different initial conditions. The "winner" is the set of initial conditions that has the best results on the test data.

But all of that depends upon using the random number generator to initialize the weights. Which is why the setting of the random number seed is important: if you rng() a particular seed then it will always use the same set of random weights in the training and will produce the same results.

[Nurliana Farhana Salehuddin]

Thank you Walter for the answer. I think I am clear with its function now. This is what I conclude or summarize.

Setting up the random number seed influences weight initialization. Different number of seeds will give you different value of weight. If the seed is not set, then random seed or initial weight will be picked automatically and the results will be different when we restart the MATLAB. Overall, setting up the seed using rng command will leads to repeatable results.

Answer 2

[KSSV]

rng sets the seed of the random numbers. When you gace same seed to rng, the random numbers will always be same.

for i = 1:10
    rng(1)
    rand
end

When you change seed value, a different random numbers will be generated. So when you fix seed as 19, the initial random weights initialized might be close and giving you the good prediction.

You may set rng to different then 19 and increase the iterations. This might take you close to target even for other values then 19.

[Nurliana Farhana Salehuddin]

Dear KSSV, thank you for your answer. As you suggested, I already tried with different seed and increase the iterations. It also gives out good prediction. We use tr = tr to check the structure of our trained model. Do you know how to check the value of initial weight and bias used?