Neural Network Output Problem

Hi,

I have a feedforward network as below:

 net = feedforwardnet([68 36], 'traingdm');
 net.numInputs = 2;
 net.inputs{1}.size = 1;
 net.inputs{2}.size = 136;
 net.layers{3}.size = 4;
 net.inputConnect = [0 1; 1 0; 0 0];
 net.trainparam.epochs = 775;
 net.trainparam.lr = 0.3;
 net.trainparam.mc = 0.3;
 net.trainparam.showCommandLine = 1;
 net.performFcn = 'mse';
 net.divideParam.trainRatio = 42.01/100;
 net.divideParam.valRatio = 20.95/100;
 net.divideParam.testRatio = 37.04/100;

I have an Inputs matrix (137x1002 double) and a Targets matrix (4x1002 double) that used for age estimation by neural network. 136 face feature + 1 gender = 137 input cell for each of 1002 face image. it must classify to 4 groups of ages:

 group 1 :  1 - 12
 group 2 : 13 - 25
 group 3 : 26 - 45
 group 4 : 46 - 63

The target matrix filled by 0-1 values

After training this network I checked network output values, but all of values was same and repeated...

Network Training Details :

network training stoped by Validation Stop event in epoch 16.

Performance Plot

Training State

Regression

Regression R Value is very low ... I don't know why?

My NN architecture and network initializing values explained in Age Estimation article that I was study before.

what is my problem? please help me!

Thanks.

[Sean de Wolski]

Congratulations on having the best written Neural Networks question ever!

Answer 1

[Greg Heath]

You should always run at least 10 trials for each candidate net. For example, if I am considering H = 0:2:20 hidden nodes, I tabulate the results in 10X11 matrices. You may have just started with a poor random choice of initial weights. Try more runs. Then consider changing the design.

Your design has Nw = (136+1)*68+ (1+68+1)*36 + (36+1)*4= 11,984 unknown weights to be estimated by Ntrneq = 42.01*1002*4 =168,380 training equations. The ratio of ~ 14 is OK.

However:

I see no reason for you not to use the simple default single-input/single-hidden-layer 137-H-4 configuration using PATTERNNET ... or am I missing something?

Another avenue to pursue is the reduction of the number of inputs. PLS may be more helpful than PCA for a classification problem.

Hope this helps.

Thank you for formally accepting my answer.

Greg

[Younes Jafari]

I trained my network more than 10 times, but the answer not changed.

I used PATTERNNET before and it hasn't this problem (same results in output), but has lot of mistakes in age estimation. So the article that I used to implementation, presents this neural network for age estimation by fg-net database.

explain me how can I tabulate results and use them in next train and what do you mean about random choice of initial weights?

[Greg Heath]

Each time the computer generates initial weights, they are chosen randomly. Therefore 10 sequential designs will result in 10 different solutions.

Search in ANSWERS and the NEWSGROUP on subsets of

heath close clear Neq Nw Ntrials

for code examples.

Hope this helps.

Geg

[Younes Jafari]

Thank you Greg!

[Younes Jafari]

Aah, sory Greg! I can't find :

heath close clear Neq Nw Ntrials

for code examples.

cause you post exactly these expressions to many of ANSWERS! You know? :)

So it is very hard to find. If you have a link please put it here...

I also have a tip : please use target links as these repeated expressions henceforward!

Thank you very very much.

[Greg Heath]

I should have just referred you to the NEWSGROUP using heath Ntrials.

However, to start, here is an ANSWERS example

Neural Network help

Asked by Amjad on 1 Apr 2012

Hope this is sufficient

Greg

[Younes Jafari]

Thanks again Greg :)

[Younes Jafari]

Link to mentioned post: (Neural Network help - Asked by Amjad on 1 Apr 2012) http://www.mathworks.nl/matlabcentral/answers/34206-neural-network-help

for everyone follows this Answer...