Problem with Neural Networks

Hi all!!

I'm new with Matlab, and i've got a problem with the parameters of my neural network. In my case, i have 1000 inputs of size 27. My output is a boolean.

- How to choose the number of hidden layers, and the number of neurons in my layers?

- How to choose the training function?

I built a neural network with 6 neurons on each of my 3 hidden layers (at random :p). I launched the training with the 'trainlm' function. But when the training is finished (after 3 iterations??), i've got weird values: mu = 10^100, grad = 10^-31, and my results are around 0.5 (whereas it must be a boolean ...)

From where does the problem come ?

Thanks all ^^

PS: here is a part of my code:

hiddenSizes = [6 6 6];
trainFcn = 'trainlm';
net = feedforwardnet(hiddenSizes,trainFcn);
net.trainParam.epochs = 10000;
net.trainParam.min_grad = 0;
net.trainParam.goal = 0;
net.trainParam.mu_max = 10^100;
net.trainParam.mu = 0.001;
net.trainParam.max_fail = 100;
net.layers{:}.transferFcn = 'logsig';
net2 = train(net,inputs,targets);
outputs = net2(inputs);

[Mohammad Al-Malki]

The design of ANN for identification or control purposes involves some trial and error. Generally speaking, any estimation of any nonlinear function can be handled by (3) layers ANN but in some cases additional hidden layer may improve the outcome of the design. Regarding the number of neurons in the input and output layers are governed by the dimension of the problem at hand. However, the number of neurons in the hidden layer can be adjusted after some training sessions. Adding mpre neurons normally improves the capability of the ANN in producing better results. However, and by reading your problem baove, it seems the problem may be attributed to the data set you used to traiin the ANN. So check that the data set is really rich enough and has no issues with it. From expperience with ANN, if you clean the data by removing the noise from it (if it is measurments), produces more sharp and impressive results. You may check some of the code in the file-exchange in Matlab cenetral or some references such as ANN Toolbox or the book by M. Hagan. Good Luck

Answer 1

[Greg Heath]

1. One hidden layer is sufficient

2. Find the smallest suitable no. of hidden nodes by trial and error

3. Use fitnet,tansig, purelin and trainlm for regression and curvefitting

4. Use patternnet, tansig, softmax or logsig and trainscg for classification and pattern recognition

5. To obtain a boolean output with softmax or logsig, use round.

6. Use as many defaults as possible (e.g., net.trainParam.*)

7. First try the documentation examples and demos that are the closest to your problem. Then search the newsgroup and ansers for similar problems that were solved.

Hope this helps

Greg

Answer 2

[AlfredLeMark]

Thank you very much!! My problem came from the number of hidden layers, I tested with hiddenSizes = [5 5], trainscg, patternnet and without logsig for classification, and it works!!

Thanks again! :D

[Greg Heath]

Repeat: A single hidden layer is sufficient.