How to choose number of hidden layers

Hi, I want to design a neural network with 3 input and 1 output. Samples are 21 Millions. For predictions and I don't know how many hidden layers and also the network parameters to use to get best results

Answer 1

[Walter Roberson]

When you call patternnet, pass it a vector of the sizes of the hidden layers.

https://www.mathworks.com/help/nnet/ref/patternnet.html

[Greg Heath]

          WHOOPS!! ATTENTION!!!

I moved MERYEME'S comment from an ANSWER box to a COMMENT box. I don't see it so I must have screwed it up.

                    SORRY!

The jist was: She is designing for regression, not pattern recognition.

Greg

[Greg Heath]

Walter's comment is valid for REGRESSION, CLASSIFICAITON, TIME-SERIES, etc

If you add FITNET to my suggested search you will see how to implement the double loop approach to minimize the number of hidden nodes for REGRESSION

                  ANSWERS 
 Hmin:dH:Hmax Ntrials FITNET     ==>  37 HITS
 Hmin:dH:Hmax Ntrials PATTERNNET ==>  28 HITS

the NEWSGROUP doesn't yield much with that detail

                  NEWSGROUP 
 Hmin  Hmax Ntrials FITNET     ==>  12 HITS
 Hmin  Hmax Ntrials PATTERNNET ==>   4 HITS

       Hmax Ntrials FITNET     ==>  12 HITS
       Hmax Ntrials PATTERNNET ==>   4 HITs

             Hmax  FITNET     ==>  14 HITS
             Hmax  PATTERNNET ==>   4 HITS

            Ntrials  FITNET     ==>  27 HITS
            Ntrials PATTERNNET ==>   15 HITS

Hope this helps.

Greg

Answer 2

[Greg Heath]

For 3 inputs and 1 output you only need 1 hidden layer. Minimize the number of hidden nodes subject to the maximum training error constraint.

 mse(errortrn) <= 0.01*mean(var(targettrn',1))

then the training error Rsquare is 99%.

I use a double loop approach over

 1. Number of hidden nodes h = Hmin:dH:Hmax

 2. j = 1:Ntrials number of random assignments for initial 
 weights and trn/val/tst data divisions.

 3. [ I N ] = size(input)% [ 3 N ]
    [ O N ] = size(target)% [ 1 N ]
    Ntrn ~ 0.7*N % default
    Ntrneq = Ntrn*O  % No of training equations
    Nw  = (I+1)*H+(H+1)*O = 5*H + 1 % No. of unknown weights

 4. A reasonable choice to prevent overtraining an overfit net is 

    Ntrneq >= 10*Nw   ==>    Ntrn >~ 50*H

 5. Obviously, a major task is to find a small ( << 21 millions!) 
 subset of data that can adequately represent the 21 million 3-dimensional 
 inputs. My experience is that for d-dimensional Gaussian distributions 
 Ntst >= 30*d  random picks is usually sufficient.

Taking Nval = Ntst ~ 30*d and

       Ntrn ~ (7/3)*(Nval+Ntst) ~ 140*d

Then, N ~ Ntrn + Nval + Ntst ~ 200*d = 600

 6.  There are zillions of examples of my double loop minimum H approach in 
both the NEWSGROUP and ANSWERS.  Just search on

           Hmin:dH:Hmax   Ntrials

Hope this helps.

Thank you for formally accepting my answer

Greg

[Meryeme BOUMAHDI]

I'm beginner with neuronal network so I work with GUI tool with default parameters with 1 hidden layer but the training error still high

[Walter Roberson]

The GUI tool is not always flexible enough. You can have the GUI tool create a network with the default number of hidden layers, and then you can tell it to generate the code for the network. You can then edit the code for the network so that it initializes the sizes of the hidden layers the way you want.