General questions about neural networks for data classification

Hi there,

I have a few general questions about data classification using neural networks.

I've been "playing around" with different networks such lvq and the nprtool, without really knowing what is the best practice and understanding what is actually happening.

I have 3000 input vectors that I can use for training. They are 27x1 matrices. And I have 9 target vectors.

A few general questions that I have:

1. What is the best network to use generally for data classification, lvq, the nprtool or some other network that I'm not aware of?

2. Is it necessary to have the same number of input vectors per target during training like 333 input vectors for each target (333x9=2997)? Or would having for example 5 times more input vectors of one target result in a bad network?

2. Is 9 target vectors too many?

3. What is the rule of thumb for the number of hidden neurons? Roughly how many should I have?

4. Should I use a particular type of learning function?

5. The 27 inputs are measurements, some with different units, I normalized all inputs between 0 and 1, is that a good thing to do?

I would appreciate any comments and suggestions as I have not been able to find these answers online.

Many thanks

John

Answer 1

[Thijs]

ugh wrote virtually an entire essay then accidentally deleted it.. So here a shorter and possibly less flippant version:

1 using lvq, or fuzzyfying the inputs can improve generelisation. but depending on your application is probably not necessary

2. it may cause suppression of the underrepresented classification targets. is 9 too much, that depends on your data set

3. in principle a single hidden layer is sufficient to approximate any function, given enough nodes. I've read that the funneling the data through 3 hidden layers with the middle one smaller than the outer ones (e.g 5 3 5) can help with strongly non linear data. in my personal experience having more hidden layers resulted in faster convergence.

4.yes backwards propagation. it is not hard too implement, it is essentially a gradient ascent algorithm

5. cant hurt i think, and i can think of situations where not doing so could hurt :). provided you know that your domain is bounded.

neural nets converge to a local optimum, so changing the starting conditions can influence the end result. this may or may not be a problem depending really on your particular situation. one approach which has worked well is to construct several networks to do the same thing and to simply average the result. generally speaking though 'playing around' is the best way forward