Data classification with neural networks - how to improve existing network.

Hi everyone, I would try to keep this short. I have just started learning about NN, byt I have some experience with Matlab. I am working on a project, which purpose is to classify some sets of data. My first assigment is to improve performance of simple feedforward netowrk with backpropagation, however I don't get any results.

I have 3 input neurons, which contains some information from data (only numbers, one hidden layer with tansigmoidal transfer function, and 2 outputs (also with tansigmoidal TF), which are in range [-1,1] so I use them to plot points.

At first I had almost 10 desirable states on plot, where I was hoping to see how data is classified and being grouped in the vicinity of those points. Almost 90% of my data is suppose to be classified as first state, but because the quantity, all I see is thousands of points all over the plot, when the number is smaller for diffrent state, results are slightly better, but still scatter is noticeable.

After some thought I decided to simplify my project and see, what the results would be if I dovide the problem for only 2 states, so my most numerous state is intact (let's call it State 1) and all others are combined (State 2). I prepared simple plot in 1D, State 1 should get value "-1", and State 2 "1". After that training session I prepare small feedback - how many (in %) of the outputs have learned to be either "-1" or "1". Plot showed that points are again scattered all over the range, and accuracy is weak, max. 40% for State 1, and State 2 - max. 25%.

Of course I can't expect any exact soultion, because every problem in NN is diffrent and requires diffrent approach. All I would like to ask is to show me some pointers, where could I get some useful information, how should I approach when setting parameters for hidden layers (should I use 1, 2 or more? and which transfer funciton may be useful?) ? Maybe there is some community, which specialize in NN in Matlab environment?

Thank you in advance.

Answer 1

[Greg Heath]

N0 = sample size

c = number of classes (states)

I = dimensionality of input vectors

O = c = dimensionality of output vectors

x = input matrix

t = target output matrix

[I N0] = size(x)

[O N0] = size(t)

The columns of t are columns of the c-dimensional unit matrix eye(c) so that vec2ind(t) yields the class indices.

If the number in each class, N0i (i=1:c, N0 = sum(N0i)) is not approximately equal, add di duplicates to each class so that N0i is increased to Ni = N0i+di = N/c where N= N0+sum(di).

Use PURELIN, LOGSIG OR SOFTMAX as the activation function for the output layer and consider the outputs as estimates of the class posterior probabilities, conditional on the input. Assign the input to the class corresponding to the largest posterior.

Hope this helps.

Greg