Neural Network - Pattern recognition network

Hello, I have NN for Pattern recognition. I use the function "patternnet". My question is, if I train my net with 3 different signals/classes, and then try to test it with a fourth one, to my understanding it will always try to give me the "closest one"...while I'd expect the net will tell me, such class doesn't exist in the data base...

Can this be done somehow?

[Greg Heath]

Moved to an answer.

[moose]

Thank you Greg. I am quite new for NN so if possible I'd like to ask for some more clarification.

To my understanding, at the moment for a certain input file, the net gives me a "probability"...something like:

The input file match with

• 50% to target A
• 35% to target B
• 15% to target C

and you're suggesting to make a condition that if I don't have a significant value of more than, let's say, 75%, I can "declare" for this input as "doesn't exist in the data base"... right?

BUT, my question is, I mean, I would expect, that for an input, that doesn't exist in the data base all the probabilities (for targets A-C) will be zero, or at least something very close to it...

Please let me know if I made my self more clear this time.

Thanks again!

[Greg Heath]

You are clear. However, you are wrong.

If the target columns are {0,1} unit vectors, indicating distinct classes, the output columns will also be unit vectors if you use the default softmax output transfer function. There is no telling what the result of an input from a vector that does not belong to any of the specified classes will be.

If the output transfer function is logsig, the outputs are not constrained to a unit sum. However, if you divide the outputs by the column sum to create a unit vector, the results are statistically consistent estimates of the input-conditional posterior probability.

So if you expect to encounter "innapropriate" data, do the following:

For each class, plot the two, probably overlapping, output histograms of that class and 'not that class'. Then you can see how that classes error rate varies as a threshold is changed.

If your data set is large enough, use training and validation data to set the thresholds. Then you can get unbiased estimates of error rates using the test set.

If your data set is not large enough, try f-fold cross-validation and final results are obtained by averaging over the f results.

Hope this helps.

Greg

Answer 1

[Greg Heath]

 You have to set thresholds on the estimated probability outputs.
 The target vectors are columns from eye(3)
 The 3 dimensional output vector probability estimates should have coordinates in [0,1] and sum to 1
 The assignment is made to the class corresponding to the highest output.
 If you want a noclassification category only make classification assignments when the highest output exceeds a certain threshold.
 The classification thresholds are determined from the training data. If necessary, they can be adjusted using the validation data.

 The sum of the outputs is always unity.

help patternnet

     [x,t] = iris_dataset;
     net = patternnet(10);
     net = train(net,x,t);
     view(net)
     y = net(x);
     perf = perform(net,t,y);
     classes = vec2ind(y);
     minmax(sum(y))