how do i calculate the accuracy of ANN?

how do i calculate the accuracy of ANN?

i have 10 structures of neural network. i want to find the best structure that can predict as close as possible to the actual value.i use trainlm. so, my question is,how can i choose the best structures other than mean absolute percentage error (MAPE)?

Answer 1

[Greg Heath]

So this is regression, not classification. Use fitnet and vary the number of hidden nodes. For each value of hidden nodes, design Ntrials nets that differ by the set of random initial weights.

For examples, search the NEWSGROUP and ANSWERS using

 greg fitnet Hub Ntrials

Use the training record tr to separate the trn/val/tst indices and performances.

Choose a performance index

MSE % scale dependent Useless unless compared with something

NMSE = MSE/mean(var(target',1)) % Normalized MSE; Typically 0 <= NMSE <= 1

R2 = 1-NMSE % fraction of target variance that is modeled by the net (Search Rsquare in Wikipedia)

Separately tabulate the trn/val/tst results. Choose the net that has the best validation performance. If there are several with performances that are not significantly different, choose one with the smallest number of hidden nodes.

Use the corresponding test performance to predict performance on unseen data if the net is deployed.

Searching the NEWSGROUP and ANSWERS with

 greg ntrials

will provide examples.

Thank you for formally accepting my answer.

Greg

Answer 2

[Greg Heath]

Insufficient info.

Are these classifiers? If so, are the class size ratios of the training data the same as that for unseen nontraining data?

There are four measures used for classifiers:

Bayesian Risk = sum(sum(Pi*Cij*pji)

 Pi  ith class prior probability

 pij posterior probability estimate for ith class when the input is from class j

 Cij misclassification cost (Cii=0)

ErrorRate = sum(Pi*Erri)

crossentropy1 = sum(ti*log(yi)) % Mutually exclusive classes

crossentropy2 = Entropy1 + sum((1-ti)*log(1-yi)% Nonmutually exclusive classes

Risk and crossentropy are best for design (Error rate is not continuous)

Error rate on nontraining data is best for selection.

[amir]

actually i have 4 input and 1 output. the inputs are cutting speed, feed rate, depth of cut and I-kaz coefficient and the outputs are flank wear measurement (mm units). so i use 10 structures of Neural network to predict flank wear. and i want to come out with one structure that capable to give me the most good prediction. so, the question is what parameter should i use to determine the best neural network structure?