train, test ,validation confusion matrix

while the a confusion matrix is a map of correct and incorrect classifications; what are train ,test,validation confusion matrices? what is their meaning

[Adam Danz]

This question is better for an internet search engine. There are lots of tutorials and videos out there. For example,

https://machinelearningmastery.com/confusion-matrix-machine-learning/

If you have a matlab related question, you're in the right place.

[Yogini Prabhu]

Thanks Adam, but I had seen the given web-page earlier. I wanted to know meaning about individual confusin matrices; train ,test ,validate

Answer 1

[Shubham Rawat]

Hi Yogini,

Confusion Matrices:

These are to evaluate the quality of the output of a classifier on the data set. The diagonal elements represent the number of points for which the predicted label is equal to the true label, while off-diagonal elements are those that are mislabeled by the classifier. The higher the diagonal values of the confusion matrix the better, indicating many correct predictions.

Train, Test , Validation Confusion matrices:

They uses different data for creating confusion matrix. For train confusion matrix it uses predicted values and actual values from train data. Similarly for the other confusion matrices.

You may also refer to the answer to this question:

How to display Confusion matrix of Testing ,Training and Validation without using nprtool - MATLAB Answers - MATLAB Central (mathworks.com)

Hope this helps!

[Yogini Prabhu]

Shubham, Thanks for the reply !

[Yogini Prabhu]

Hi Shubham, ay I request you to explain the same with the help of cancer dataset , which is sample dataset given by nprtool ?

[Yogini Prabhu]

Please let know.

[Shubham Rawat]

Hi Yogini,

Here is the code for this using Cancer dataset:

load cancer_dataset.mat
inputs = cancerInputs;
targets = cancerTargets;

% Create a Pattern Recognition Network
hiddenLayerSize = 10;
net = patternnet(hiddenLayerSize);

% Set up Division of Data for Training, Validation, Testing
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
 
% Train the Network
[net,tr] = train(net,inputs,targets);

%plot confusion matrix for training
yTrn = net(inputs(:,tr.trainInd));
tTrn = targets(:,tr.trainInd);
figure, plotconfusion(tTrn,yTrn,'Training');

%plot confusion matrix for validation
yVal = net(inputs(:,tr.valInd));
tVal = targets(:,tr.valInd);
figure, plotconfusion(tVal,yVal,'Valdation');

%plot confusion matrix for testing
yTst = net(inputs(:,tr.testInd));
tTst = targets(:,tr.testInd);
figure, plotconfusion(tTst,yTst,'Testing');

Hope this Helps!

[Yogini Prabhu]

'The diagonal elements represent the number of points for which the predicted label is equal to the true label, while off-diagonal elements are those that are mislabeled by the classifier.'

which is the prediction, and how is it implemented?

Answer 2

[Yogini Prabhu]

okay. i wil try out that, how about ROcs of the same? can i get an example of a confusion matrix (with not classification rate of 100% )and its corresponding ROC.