How to obtain a ROC curve through cross validation on the out of fold data in cross validation?

조회 수: 22 (최근 30일)
I am using fitcsvm and need to obtain ROC curve for the fold that is not used in training.
Here is the code:
classificationSVM = fitcsvm(...
predictors, ...
response, ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'BoxConstraint', 1, ...
'Standardize', true, ...
'ClassNames', categorical(classnames_fitcsvm),'KFold',10);
[fitPosteriorSVM,ScoreTransform] = fitPosterior(classificationSVM);
for the prediction step:
I can not use resubPredict because that is set for training data and when I use kfoldPredict I recive error. I need to find the predicted score for the fold that is not used for training in 10 fold cross validation to be able to run perfcurve.
Any idea?

답변 (1개)

Divya Gaddipati
Divya Gaddipati 2020년 12월 30일
Hi,
Currently, you have to split the data into training and validation manually to generate results on validation set.
To use 10-fold cross-validation, you can fit the model on 90% of the data, and compute results for the remaining 10% of data which was not used for fitting. You can then loop over each of the 10 subsets to plot the ROC curves for individual runs.
Here is a small example on how to do that:
k = 10;
cvFolds = crossvalind('Kfold', labels, k);
for i = 1:k
testIdx = (cvFolds == i); % get indices of test instances
trainIdx = ~testIdx; % get indices training instances
classificationSVM = fitcsvm(data_features(trainIdx,:), labels(trainIdx), ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'BoxConstraint', 1, ...
'Standardize', true);
[predictions, score] = predict(svmModel, data_features(testIdx,:));
[X,Y] = perfcurve(labels(trainIdx), predictions,'versicolor');
plot(X,Y)
xlabel('False positive rate'); ylabel('True positive rate')
title('ROC curve')
end
Hope this helps!

카테고리

Help CenterFile Exchange에서 Statistics and Machine Learning Toolbox에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by