find optimal hyperparameters in SVM

Hello

I'm trying to optimize a SVM model for my training data then predict the labels of new data with it. Also I must find SVM with best hyperparameter by using k-fold crossvalidation. TO do so I wrote the following code:

Mdl = fitcsvm(trainingData,labels,'OptimizeHyperparameters','auto',...

'HyperparameterOptimizationOptions',struct('Optimizer','gridsearch','AcquisitionFunctionName',...

'expected-improvement-per-second','MaxObjectiveEvaluation',10,'ShowPlots',false,'Verbose',0));

label = predict(Mdl,testData);

the problem is every time I ran this code and calculated the classification accuracy for test data I got different classification accuracy.

I should mention that when I train SVM without optimizing hyperparameters results are alaways the same. Is this mean every time I have diffierent hyperparameters? How can I solve this and obtain unique classification accuracy?

Answer 1

[Alan Weiss]

Read what the bayesopt documentation has to say about your chosen acquisition function: "Acquisition functions whose names include per-second do not yield reproducible results because the optimization depends on the runtime of the objective function. " In other words, choose the 'expected-improvement-plus' or 'expected-improvement' acquisition function for reproducibility.

Alan Weiss

MATLAB mathematical toolbox documentation

[khatere darvish]

thank you for your answer

I applied the changes in my program but I still have the same issue

[Alan Weiss]

The other possibility is to reset the random number stream before each optimization:

rng default % Or any seed you like

Alan Weiss

MATLAB mathematical toolbox documentation

[khatere darvish]

thank you it's working now