Hyperparamter optimization - how to manually specify SVM kernel functions to try using optimizableVariable

I am following the example here to perform hyperparameter optimization by specifying possible candidate values of parameters:

https://www.mathworks.com/help/stats/bayesopt.html#bvamydy-2

Code I am running:

% Load dataset - ionoshpere
load ionosphere
dataX = X;
dataY = Y;
cvo = cvpartition(dataY, 'KFold', nFolds);

%     box = optimizableVariable('box', []);
kernel = optimizableVariable('kernel', {'gaussian', 'polynomial'}, 'Type', 'categorical');
kernelScale = optimizableVariable('kernelScale', [1, 30]);
polyOrder = optimizableVariable('polyOrder', [2, 3], 'Type', 'integer');

fun = @(x)svmfun(x, dataX, dataY, cvo);

results = bayesopt(fun, [kernel, kernelScale, polyOrder]);
    
function [objective] = svmfun(x, dataX, dataY, cvo)
    svmModel = fitcsvm(dataX, dataY, ...
        'BoxConstraint', 1, ...
        'KernelFunction', x.kernel, ...
        'KernelScale', x.kernelScale, ...
        'PolynomialOrder', x.polyOrder, ...
        'Standardize', true, ...
        'CVPartition', cvo, ...
        'ClassNames', [0, 1]);

    [label,  score] = kfoldPredict(svmModel);
    loss = kfoldLoss(svmModel);
    [~, ~, ~, aucRoc] = perfcurve(dataY, score(:,2), 1);
    [~, ~, ~, aucPrc] = perfcurve(dataY, score(:,2), 1, ...
        'xCrit', 'tpr', 'yCrit', 'prec');
    aucRocLoss = 1 - aucRoc;
    aucPrcLoss = 1 - aucPrc;

    objective = loss;
end

I get the following error:

Error using classreg.learning.modelparams.SVMParams.make
(line 225)
'KernelFunction' value must be a character vector or string
scalar.

Error in classreg.learning.FitTemplate/fillIfNeeded (line
660)
                    this.MakeModelParams(this.Type,this.MakeModelInputArgs{:});

Error in classreg.learning.FitTemplate.make (line 125)
            temp = fillIfNeeded(temp,type);

Error in classreg.learning.FitTemplate/fillIfNeeded (line
480)
                                classreg.learning.FitTemplate.make(this.Method,'type',this.Type,...

Error in classreg.learning.FitTemplate.make (line 125)
            temp = fillIfNeeded(temp,type);

Error in ClassificationSVM.template (line 235)
            temp =
            classreg.learning.FitTemplate.make('SVM','type','classification',varargin{:});

Error in ClassificationSVM.fit (line 239)
            temp = ClassificationSVM.template(varargin{:});

Error in fitcsvm (line 343)
    obj = ClassificationSVM.fit(X,Y,RemainingArgs{:});

Error in svmHyperparameterOptimization>svmfun (line 25)
    svmModel = fitcsvm(dataX, dataY, ...

Error in
svmHyperparameterOptimization>@(x)svmfun(x,dataX,dataY,cvo)
(line 13)
fun = @(x)svmfun(x, dataX, dataY, cvo);

Error in BayesianOptimization/callObjNormally (line 2553)
                        Objective =
                        this.ObjectiveFcn(conditionalizeX(this,
                        X));

Error in BayesianOptimization/callObjFcn (line 481)
                        = callObjNormally(this, X);

Error in BayesianOptimization/runSerial (line 1989)
                    ObjectiveFcnObjectiveEvaluationTime,
                    ObjectiveNargout] = callObjFcn(this,
                    this.XNext);

Error in BayesianOptimization/run (line 1941)
                this = runSerial(this);

Error in BayesianOptimization (line 457)
            this = run(this);

Error in bayesopt (line 323)
Results = BayesianOptimization(Options);

Error in svmHyperparameterOptimization (line 15)
results = bayesopt(fun, [kernel, kernelScale, polyOrder]);

I believe KernelFunction is eligible parameters for the hyperparameter tuning:https://www.mathworks.com/help/stats/fitcsvm.html#d120e288389

But I have no clue why it doesn't work. Any help will be greatly appreciated

Answer 1

[Don Mathis]

You need to write

'KernelFunction', string(x.kernel), ...

Answer 2

[Kani Mozhi]

This might help

https://researchprojects5489728.wordpress.com/matlab-code-for-hyper-parameter-optimization-of-svm-2/

Answer 3

[Kani Mozhi]

Here's a code for SVM parameter optimization using HHO algorithm - Code