Differences between svmtrain and fitcsvm

I have a set of data composed of list of 35 features. I notice when I give the data to svmtrain I get the message:

no convergence achieved within maximum number of iterations

Than, when I increase the number if iteration " MaxIter " up to around 1,000,000 the above error disappear and I start getting good classification using " svmclassify ".

On the other hand, when I give the data to " fitcsvm " it converge quickly within the default number of iteration "15,000". However, the problem is when I try to classify the data using " predict ", I got wrong classification.

So in a nutshell, at last " svmtrain " classify the data correctly after increasing number of iteration. However," fitcsvm " neither classify the data correctly, nor it gives me the opportunity to increase number of iteration because it looks from checking the ConvergenceInfo.Converged property that it converge successfully.

Any advice please? notice I'm new to matlab and SVM.

Answer 1

[Ilya]

Look at the doc/help for fitcsvm or, alternatively look at the ConvergenceInfo property in the returned object. There are several tolerances. Pass low values of these tolerances to fitcsvm, say 1e-10. This usually ensures that optimization runs until the max number of iterations is met. You can then resume if desired.

Or try a different solver such as 'isda'. If you have a thousand observations or less, try 'l1qp' which dispatches to quadprog (this requires Optimization Toolbox installed).

[Diver]

There is 'GapTolerance' and 'DeltaGapTolerance, which one of these I need to use. Moroover, my data have around 50,0000 observations, could I use solver 'l1qp' with these?

[Ilya]

There is no DeltaGapTolerance - there is a DeltaGradientTolerance. Convergence is detected when any tolerance is satisfied, so set them all low. No, you can't use l1qp for so many observations.

[Diver]

I tried to set the parameters: - DeltaGradientTolerance. - GapTolerance - KKTTolerance - Gap

I notice when I set these values to value such as 1e-100 it return with Converged SVM (ConvergenceInfo.Converged =1). However, the result is un satisfying at all compared to SVMTRAIN.

When I decrease the parametrs to value such as 1e-400, it doesn't converge even if I resume it with 'Iterationlimit' up to 10,000,000

Is there any other parameters I could set? please advice.

[Ilya]

You could set lots of parameters. What are you trying to accomplish?

If you want to fit an accurate model, the box constraint and kernel scale for non-linear kernels are the two common tuning knobs. Also, try both SMO and ISDA solvers. The fact that optimization does not converge doesn't mean the model is not accurate.

If you are saying that no matter what you do, a model fitted by svmtrain is more accurate than the one fitted by fitcsvm, I'd like to take a look at your data. Can you post it somewhere?

Answer 2

[Victor Vergara]

I am having exactly the same problem. fitcsvm and fitclinear do not perform as well as svmtrain. I changed the convergence tolerances of fitcsvm and didn't work. I would like to see how was this solved.