Train an SVM classifier and intentionally cause the solver to fail to converge onto a solution. Then resume training the classifier without having to restart the entire learning process. Compare values of the resubstitution loss for the partially trained classifier and the fully trained classifier.
Load the ionosphere
data set.
Train an SVM classifier. Specify that the optimization routine uses at most 100 iterations. Monitor the algorithm specifying that the software prints diagnostic information every 50
iterations.
|===================================================================================================================================|
| Iteration | Set | Set Size | Feasibility | Delta | KKT | Number of | Objective | Constraint |
| | | | Gap | Gradient | Violation | Supp. Vec. | | Violation |
|===================================================================================================================================|
| 0 |active| 351 | 9.971591e-01 | 2.000000e+00 | 1.000000e+00 | 0 | 0.000000e+00 | 0.000000e+00 |
| 50 |active| 351 | 8.064425e-01 | 3.736929e+00 | 2.161317e+00 | 60 | -3.628863e+01 | 1.110223e-16 |
SVM optimization did not converge to the required tolerance.
The software prints an iterative display to the Command Window. The printout indicates that the optimization routine has not converged onto a solution.
Estimate the resubstitution loss of the partially trained SVM classifier.
The training sample misclassification error is approximately 12%.
Resume training the classifier for another 1500
iterations. Specify that the software print diagnostic information every 250
iterations.
|===================================================================================================================================|
| Iteration | Set | Set Size | Feasibility | Delta | KKT | Number of | Objective | Constraint |
| | | | Gap | Gradient | Violation | Supp. Vec. | | Violation |
|===================================================================================================================================|
| 250 |active| 351 | 1.137406e-01 | 1.688486e+00 | 1.064098e+00 | 100 | -7.654307e+01 | 1.477984e-15 |
| 500 |active| 351 | 2.458986e-03 | 8.900780e-02 | 5.353919e-02 | 103 | -7.819650e+01 | 1.570792e-15 |
| 750 |active| 351 | 6.861149e-04 | 2.041818e-02 | 1.045385e-02 | 103 | -7.820930e+01 | 1.499668e-15 |
| 1000 |active| 351 | 5.992844e-05 | 1.878806e-03 | 1.095583e-03 | 103 | -7.820958e+01 | 1.606354e-15 |
| 1072 |active| 351 | 3.992245e-05 | 9.877142e-04 | 5.324559e-04 | 103 | -7.820959e+01 | 1.823194e-15 |
Exiting Active Set upon convergence due to DeltaGradient.
UpdatedSVMModel =
ClassificationSVM
ResponseName: 'Y'
CategoricalPredictors: []
ClassNames: {'b' 'g'}
ScoreTransform: 'none'
NumObservations: 351
Alpha: [103x1 double]
Bias: -3.8829
KernelParameters: [1x1 struct]
BoxConstraints: [351x1 double]
ConvergenceInfo: [1x1 struct]
IsSupportVector: [351x1 logical]
Solver: 'SMO'
The software resumes at iteration 1000
and uses the same verbosity level as the one set when you trained the model using fitcsvm
. The printout indicates that the algorithm converged. Therefore, UpdatedSVMModel
is a fully trained ClassificationSVM
classifier.
The training sample misclassification error of the fully trained classifier is approximately 8%.