predict

Classify observations using support vector machine (SVM) classifier

Syntax

label = predict(SVMModel,X)

[label,score] = predict(SVMModel,X)

Description

label = predict(SVMModel,X) returns a vector of predicted class labels for the predictor data in the table or matrix X, based on the trained support vector machine (SVM) classification model SVMModel. The trained SVM model can either be full or compact.

[label,score] = predict(SVMModel,X) also returns a matrix of scores (score) indicating the likelihood that a label comes from a particular class. For SVM, likelihood measures are either classification scores or class posterior probabilities. For each observation in X, the predicted class label corresponds to the maximum score among all classes.

Examples

Label Test Sample Observations of SVM Classifiers

Load the ionosphere data set.

load ionosphere
rng(1); % For reproducibility

Train an SVM classifier. Specify a 15% holdout sample for testing, standardize the data, and specify that 'g' is the positive class.

CVSVMModel = fitcsvm(X,Y,'Holdout',0.15,'ClassNames',{'b','g'},...
    'Standardize',true);
CompactSVMModel = CVSVMModel.Trained{1}; % Extract trained, compact classifier
testInds = test(CVSVMModel.Partition);   % Extract the test indices
XTest = X(testInds,:);
YTest = Y(testInds,:);

CVSVMModel is a ClassificationPartitionedModel classifier. It contains the property Trained, which is a 1-by-1 cell array holding a CompactClassificationSVM classifier that the software trained using the training set.

Label the test sample observations. Display the results for the first 10 observations in the test sample.

[label,score] = predict(CompactSVMModel,XTest);
table(YTest(1:10),label(1:10),score(1:10,2),'VariableNames',...
    {'TrueLabel','PredictedLabel','Score'})

ans=10×3 table
    TrueLabel    PredictedLabel     Score  
    _________    ______________    ________

      {'b'}          {'b'}          -1.7175
      {'g'}          {'g'}           2.0003
      {'b'}          {'b'}          -9.6836
      {'g'}          {'g'}           2.5616
      {'b'}          {'b'}           -1.548
      {'g'}          {'g'}           2.0983
      {'b'}          {'b'}          -2.7016
      {'b'}          {'b'}         -0.66331
      {'g'}          {'g'}           1.6047
      {'g'}          {'g'}            1.773

Predict Labels and Posterior Probabilities of SVM Classifiers

Label new observations using an SVM classifier.

Load the ionosphere data set. Assume that the last 10 observations become available after you train the SVM classifier.

load ionosphere
rng(1); % For reproducibility
n = size(X,1);       % Training sample size 
isInds = 1:(n-10);   % In-sample indices 
oosInds = (n-9):n;   % Out-of-sample indices

Train an SVM classifier. Standardize the data and specify that 'g' is the positive class. Conserve memory by reducing the size of the trained SVM classifier.

SVMModel = fitcsvm(X(isInds,:),Y(isInds),'Standardize',true,...
    'ClassNames',{'b','g'});
CompactSVMModel = compact(SVMModel);
whos('SVMModel','CompactSVMModel')

  Name                 Size             Bytes  Class                                                 Attributes

  CompactSVMModel      1x1              30173  classreg.learning.classif.CompactClassificationSVM              
  SVMModel             1x1             136713  ClassificationSVM                                               

The CompactClassificationSVM classifier (CompactSVMModel) uses less space than the ClassificationSVM classifier (SVMModel) because SVMModel stores the data.

Estimate the optimal score-to-posterior-probability transformation function.

CompactSVMModel = fitPosterior(CompactSVMModel,...
    X(isInds,:),Y(isInds))

CompactSVMModel = 
  CompactClassificationSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: {'b'  'g'}
           ScoreTransform: '@(S)sigmoid(S,-1.968371e+00,3.122227e-01)'
                    Alpha: [88x1 double]
                     Bias: -0.2142
         KernelParameters: [1x1 struct]
                       Mu: [0.8886 0 0.6365 0.0457 0.5933 0.1200 0.5414 0.1217 0.5020 0.1872 0.4659 0.1596 0.3889 0.0970 0.3308 0.0723 0.3685 -0.0039 0.3453 -0.0256 0.3231 0.0097 0.3490 -0.0596 0.3839 -0.0731 0.5343 -0.0718 0.3659 ... ] (1x34 double)
                    Sigma: [0.3151 0 0.5032 0.4476 0.5251 0.4668 0.4966 0.5275 0.5107 0.4896 0.5681 0.5011 0.6267 0.5009 0.6569 0.4639 0.6214 0.5035 0.6296 0.5265 0.6128 0.5250 0.6067 0.5349 0.5817 0.5157 0.5212 0.5577 0.5787 0.5151 ... ] (1x34 double)
           SupportVectors: [88x34 double]
      SupportVectorLabels: [88x1 double]

The optimal score transformation function (CompactSVMModel.ScoreTransform) is the sigmoid function because the classes are inseparable.

Predict the out-of-sample labels and positive class posterior probabilities. Because true labels are available, compare them with the predicted labels.

[labels,PostProbs] = predict(CompactSVMModel,X(oosInds,:));
table(Y(oosInds),labels,PostProbs(:,2),'VariableNames',...
    {'TrueLabels','PredictedLabels','PosClassPosterior'})

ans=10×3 table
    TrueLabels    PredictedLabels    PosClassPosterior
    __________    _______________    _________________

      {'g'}            {'g'}              0.98418     
      {'g'}            {'g'}              0.95545     
      {'g'}            {'g'}              0.67793     
      {'g'}            {'g'}              0.94447     
      {'g'}            {'g'}              0.98744     
      {'g'}            {'g'}               0.9248     
      {'g'}            {'g'}               0.9711     
      {'g'}            {'g'}              0.96986     
      {'g'}            {'g'}              0.97802     
      {'g'}            {'g'}               0.9436     

PostProbs is a 10-by-2 matrix, where the first column is the negative class posterior probabilities, and the second column is the positive class posterior probabilities corresponding to the new observations.

Input Arguments

SVMModel — SVM classification model

ClassificationSVM model object | CompactClassificationSVM model object

SVM classification model, specified as a ClassificationSVM model object or CompactClassificationSVM model object returned by fitcsvm or compact, respectively.

X — Predictor data to be classified

numeric matrix | table

Predictor data to be classified, specified as a numeric matrix or table.

Each row of X corresponds to one observation, and each column corresponds to one variable.

• For a numeric matrix:

  • The variables in the columns of X must have the same order as the predictor variables that trained SVMModel.

  • If you trained SVMModel using a table (for example, Tbl) and Tbl contains all numeric predictor variables, then X can be a numeric matrix. To treat numeric predictors in Tbl as categorical during training, identify categorical predictors by using the CategoricalPredictors name-value pair argument of fitcsvm. If Tbl contains heterogeneous predictor variables (for example, numeric and categorical data types) and X is a numeric matrix, then predict throws an error.

• For a table:

  • predict does not support multicolumn variables or cell arrays other than cell arrays of character vectors.

  • If you trained SVMModel using a table (for example, Tbl), then all predictor variables in X must have the same variable names and data types as those that trained SVMModel (stored in SVMModel.PredictorNames). However, the column order of X does not need to correspond to the column order of Tbl. Also, Tbl and X can contain additional variables (response variables, observation weights, and so on), but predict ignores them.

  • If you trained SVMModel using a numeric matrix, then the predictor names in SVMModel.PredictorNames and corresponding predictor variable names in X must be the same. To specify predictor names during training, see the PredictorNames name-value pair argument of fitcsvm. All predictor variables in X must be numeric vectors. X can contain additional variables (response variables, observation weights, and so on), but predict ignores them.

If you set 'Standardize',true in fitcsvm to train SVMModel, then the software standardizes the columns of X using the corresponding means in SVMModel.Mu and the standard deviations in SVMModel.Sigma.

Data Types: table | double | single

Output Arguments

label — Predicted class labels

categorical array | character array | logical vector | numeric vector | cell array of character vectors

Predicted class labels, returned as a categorical or character array, logical or numeric vector, or cell array of character vectors.

label has the same data type as the observed class labels (Y) that trained SVMModel, and its length is equal to the number of rows in X. (The software treats string arrays as cell arrays of character vectors.)

The predict function classifies an observation into the class yielding the highest score. For an observation with NaN scores, the function classifies the observation into the majority class, which makes up the largest proportion of the training labels.

For one-class learning, each value in label is the same—the one class in the training data. Use score to identify anomalies.

score — Predicted class scores or posterior probabilities

numeric column vector | numeric matrix

Predicted class scores or posterior probabilities, returned as a numeric column vector or numeric matrix.

• For one-class learning, score is a column vector with the same number of rows as the observations (X). The elements of score are anomaly scores for the corresponding observations. Negative score values indicate that the corresponding observations are outliers. You cannot obtain posterior probabilities for one-class learning.

• For two-class learning, score is a two-column matrix with the same number of rows as X.

  • If you fit the optimal score-to-posterior-probability transformation function using fitPosterior or fitSVMPosterior, then score contains class posterior probabilities. That is, if the value of SVMModel.ScoreTransform is not none, then the first and second columns of score contain the negative class (SVMModel.ClassNames{1}) and positive class (SVMModel.ClassNames{2}) posterior probabilities for the corresponding observations, respectively.

  • Otherwise, the first column contains the negative class scores and the second column contains the positive class scores for the corresponding observations.

If SVMModel.KernelParameters.Function is 'linear', then the classification score for the observation x is

$$f\left(x\right)=\left(x/s\right)\prime \beta +b.$$

SVMModel stores β, b, and s in the properties Beta, Bias, and KernelParameters.Scale, respectively.

To estimate classification scores manually, you must first apply any transformations to the predictor data that were applied during training. Specifically, if you specify 'Standardize',true when using fitcsvm, then you must standardize the predictor data manually by using the mean SVMModel.Mu and standard deviation SVMModel.Sigma, and then divide the result by the kernel scale in SVMModel.KernelParameters.Scale.

All SVM functions, such as resubPredict and predict, apply any required transformation before estimation.

If SVMModel.KernelParameters.Function is not 'linear', then Beta is empty ([]).

More About

Classification Score

The SVM classification score for classifying observation x is the signed distance from x to the decision boundary ranging from -∞ to +∞. A positive score for a class indicates that x is predicted to be in that class. A negative score indicates otherwise.

The positive class classification score $$f(x)$$ is the trained SVM classification function. $$f(x)$$ is also the numerical predicted response for x, or the score for predicting x into the positive class.

$$f(x)={\displaystyle \sum _{j=1}^n{\alpha }_j}{y}_{j}G(x_j,x)+b,$$

where $$({\alpha }_1,\mathrm{...},{\alpha }_n,b)$$ are the estimated SVM parameters, $$G(x_j,x)$$ is the dot product in the predictor space between x and the support vectors, and the sum includes the training set observations. The negative class classification score for x, or the score for predicting x into the negative class, is –f(x).

If G(x_j,x) = x_j′x (the linear kernel), then the score function reduces to

$$f\left(x\right)=\left(x/s\right)\prime \beta +b.$$

s is the kernel scale and β is the vector of fitted linear coefficients.

For more details, see Understanding Support Vector Machines.

Posterior Probability

The posterior probability is the probability that an observation belongs in a particular class, given the data.

For SVM, the posterior probability is a function of the score P(s) that observation j is in class k = {-1,1}.

• For separable classes, the posterior probability is the step function

  $$P\left(s_j\right)=\{\begin{array}{l}\begin{array}{cc}0;& s<\underset{y_k=-1}{\max }{s}_k\end{array}\\ \begin{array}{cc}\pi ;& \underset{y_k=-1}{\max }{s}_k\le s_j\le \underset{y_k=+1}{\min }{s}_k\end{array}\\ \begin{array}{cc}1;& s_j>\underset{y_k=+1}{\min }{s}_k\end{array}\end{array},$$

  where:

  • s_j is the score of observation j.

  • +1 and –1 denote the positive and negative classes, respectively.

  • π is the prior probability that an observation is in the positive class.

• For inseparable classes, the posterior probability is the sigmoid function

  $$P(s_j)=\frac{1}{1+\exp (A{s}_j+B)},$$

  where the parameters A and B are the slope and intercept parameters, respectively.

Prior Probability

The prior probability of a class is the assumed relative frequency with which observations from that class occur in a population.

Tips

• If you are using a linear SVM model for classification and the model has many support vectors, then using predict for the prediction method can be slow. To efficiently classify observations based on a linear SVM model, remove the support vectors from the model object by using discardSupportVectors.

Algorithms

• By default and irrespective of the model kernel function, MATLAB^® uses the dual representation of the score function to classify observations based on trained SVM models, specifically

  $$\widehat{f}(x)={\displaystyle \sum _{j=1}^n{\widehat{\alpha }}_j}{y}_{j}G(x,x_j)+\widehat{b}.$$

  This prediction method requires the trained support vectors and α coefficients (see the SupportVectors and Alpha properties of the SVM model).

• By default, the software computes optimal posterior probabilities using Platt’s method [3]:

  1. Perform 10-fold cross-validation.

  2. Fit the sigmoid function parameters to the scores returned from the cross-validation.

  3. Estimate the posterior probabilities by entering the cross-validation scores into the fitted sigmoid function.

• The software incorporates prior probabilities in the SVM objective function during training.

• For SVM, predict and resubPredict classify observations into the class yielding the largest score (the largest posterior probability). The software accounts for misclassification costs by applying the average-cost correction before training the classifier. That is, given the class prior vector P, misclassification cost matrix C, and observation weight vector w, the software defines a new vector of observation weights (W) such that

  $$W_j=w_j{P}_j{\displaystyle \sum _{k=1}^K{C}_{jk}}.$$

Alternative Functionality

Simulink Block

To integrate the prediction of an SVM classification model into Simulink^®, you can use the ClassificationSVM Predict block in the Statistics and Machine Learning Toolbox™ library or a MATLAB Function block with the predict function. For examples, see Predict Class Labels Using ClassificationSVM Predict Block and Predict Class Labels Using MATLAB Function Block.

When deciding which approach to use, consider the following:

• If you use the Statistics and Machine Learning Toolbox library block, you can use the Fixed-Point Tool (Fixed-Point Designer) to convert a floating-point model to fixed point.

• Support for variable-size arrays must be enabled for a MATLAB Function block with the predict function.

• If you use a MATLAB Function block, you can use MATLAB functions for preprocessing or post-processing before or after predictions in the same MATLAB Function block.

References

[1] Christianini, N., and J. C. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, UK: Cambridge University Press, 2000.

[2] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning, Second Edition. NY: Springer, 2008.

[3] Platt, J. “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods.” Advances in Large Margin Classifiers. MIT Press, 1999, pages 61–74.

Extended Capabilities

Tall Arrays

Calculate with arrays that have more rows than fit in memory.

The predict function fully supports tall arrays. For more information, see Tall Arrays.

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• You can generate C/C++ code for both predict and update by using a coder configurer. Or, generate code only for predict by using saveLearnerForCoder, loadLearnerForCoder, and codegen.

  • Code generation for predict and update — Create a coder configurer by using learnerCoderConfigurer and then generate code by using generateCode. Then you can update model parameters in the generated code without having to regenerate the code.

  • Code generation for predict — Save a trained model by using saveLearnerForCoder. Define an entry-point function that loads the saved model by using loadLearnerForCoder and calls the predict function. Then use codegen (MATLAB Coder) to generate code for the entry-point function.

• For single-precision code generation, use standardized data by specifying 'Standardize',true when you train the model. To generate single-precision C/C++ code for predict, specify the name-value argument "DataType","single" when you call the loadLearnerForCoder function.

• You can also generate fixed-point C/C++ code for predict. Fixed-point code generation requires an additional step that defines the fixed-point data types of the variables required for prediction. Create a fixed-point data type structure by using the data type function generated by generateLearnerDataTypeFcn, and then use the structure as an input argument of loadLearnerForCoder in an entry-point function. Generating fixed-point C/C++ code requires MATLAB Coder™ and Fixed-Point Designer™.

• This table contains notes about the arguments of predict. Arguments not included in this table are fully supported.

  Argument	Notes and Limitations
  SVMModel	If you use saveLearnerForCoder to save a model that is equipped to predict posterior probabilities, and use loadLearnerForCoder to load the model, then loadLearnerForCoder cannot restore the ScoreTransform property into the MATLAB Workspace. However, loadLearnerForCoder can load the model, including the ScoreTransform property, within an entry-point function at compile time for code generation. For the usage notes and limitations of the model object, see Code Generation of the CompactClassificationSVM object.
  X	For general code generation, X must be a single-precision or double-precision matrix or a table containing numeric variables, categorical variables, or both. In the coder configurer workflow, X must be a single-precision or double-precision matrix. For fixed-point code generation, X must be a fixed-point matrix. The number of rows, or observations, in X can be a variable size, but the number of columns in X must be fixed. If you want to specify X as a table, then your model must be trained using a table, and your entry-point function for prediction must do the following: Accept data as arrays. Create a table from the data input arguments and specify the variable names in the table. Pass the table to predict. For an example of this table workflow, see Generate Code to Classify Data in Table. For more information on using tables in code generation, see Code Generation for Tables (MATLAB Coder) and Table Limitations for Code Generation (MATLAB Coder).

For more information, see Introduction to Code Generation.

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

Usage notes and limitations:

• The predict function does not support one-class classification models.

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2014a