oobLoss

Out-of-bag classification error

Syntax

```L = oobLoss(ens) L = oobLoss(ens,Name,Value) ```

Description

`L = oobLoss(ens)` returns the classification error for `ens` computed for out-of-bag data.

`L = oobLoss(ens,Name,Value)` computes error with additional options specified by one or more `Name,Value` pair arguments. You can specify several name-value pair arguments in any order as `Name1,Value1,…,NameN,ValueN`.

Input Arguments

 `ens` A classification bagged ensemble, constructed with `fitcensemble`.

Name-Value Arguments

Specify optional pairs of arguments as `Name1=Value1,...,NameN=ValueN`, where `Name` is the argument name and `Value` is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose `Name` in quotes.

`learners`

Indices of weak learners in the ensemble ranging from `1` to `ens``.NumTrained`. `oobLoss` uses only these learners for calculating loss.

Default: `1:NumTrained`

`lossfun`

Loss function, specified as the comma-separated pair consisting of `'LossFun'` and a built-in loss function name or function handle.

• The following table lists the available loss functions. Specify one using its corresponding character vector or string scalar.

ValueDescription
`'binodeviance'`Binomial deviance
`'classifcost'`Observed misclassification cost
`'classiferror'`Misclassified rate in decimal
`'exponential'`Exponential loss
`'hinge'`Hinge loss
`'logit'`Logistic loss
`'mincost'`Minimal expected misclassification cost (for classification scores that are posterior probabilities)
`'quadratic'`Quadratic loss

`'mincost'` is appropriate for classification scores that are posterior probabilities. Bagged ensembles return posterior probabilities as classification scores by default.

• Specify your own function using function handle notation.

Suppose that `n` be the number of observations in `X` and `K` be the number of distinct classes (`numel(ens.ClassNames)`, `ens` is the input model). Your function must have this signature

``lossvalue = lossfun(C,S,W,Cost)``
where:

• The output argument `lossvalue` is a scalar.

• You choose the function name (`lossfun`).

• `C` is an `n`-by-`K` logical matrix with rows indicating which class the corresponding observation belongs. The column order corresponds to the class order in `ens.ClassNames`.

Construct `C` by setting ```C(p,q) = 1``` if observation `p` is in class `q`, for each row. Set all other elements of row `p` to `0`.

• `S` is an `n`-by-`K` numeric matrix of classification scores. The column order corresponds to the class order in `ens.ClassNames`. `S` is a matrix of classification scores, similar to the output of `predict`.

• `W` is an `n`-by-1 numeric vector of observation weights. If you pass `W`, the software normalizes them to sum to `1`.

• `Cost` is a K-by-`K` numeric matrix of misclassification costs. For example, ```Cost = ones(K) - eye(K)``` specifies a cost of `0` for correct classification, and `1` for misclassification.

Specify your function using `'LossFun',@lossfun`.

For more details on loss functions, see Classification Loss.

Default: `'classiferror'`

`mode`

Character vector or string scalar representing the meaning of the output `L`:

• `'ensemble'``L` is a scalar value, the loss for the entire ensemble.

• `'individual'``L` is a vector with one element per trained learner.

• `'cumulative'``L` is a vector in which element `J` is obtained by using learners `1:J` from the input list of learners.

Default: `'ensemble'`

`UseParallel`

Indication to perform inference in parallel, specified as `false` (compute serially) or `true` (compute in parallel). Parallel computation requires Parallel Computing Toolbox™. Parallel inference can be faster than serial inference, especially for large datasets. Parallel computation is supported only for tree learners.

Default: `false`

Output Arguments

 `L` Classification loss of the out-of-bag observations, a scalar. `L` can be a vector, or can represent a different quantity, depending on the name-value settings.

Examples

expand all

`load fisheriris`

Grow a bag of 100 classification trees.

`ens = fitcensemble(meas,species,'Method','Bag');`

Estimate the out-of-bag classification error.

`L = oobLoss(ens)`
```L = 0.0400 ```