Perform Incremental Learning Using IncrementalClassificationLinear Fit and Predict Blocks

Since R2023b

This example shows how to use the IncrementalClassificationLinear Predict and IncrementalClassificationLinear Fit blocks for incremental learning and classification in Simulink®. The IncrementalClassificationLinear Fit block fits a chunk of observations (predictor data) using a configured incremental linear model for binary classification (incrementalClassificationLinear) and outputs updated incremental learning model parameters as a bus signal. The IncrementalClassificationLinear Predict block accepts an IncrementalClassificationLinear model and a chunk of predictor data, and outputs the predicted labels for the observations.

Load and Preprocess Data

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
rng(0,"twister") % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

For details on the data set, enter Description at the command line.

Responses can be one of five classes: Sitting, Standing, Walking, Running, or Dancing. Dichotomize the response by identifying whether the subject is moving (actid > 2).

Y = Y > 2;

Create Incremental Learning Model

Create an incremental linear model for binary classification. Specify that the data has 60 predictors and that the data type of the responses is logical. Specify to standardize the data using an estimation period of 500 observations. Create a workspace variable linearMdl to store the initial incremental learning model.

Mdl = incrementalClassificationLinear(NumPredictors=60,ClassNames=[false,true], ...
    Standardize=true,EstimationPeriod=500);
linearMdl = Mdl;

Create Input Data for Simulink

Simulate streaming data by dividing the training data into chunks of 50 observations. For each chunk, select a single observation as a test set to import into the IncrementalClassificationLinear Predict block.

numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;   
    Xin(:,:,j) = X(idx,:);
    Yin(:,j) = Y(idx);
    Xtest(1,:,j) = X(idx(1),:);
 end

Convert the training and test set chunks into time series objects.

t = 0:size(Xin,3)-1;
Xtrain_ts = timeseries(Xin,t,InterpretSingleRowDataAs3D=true);
Ytrain_ts = timeseries(Yin',t,InterpretSingleRowDataAs3D=true);
Xtest_ts = timeseries(Xtest,t,InterpretSingleRowDataAs3D=true);

Open Provided Simulink Model

This example provides the Simulink model slexIncClassificationLinearPredictExample.slx, which includes the IncrementalClassificationLinear Predict and IncrementalClassificationLinear Fit blocks. The Simulink model is configured to use linearMdl as the initial model for incremental learning and classification.

Open the Simulink model slexIncClassificationLinearPredictExample.slx.

slName = "slexIncClassificationLinearPredictExample";
open_system(slName);

Simulate Model

Simulate the Simulink model to perform incremental learning and predict responses to the test set observations. Export the simulation outputs to the workspace. You can use the Simulation Data Inspector (Simulink) to view the logged data of an Outport block.

simOut = sim(slName,"StopTime",num2str(numel(t)-1));

% Extract labels
yfit_sig = simOut.yout.getElement(1);
yfit_sl = squeeze(yfit_sig.Values.Data);

% Extract score values
scores_sig = simOut.yout.getElement(2);
scores_sl = squeeze(scores_sig.Values.Data);

% Extract beta values
beta_sig = simOut.yout.getElement(3);
beta_sl = squeeze(beta_sig.Values.Data);

% Extract bias values
bias_sig = simOut.yout.getElement(4);
bias_sl = squeeze(bias_sig.Values.Data);

At each iteration, the IncrementalClassificationLinear Fit block trains the model and updates the model parameters. The IncrementalClassificationLinear Predict block calculates the predicted label for the test set observation.

Analyze Model During Training

To see how the model parameters and response values evolve during training, plot them on separate tiles.

figure
tiledlayout(4,1);
nexttile
plot(scores_sl(1,:),".")
ylabel("Score")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(yfit_sl,".")
ylabel("Label")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(beta_sl(1,:),".-")
ylabel("\beta_1")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(bias_sl,".-")
ylabel("Bias")
xlabel("Iteration")
xlim([0 nchunk])

During the estimation period, the IncrementalClassificationLinear Fit block estimates hyperparameters but does not fit the initial model (see incrementalRegressionLinear). Therefore, the score output of the IncrementalClassificationLinear Predict block, model beta coefficients, and model bias all are equal to zero during the first 10 iterations. At the end of the estimation period, the IncrementalClassificationLinear Fit block updates the model parameters and predicts labels. After the estimation period, the score values vary between approximately –12 and 3. The first beta coefficient varies significantly during the first 22 iterations following the estimation period, and approaches a constant value of approximately 0.02 thereafter. The bias (intercept) term has an initial value of 0 and gradually approaches –0.12.