DriftDetectionMethod

Incremental drift detector that utilizes Drift Detection Method (DDM)

Since R2022a

Description

DriftDetectionMethod model object represents an incremental concept drift detector that uses the Drift Detection Method [1]. After creating the object, you can use the detectdrift object function to update the statistics and check for any drift in the concept data (for example, failure rate, regression loss, and so on).

DriftDetectionMethod is suitable for incremental concept drift detection. For drift detection on raw data, see detectdrift for batch drift detection.

Creation

You can create DriftDetectionMethod by specifying the DetectionMethod argument as "ddm" in the call to incrementalConceptDriftDetector.

Properties

Alternative — Type of alternative hypothesis

'greater' (default) | 'less'

Type of alternative hypothesis for determining the drift status, specified as either 'greater' or 'less'.

Data Types: char

DriftDetected — Flag indicating whether software detects drift

1 | 0

This property is read-only.

Flag indicating whether software detects drift or not, specified as either 1 or 0. Value of 1 means DriftStatus is 'Drift'.

Data Types: logical

DriftStatus — Current drift status

'Stable' | 'Warning' | 'Drift'

This property is read-only.

Current drift status, specified as 'Stable', 'Warning', or 'Drift'. You can see the transition in the drift status by comparing DriftStatus and PreviousDriftStaus.

Data Types: char

DriftThreshold — Number of standard deviations for drift limit

nonnegative scalar value

This property is read-only.

Number of standard deviations for drift limit, specified as a nonnegative scalar value. This is the number of standard deviations the overall test statistic can be away from the optimal test statistic before the software sets DriftStatus to 'Drift'.

Data Types: double

InputType — Type of input data

'binary' (default) | 'continuous'

This property is read-only.

Type of input data, specified as either 'binary' or 'continuous'.

Data Types: char

IsWarm — Flag indicating whether warmup period is over

1 | 0

This property is read-only.

Flag indicating whether the warmup period is over or not, specified as 1 (true) or 0(false).

Data Types: logical

Mean — Weighted average of all input data

numeric value

This property is read-only.

Weighted average of all input data used for training the drift detector, specified as a numeric value.

Data Types: double

NumTrainingObservations — Number of observations used for training

nonnegative integer value

This property is read-only.

Number of observations used for training the drift detector, specified as a nonnegative integer value.

Data Types: double

OptimalMean — Optimal weighted average

numeric value

Optimal weighted average detectdrift observes up to the most current data point, specified as a numeric value.

detectdrift updates the OptimalMean and OptimalStandardDeviation under any of these conditions:

• When Alternative is 'greater' and Mean + StandardDeviation is less than or equal to OptimalMean + OptimalStandardDeviation.

• When Alternative is 'less' and Mean - StandardDeviation is greater than or equal to OptimalMean - OptimalStandardDeviation.

Data Types: double

OptimalStandardDeviation — Optimal weighted standard deviation

numeric value

This property is read-only.

Optimal weighted standard deviation detectdrift observes up to the most current data point, specified as a numeric value.

detectdrift updates the OptimalMean and OptimalStandardDeviation under any of these conditions:

• When Alternative is 'greater' and Mean + StandardDeviation is less than or equal to OptimalMean + OptimalStandardDeviation.

• When Alternative is 'less' and Mean - StandardDeviation is greater than or equal to OptimalMean - OptimalStandardDeviation.

Data Types: double

PreviousDriftStatus — Drift status prior to the latest training

'Stable' | 'Warning' | 'Drift'

This property is read-only.

Drift status prior to the latest training using the most recent batch of data, specified as 'Stable', 'Warning', or 'Drift'. You can see the transition in the drift status by comparing DriftStatus and PreviousDriftStaus.

Data Types: char

StandardDeviation — Weighted standard deviation of all input data

numeric value

This property is read-only.

Weighted standard deviation of all input data used for training the drift detector, specified as a numeric value.

Data Types: double

WarmupPeriod — Number of observations for drift detector warmup

nonnegative integer value

This property is read-only.

Number of observations for drift detector warmup, specified as a nonnegative integer.

Data Types: double

WarningDetected — Flag indicating whether there is warning

1 | 0

This property is read-only.

Flag indicating whether there is warning or not, specified as either 1 or 0. Value of 1 means DriftStatus is 'Warning'.

Data Types: logical

WarningThreshold — Number of standard deviations for warning limit

nonnegative scalar value

This property is read-only.

Number of standard deviations for warning limit, specified as a nonnegative scalar value. This is the number of standard deviations the overall test statistic can be away from the optimal test statistic before the software sets DriftStatus to 'Warning'.

Data Types: double

Object Functions

detectdrift	Update drift detector states and drift status with new data
reset	Reset incremental concept drift detector

Examples

Monitor Data Stream for Potential Drift

Initiate the concept drift detector using the Drift Detection Method (DDM).

incCDDetector = incrementalConceptDriftDetector("ddm");

Create a random stream such that for the first 1000 observations, failure rate is 0.1 and after 1000 observations, failure rate increases to 0.6.

rng(1234)  % For reproducibility
numObservations = 3000;
switchPeriod = 1000;

for i = 1:numObservations
    if i <= switchPeriod
       failurerate = 0.1;
    else
       failurerate = 0.6;
    end
       X(i) = rand()<failurerate; % Value 1 represents failure
end

Preallocate variables for tracking drift status.

status = zeros(numObservations,1);
statusname = strings(numObservations,1);

Continuously feed the data to the drift detector and perform incremental drift detection. At each iteration:

• Update statistics of the drift detector and monitor for drift using the new data point with detectdrift. (Note: detectdrift checks for drift after the warmup period.)

• Track and record the drift status for visualization purposes.

• When a drift is detected, reset the incremental concept drift detector by using reset.

for i = 1:numObservations     
    
    incCDDetector = detectdrift(incCDDetector,X(i));
    statusname(i) = string(incCDDetector.DriftStatus);
          
    if incCDDetector.DriftDetected
       status(i) = 2;
       incCDDetector = reset(incCDDetector); % If drift detected, reset the detector
       sprintf("Drift detected at Observation #%d. Detector reset.",i)
    elseif incCDDetector.WarningDetected
       status(i) = 1;
    else 
       status(i) = 0;
    end   
end

ans = 
"Drift detected at Observation #1078. Detector reset."

After the change in the failure rate at observation number 1000, detectdrift detects the shift at observation number 1078.

Plot the drift status versus the observation number.

gscatter(1:numObservations,status,statusname,'gyr','*',4,'on',"Observation number","Drift status")

Monitor Data Stream for Decrease in Failure Rate

Initiate the concept drift detector using the Drift Detection Method (DDM).

incCDDetector = incrementalConceptDriftDetector("ddm",Alternative="less",WarmupPeriod=100);

Create a random stream such that for the first 1000 observations, failure rate is 0.4 and after 1000 failure rate decreases to 0.1.

rng(1234)  % For reproducibility
numObservations = 3000;
switchPeriod = 1000;
for i = 1:numObservations
    if i <= switchPeriod
       failurerate = 0.4;
    else
       failurerate = 0.125;
    end
       X(i) = rand()<failurerate; % Value 1 represents failure
end

Preallocate variables for tracking drift status and the optimal mean and optimal standard deviation value.

optmean = zeros(numObservations,1);
optstddev = zeros(numObservations,1);
status = zeros(numObservations,1);
statusname = strings(numObservations,1);

Continuously feed the data to the drift detector and monitor for any potential change. Record the drift status for visualization purposes.

for i = 1:numObservations     
    
    incCDDetector = detectdrift(incCDDetector,X(i)); 

    statusname(i) = string(incCDDetector.DriftStatus);
    optmean(i) = incCDDetector.OptimalMean;
    optstddev(i) = incCDDetector.OptimalStandardDeviation;

    if incCDDetector.DriftDetected
       status(i) = 2;
       incCDDetector = reset(incCDDetector); % If drift detected, reset the detector
       sprintf("Drift detected at Observation #%d. Detector reset.",i)
    elseif incCDDetector.WarningDetected
       status(i) = 1;
    else 
       status(i) = 0;
    end   
end

ans = 
"Drift detected at Observation #1107. Detector reset."

After the change in the failure rate at observation number 1000, detectdrift detects the shift at observation number 1096.

Plot the change in the optimal mean and optimal standard deviation.

tiledlayout(2,1);
ax1 = nexttile;
plot(ax1,1:numObservations,optmean)
ax2 = nexttile;
plot(ax2,1:numObservations,optstddev)

Plot the drift status versus the observation number.

figure();
gscatter(1:numObservations,status,statusname,'gyr','*',4,'on',"Observation number","Drift status")

detectdrift concludes on a warning status for multiple observations before it decides on a drift.

References

[1] Gama, Joao, Pedro Medas, Gladys Castillo, and Pedro P. Rodrigues. “Learning with drift detection.“ In Brazilian symposium on artificial intelligence, pp. 286-295. Berlin, Heidelberg: Springer. 2004, September.

Version History

Introduced in R2022a