Main Content

anomalyDetectionMetrics

Anomaly detection metrics

Since R2022b

    Description

    An anomalyDetectionMetrics object encapsulates anomaly detection quality metrics for a set of images. Images of anomalies belong to the positive (true) class. Normal images belong to the negative (false) class.

    Note

    This functionality requires the Computer Vision Toolbox™ Automated Visual Inspection Library. You can install the Computer Vision Toolbox Automated Visual Inspection Library from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    Creation

    Create an anomalyDetectionMetrics object using the evaluateAnomalyDetection function.

    Properties

    expand all

    This property is read-only.

    Confusion matrix, specified as a 2-by-2 table. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j.

    This property is read-only.

    Normalized confusion matrix, specified as a 2-by-2 table. NormalizedConfusionMatrix represents a confusion matrix normalized by the number of images known to belong to each class. Each table element (i, j) is the count of images known to belong to class i but predicted to belong to class j, divided by the total number of images predicted in class i. Elements are in the range [0, 1].

    This property is read-only.

    Metrics aggregated over the data set, specified as a 1-by-8 table with columns corresponding to these metrics:

    • GlobalAccuracy — Ratio of correctly classified images to total images, regardless of class

    • MeanAccuracy — Ratio of correctly classified images of each class to total images, averaged over all classes

    • Precision — Positive predictive value

    • Recall — True positive (TP) rate

    • Specificity — True negative (TN) rate

    • F1Score — Harmonic mean of precision and recall

    • FalsePositiveRate — False positive (FP) rate, or fallout

    • FalseNegativeRate — False negative (FN) rate, or miss

    This property is read-only.

    Anomaly detection metrics for each class, specified as a 2-by-2 table. The table has rows "Normal" and "Anomaly" and columns corresponding to these metrics:

    • Accuracy — Fraction of correctly classified images for each class (normal or anomaly).

    • AccuracyPerSubClass — Fraction of correctly classified images for each subclass within the normal and anomaly classes. The value of AccuracyPerSubClass is a table with C rows and one column, where C is the number of subclasses.

    Examples

    collapse all

    Load test images and corresponding labels, then create a datastore that reads the test data. The data set consists of grayscale images of handwritten digits 0–9.

    [Xtest,gtLabels] = digitTest4DArrayData;
    dsTest = arrayDatastore(Xtest,IterationDimension=4);

    Load a pretrained FCDD anomaly detector. This detector has been trained to classify the digit 8 as normal and all other digits as anomalies. Therefore, specify the set of anomaly labels as the set of digits between 0 and 9, excluding 8.

    load("digit8AnomalyDetector.mat");
    anomalyLabels = setdiff(string(0:9),"8");

    Classify each test image as normal or anomalous.

    predLabels = classify(detector,dsTest);

    Calculate metrics for the anomaly detection results.

    metrics = evaluateAnomalyDetection(predLabels,gtLabels,anomalyLabels)
    Evaluating anomaly detection results
    ------------------------------------
    * Finalizing... Done.
    * Data set metrics:
    
        GlobalAccuracy    MeanAccuracy    Precision    Recall    Specificity    F1Score    FalsePositiveRate    FalseNegativeRate
        ______________    ____________    _________    ______    ___________    _______    _________________    _________________
    
            0.7662           0.839         0.98971     0.748        0.93        0.85204          0.07                 0.252      
    
    metrics = 
      anomalyDetectionMetrics with properties:
    
                  ConfusionMatrix: [2x2 table]
        NormalizedConfusionMatrix: [2x2 table]
                   DataSetMetrics: [1x8 table]
                     ClassMetrics: [2x2 table]
    
    

    Display the confusion matrix.

    cm = metrics.ConfusionMatrix
    cm=2×2 table
                   Normal    Anomaly
                   ______    _______
    
        Normal       465        35  
        Anomaly     1134      3366  
    
    

    Version History

    Introduced in R2022b