Main Content

trackOSPAMetric

Optimal subpattern assignment (OSPA) metric

Since R2019b

Description

The trackOSPAMetric System object™ computes the optimal subpattern assignment metric between a set of tracks and the known truths. You can enable different types of OSPA metrics by configuring these properties:

  • Traditional OSPA — Specify the Metric property as "OSPA" and specify the LabelingError property as 0. The traditional OSPA metric, which evaluates instantaneous tracking performance, contains two components:

    • Localization error component — Accounts for state estimation errors between assigned tracks and truths.

    • Cardinality error component— Accounts for the number of unassigned tracks and truths.

  • Labeled OSPA — Specify the Metric property as "OSPA" and specify the LabelingError property as a positive scalar. The labeled OSPA (LOSPA) metric, which evaluates instantaneous tracking performance and includes penalties for incorrect assignments, contains three components:

    • Localization error component — Accounts for state estimation errors between assigned tracks and truths.

    • Cardinality error component— Accounts for the number of unassigned tracks and truths.

    • Labeling error component — Accounts for the error of incorrect assignments.

  • OSPA(2) — Specify the Metric property as "OSPA(2)". The OSPA(2) metric evaluates cumulative tracking performance for a duration of time.

For more details, see Algorithms and References.

To use trackOSPAMetric:

  1. Create the trackOSPAMetric object and set its properties.

  2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Description

OSPAMetric = trackOSPAMetric creates a trackOSPAMetric System object, OSPAMetric, with default property values.

OSPAMetric = trackOSPAMetric(Name,Value) sets properties for the trackOSPAMetric object using one or more name-value pairs. For example, OSPAMetric = trackOSPAMetric('CutoffDistance',5) creates a trackOSPAMetric object with the cut off distance equal to 5. Enclose property names in single quotes.

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

Metric option, specified as "OSPA" or "OSPA(2)".

  • "OSPA" — Computes the traditional OSPA metric by default, or computes the labeled OSPA metric by additionally specifying the LabelingError property as a positive value.

  • "OSPA(2)" — Computes the OSPA(2) metric, which evaluates cumulative tracking performance. Selecting this option enables these properties for configuring the metric:

    • WindowLength

    • WindowSumOrder

    • WindowWeights

    • WindowWeightExponent

    • CustomWindowWeights

    Selecting this option also disables two properties used to evaluate the labeling error component:

    • HasAssignmentInput

    • LabelingError

    When selecting this option, the object internally saves the accumulated track and truth history up to a number of steps defined by the WindowLength property.

Data Types: char | string

Threshold for cutoff distance between track and truth, specified as a real positive scalar. If the computed distance between a track and the assigned truth is higher than the threshold, the actual distance incorporated in the metric is reduced to the threshold.

Example: 40

Data Types: single | double

Order of the metric, specified as a positive integer.

Example: 3

Data Types: single | double

Distance type, specified as 'posnees', 'velnees', 'posabserr', or 'velabserr'. The distance type specifies the physical quantity used for distance calculations:

  • 'posnees' – Normalized estimation error squared (NEES) of track position

  • 'velnees' – NEES error of track velocity

  • 'posabserr' – Absolute error of track position

  • 'velabserr' – Absolute error of track velocity

  • 'custom' – Custom distance error

If you specify the Distance property as 'custom', you must also specify the distance function in the DistanceFcn property.

Custom distance function, specified as a function handle. The function must support the following syntax:

d = myCustomFcn(Track,Truth)
where Track is a structure or an object of track information, Truth is a structure or an object of truth information, and d is the distance between the truth and the track. See objectTrack for an example on how to organize track information.

Example: @myCustomFcn

Dependencies

To enable this property, set the Distance property to 'custom'.

Desired platform motion model, specified as 'constvel', 'constacc', 'constturn', or 'singer'. This property selects the motion model used by the tracks input.

The motion models expect the 'State' field of the tracks to have a column vector containing these values:

  • 'constvel' — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

  • 'constacc' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

  • 'constturn' — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

  • 'singer' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The 'StateCovariance' field of the tracks input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the 'State' field of the tracks input.

Track identifier function, specified as a function handle. The function extracts track ID from the track input. The function must support the following syntax:

Trackids = trackIdentifier(Tracks)
where Tracks is an array of structures or objects containing the information of tracks, and Trackids is a numeric array of the same size as Tracks. For an example of track object, see objectTrack. For the default identifier function, defaultTrackIdentifier, the track ID must be contained in Tracks as the value of the TrackID field or property.

Example: @myTrackIdetifier

Truth identifier function, specified as a function handle. The function extracts truth ID from truth input. The function must support the following syntax:

TruthIDs = truthIdentifier(Truths)
where Truths is an array of structures or objects containing the information of truths, and TruthIDs is a numeric array of the same size as Truths. For the use of the default identifier function, defaultTruthIdentifier, the truth ID must be contained in Truth as a value of the PlatformID field or property.

Example: @myTruthIdetifier

OSPA(2)-Only Properties

Sliding window length for the OSPA(2) metric, specified as a positive integer. The window length defines the number of time steps from a previous time to the current time used to evaluate the metric. For more details, see OSPA(2) Metric.

Dependencies

To enable this property, set the Metric property to "OSPA(2)".

Data Types: single | double

Order of the weighted sum for the track and truth history, specified as a positive scalar. For more details, see OSPA(2) Metric.

Dependencies

To enable this property, set the Metric property to "OSPA(2)".

Data Types: single | double

Options for window weights, specified as "auto" or "custom".

  • "auto" — Automatically generates the window weights using the algorithm given in OSPA(2) Metric.

  • "custom" — Customizes the window weights using the CustomWindowWeights property.

Dependencies

To enable this property, set the Metric property to "OSPA(2)".

Data Types: single | double

Exponent for automatic weight calculation, specified as a nonnegative scalar. An exponent value, r, of 0 represents equal weights in the window. A higher value of r assigns more weight to recent data. For more details, see OSPA(2) Metric.

Dependencies

To enable this property, set the WindowWeights property to "auto".

Data Types: single | double

Custom weights in the time window, specified as an N-element of vector of nonnegative values, where N is the window length specified in the WindowLength property.

Dependencies

To enable this property, set the WindowWeights property to "custom".

Data Types: single | double

LOSPA-Only Properties

Penalty for incorrect assignment of track to truth, specified as a real positive scalar. The function decides if an assignment is correct based on the provided known assignment input. If the assignment is not provided as an input, the last known "optimal" assignment is assumed to be correct.

Example: 5

Dependencies

To enable this property, set the Metric property to "OSPA".

Data Types: single | double

Enable assignment input, specified as true or false.

Dependencies

To enable this property, set the Metric property to "OSPA".

Data Types: logical

Usage

Description

metric = OSPAMetric(tracks,truths) returns the tracking performance metric between the set of tracks and truths.

metric = OSPAMetric(tracks,truths,assignment) specifies the known assignment between tracks and truths at the current time step. To use this syntax, specify the HasAssignmentInput property as true.

[metric,local] = OSPAMetric(___) returns the localization error component of the OSPA metric using any of the input combinations in the previous syntaxes.

example

[metric,local,card] = OSPAMetric(___) also returns the cardinality error component of the OSPA metric.

[metric,local,card,label] = OSPAMetric(___) also returns the labeling error component of the OSPA metric.

To use this syntax, specify the Metric property as "OSPA(2)".

Input Arguments

expand all

Track information, specified as an array of structures or objects for non-custom (built-in) distance functions. Each structure or object must contain State as a field or property. Additionally, if an NEES-based distance (posnees or velnees) is specified in the Distance property, each structure or object must also contain StateCovariance as a field or property. Moreover, if the default track identifier function is used in the TrackIdentifierFcn property, then each structure or object must also contain TrackID as a field or property.

Data Types: struct | object

Truth information at the current time, specified as an array of structures or objects for noncustomized (built-in) distance functions. Each structure or object must contain Position and Velocity as fields or properties. If the default truth identifier function is used in the TruthIdentifierFcn property, then each structure or object must also contain PlatformID as a field or property.

Data Types: struct | object

Known assignment, specified as an N-by-2 matrix of nonnegative integers. The first column elements are track IDs, and the second column elements are truth IDs. The IDs in the same row are assigned to each other. If a track or truth is not assigned, specify 0 as the same row element.

Note that the assignment must be a unique assignment between tracks and truths. Redundant or false tracks should be treated as unassigned tracks by assigning them to the "0" TruthID.

Data Types: single | double

Output Arguments

expand all

Tracking performance metric, returned as a nonnegative real scalar. Depending on the values of the Metric and LabelingError properties, the returned metric can be traditional OSPA, labeled OSPA (LOSPA), or OSPA(2).

Metric Property ValueLabelingError Property ValueMetric
"OSPA"0OSPA
"OSPA"Positive scalarLOSPA
"OSPA(2)"Not applicableOSPA(2)

Example: 10.1

Localization error component, returned as a nonnegative real scalar.

Example: 8.5

Cardinality error component, returned as a nonnegative real scalar.

Example: 6

Labeling error component, returned as a nonnegative real scalar.

Example: 7.5

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

stepRun System object algorithm
releaseRelease resources and allow changes to System object property values and input characteristics
resetReset internal states of System object
cloneCreate duplicate System object

Examples

collapse all

Load prerecorded track data and truth data.

load trackmetricex tracklog truthlog

Construct a trackOSPAMetric object.

tom = trackOSPAMetric;

Initialize output variables.

ospa = zeros(numel(tracklog),1);
cardOspa = zeros(numel(tracklog),1);
locOspa = zeros(numel(tracklog),1);

Calculate three OSPA components in a loop.

for i = 1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [ospa(i), locOspa(i), cardOspa(i)] = tom(tracks, truths);
end

Visualize the results.

figure()
plot(ospa,'g');
hold on;
plot(locOspa,'r:');
plot(cardOspa,'b--');
legend('OSPA','Localization OSPA','Cardinality OSPA');

Figure contains an axes object. The axes object contains 3 objects of type line. These objects represent OSPA, Localization OSPA, Cardinality OSPA.

Load prerecorded tracking data that includes tracks and truth trajectories into the workspace.

load ("ospa2TestLog.mat","trackLog","truthLog");
steps = numel(trackLog);

Show the tracks and truth trajectories. The recorded data contains three tracks and three truth trajectories.

positionSelector = [1 0 0 0 0 0;
                    0 0 1 0 0 0];
figure
hold on
for i = 1:steps   
    tracksi = trackLog{i};
    if ~isempty(tracksi)
        xyTrackPositions = positionSelector*[tracksi.State];
        plot(xyTrackPositions(1,:),xyTrackPositions(2,:),"b.")
    end

    truthsi = truthLog{i};
    if ~isempty(truthsi)
        xyTtruthPositions = cat(1,truthsi.Position)';
        plot(xyTtruthPositions(1,:),xyTtruthPositions(2,:),"r.")
    end
end
xlabel("x (m)")
ylabel("y (m)")
title("Tracks vs. Truths")

Figure contains an axes object. The axes object with title Tracks vs. Truths, xlabel x (m), ylabel y (m) contains 282 objects of type line.

Create a trackOSPAMetric object and enable the OSPA(2) metric. Specify the window length as 75.

ospa2Obj = trackOSPAMetric(Metric="OSPA(2)", ...
    WindowLength=75, ...
    CutoffDistance=50, ...
    WindowWeightExponent=3,...
    Order=1,...
    Distance="posabserr");

Loop over the data to obtain the OSPA(2) metric over time.

ospa2 = NaN(steps,1);
for i = 1:numel(trackLog)
    tracks = trackLog{i};
    truths = truthLog{i};

    ospa2(i) = ospa2Obj(tracks, truths);
end

Visualize the results.

figure
plot(ospa2)
axis padded
xlabel("Steps")
ylabel("OSPA(2) metric")

Figure contains an axes object. The axes object with xlabel Steps, ylabel OSPA(2) metric contains an object of type line.

Algorithms

expand all

References

[1] Schuhmacher, B., B. -T. Vo, and B. -N. Vo. "A Consistent Metric for Performance Evaluation of Multi-Object Filters." IEEE Transactions on Signal Processing, Vol, 56, No, 8, pp. 3447–3457, 2008.

[2] Ristic, B., B. -N. Vo, D. Clark, and B. -T. Vo. "A Metric for Performance Evaluation of Multi-Target Tracking Algorithms." IEEE Transactions on Signal Processing, Vol, 59, No, 7, pp. 3452–3457, 2011.

[3] M. Beard, B. -T. Vo, and B. -N. Vo. “OSPA (2) : Using the OSPA Metric to Evaluate Multi-Target Tracking Performance.” 2017 International Conference on Control, Automation and Information Sciences, IEEE, 2017, pp. 86–91.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

Version History

Introduced in R2019b