Main Content

Simulink.sdi.compareRuns

Compare data in two simulation runs

Description

example

diffResult = Simulink.sdi.compareRuns(runID1,runID2) compares the data in the runs that correspond to runID1 and runID2 and returns the result in the Simulink.sdi.DiffRunResult object diffResult. The comparison uses the Simulation Data Inspector comparison algorithm. For more information about the algorithm, see How the Simulation Data Inspector Compares Data.

example

diffResult = Simulink.sdi.compareRuns(runID1,runID2,Name,Value) compares the simulation runs that correspond to runID1 and runID2 using the options specified by one or more Name,Value pair arguments. For more information about how the options can affect the comparison, see How the Simulation Data Inspector Compares Data.

Examples

collapse all

You can specify global tolerance values to use when comparing two simulation runs. Global tolerance values are applied to all signals within the run. This example shows how to specify global tolerance values for a run comparison and how to analyze and save the comparison results.

First, load the session file that contains the data to compare. The session file contains data for four simulations of an aircraft longitudinal controller. This example compares data from two runs that use different input filter time constants.

Simulink.sdi.load('AircraftExample.mldatx');

To access the run data to compare, use the Simulink.sdi.getAllRunIDs function to get the run IDs that correspond to the last two simulation runs.

runIDs = Simulink.sdi.getAllRunIDs;
runID1 = runIDs(end - 1);
runID2 = runIDs(end);

Use the Simulink.sdi.compareRuns function to compare the runs. Specify a global relative tolerance value of 0.2 and a global time tolerance value of 0.5.

runResult = Simulink.sdi.compareRuns(runID1,runID2,'reltol',0.2,'timetol',0.5);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object to see whether signals were within the tolerance values or out of tolerance.

runResult.Summary
ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 3
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

All three signal comparison results fell within the specified global tolerance.

You can save the comparison results to an MLDATX file using the saveResult function.

saveResult(runResult,'InputFilterComparison');

You can programmatically specify signal tolerance values to use in comparisons performed using the Simulation Data Inspector. In this example, you compare data collected by simulating a model of an aircraft longitudinal flight control system. Each simulation uses a different value for the input filter time constant and logs the input and output signals. You analyze the effect of the time constant change by comparing results using the Simulation Data Inspector and signal tolerances.

First, load the session file that contains the simulation data.

Simulink.sdi.load('AircraftExample.mldatx');

The session file contains four runs. In this example, you compare data from the first two runs in the file. Access the Simulink.sdi.Run objects for the first two runs loaded from the file.

runIDs = Simulink.sdi.getAllRunIDs;
runIDTs1 = runIDs(end-3);
runIDTs2 = runIDs(end-2);

Now, compare the two runs without specifying any tolerances.

noTolDiffResult = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);

Use the getResultByIndex function to access the comparison results for the q and alpha signals.

qResult = getResultByIndex(noTolDiffResult,1);
alphaResult = getResultByIndex(noTolDiffResult,2);

Check the Status of each signal result to see whether the comparison result fell within our out of tolerance.

qResult.Status
ans = 
OutOfTolerance
alphaResult.Status
ans = 
OutOfTolerance

The comparison used a value of 0 for all tolerances, so the OutOfTolerance result means the signals are not identical.

You can further analyze the effect of the time constant by specifying tolerance values for the signals. Specify the tolerances by setting the properties for the Simulink.sdi.Signal objects that correspond to the signals being compared. Comparisons use tolerances specified for the baseline signals. This example specifies a time tolerance and an absolute tolerance.

To specify a tolerance, first access the Signal objects from the baseline run.

runTs1 = Simulink.sdi.getRun(runIDTs1);
qSig = getSignalsByName(runTs1,'q, rad/sec');
alphaSig = getSignalsByName(runTs1,'alpha, rad');

Specify an absolute tolerance of 0.1 and a time tolerance of 0.6 for the q signal using the AbsTol and TimeTol properties.

qSig.AbsTol = 0.1;
qSig.TimeTol = 0.6;

Specify an absolute tolerance of 0.2 and a time tolerance of 0.8 for the alpha signal.

alphaSig.AbsTol = 0.2;
alphaSig.TimeTol = 0.8;

Compare the results again. Access the results from the comparison and check the Status property for each signal.

tolDiffResult = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);
qResult2 = getResultByIndex(tolDiffResult,1);
alphaResult2 = getResultByIndex(tolDiffResult,2);

qResult2.Status
ans = 
WithinTolerance
alphaResult2.Status
ans = 
WithinTolerance

You can use the Simulink.sdi.compareRuns function to compare signal data and metadata, including data type and start and stop times. A single comparison may check for mismatches in one or more pieces of metadata. When you check for mismatches in signal metadata, the Summary property of the Simulink.sdi.DiffRunResult object may differ from a basic comparison because the Status property for a Simulink.sdi.DiffSignalResult object can indicate the metadata mismatch. You can configure comparisons using the Simulink.sdi.compareRuns function for imported data and for data logged from a simulation.

This example configures a comparison of runs created from workspace data three ways to show how the Summary of the DiffSignalResult object can provide specific information about signal mismatches.

Create Workspace Data

The Simulink.sdi.compareRuns function compares time series data. Create data for a sine wave to use as the baseline signal, using the timeseries format. Give the timeseries the name Wave Data.

time = 0:0.1:20;
sig1vals = sin(2*pi/5*time);
sig1_ts = timeseries(sig1vals,time);
sig1_ts.Name = 'Wave Data';

Create a second sine wave to compare against the baseline signal. Use a slightly different time vector and attenuate the signal so the two signals are not identical. Cast the signal data to the single data type. Also name this timeseries object Wave Data. The Simulation Data Inspector comparison algorithm will align these signals for comparison using the name.

time2 = 0:0.1:22;
sig2vals = single(0.98*sin(2*pi/5*time2));
sig2_ts = timeseries(sig2vals,time2);
sig2_ts.Name = 'Wave Data';

Create and Compare Runs in the Simulation Data Inspector

The Simulink.sdi.compareRuns function compares data contained in Simulink.sdi.Run objects. Use the Simulink.sdi.createRun function to create runs in the Simulation Data Inspector for the data. The Simulink.sdi.createRun function returns the run ID for each created run.

runID1 = Simulink.sdi.createRun('Baseline Run','vars',sig1_ts);
runID2 = Simulink.sdi.createRun('Compare to Run','vars',sig2_ts);

You can use the Simulink.sdi.compareRuns function to compare the runs. The comparison algorithm converts the signal data to the double data type and synchronizes the signal data before computing the difference signal.

basic_DRR = Simulink.sdi.compareRuns(runID1,runID2);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object to see the result of the comparison.

basic_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 1
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The difference between the signals is out of tolerance.

Compare Runs and Check for Data Type Match

Depending on your system requirements, you may want the data types for signals you compare to match. You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check for and report data type mismatches.

dataType_DRR = Simulink.sdi.compareRuns(runID1,runID2,'DataType','MustMatch');
dataType_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 1
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The result of the signal comparison is now DataTypeMismatch because the data for the baseline signal is double data type, while the data for the signal compared to the baseline is single data type.

Compare Runs and Check for Start and Stop Time Match

You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check whether the aligned signals have the same start and stop times.

startStop_DRR = Simulink.sdi.compareRuns(runID1,runID2,'StartStop','MustMatch');
startStop_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 1
          Unsupported: 0

The signal comparison result is now StartStopMismatch because the signals created in the workspace have different stop times.

When you compare runs using the Simulation Data Inspector, you can specify alignment criteria that determine how signals are paired with each other for comparison. This example compares data from simulations of a model of an aircraft longitudinal control system. The simulations used a square wave input. The first simulation used an input filter time constant of 0.1s and the second simulation used an input filter time constant of 0.5s.

First, load the simulation data from the session file that contains the data for this example.

Simulink.sdi.load('AircraftExample.mldatx');

The session file contains data for four simulations. This example compares data from the first two runs. Access the run IDs for the first two runs loaded from the session file.

runIDs = Simulink.sdi.getAllRunIDs;
runIDTs1 = runIDs(end-3);
runIDTs2 = runIDs(end-2);

Before running the comparison, define how you want the Simulation Data Inspector to align the signals between the runs. This example aligns signals by their name, then by their block path, and then by their Simulink identifier.

alignMethods = [Simulink.sdi.AlignType.SignalName
               Simulink.sdi.AlignType.BlockPath
               Simulink.sdi.AlignType.SID];

Compare the simulation data in your two runs, using the alignment criteria you specified. The comparison uses a small time tolerance to account for the effect of differences in the step size used by the solver on the transition of the square wave input.

diffResults = Simulink.sdi.compareRuns(runIDTs1,runIDTs2,'align',alignMethods,...
    'timetol',0.005);

You can use the getResultByIndex function to access the comparison results for the aligned signals in the runs you compared. You can use the Count property of the Simulink.sdi.DiffRunResult object to set up a for loop to check the Status property for each Simulink.sdi.DiffSignalResult object.

numComparisons = diffResults.count;

for k = 1:numComparisons
    resultAtIdx = getResultByIndex(diffResults,k);
    
    sigID1 = resultAtIdx.signalID1;
    sigID2 = resultAtIdx.signalID2;
    
    sig1 = Simulink.sdi.getSignal(sigID1);
    sig2 = Simulink.sdi.getSignal(sigID2);
    
    displayStr = 'Signals %s and %s: %s \n';
    fprintf(displayStr,sig1.Name,sig2.Name,resultAtIdx.Status);
end
Signals q, rad/sec and q, rad/sec: OutOfTolerance 
Signals alpha, rad and alpha, rad: OutOfTolerance 
Signals Stick and Stick: WithinTolerance 

Input Arguments

collapse all

Numeric identifier for the baseline run in the comparison, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

Numeric identifier for the run to compare, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'abstol',x,'align',alignOpts

Signal alignment options, specified as the comma-separated pair consisting of 'Align' and a string array or array of character vectors.

Array specifying alignment options to use for pairing signals from the runs being compared. The Simulation Data Inspector aligns signals first by the first element in the array, then by the second element in the array, and so on. For more information, see Signal Alignment.

ValueAligns By
Simulink.sdi.AlignType.BlockPathPath to the source block for the signal
Simulink.sdi.AlignType.SIDSimulink® identifier Simulink Identifiers
Simulink.sdi.AlignType.SignalNameSignal name
Simulink.sdi.AlignType.DataSourcePath of the variable in the MATLAB® workspace

Example: [Simulink.sdi.AlignType.SignalName,Simulink.sdi.AlignType.SID] specifies signal alignment by name and then by SID.

Positive-valued global absolute tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'AbsTol' and a scalar. For more information about how tolerances are used in comparisons, see Tolerance Specification.

Example: 0.5

Data Types: double

Positive-valued global relative tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'RelTol' and a scalar. The relative tolerance is expressed as a fractional multiplier. For example, 0.1 specifies a 10 percent tolerance. For more information about how the relative tolerance is applied in the Simulation Data Inspector, see Tolerance Specification.

Example: 0.1

Data Types: double

Positive-valued global time tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'TimeTol' and a scalar. Specify the time tolerance in units of seconds. For more information about tolerances in the Simulation Data Inspector, see Tolerance Specification.

Example: 0.2

Data Types: double

Specify the name-value pair 'DataType','MustMatch' when you want the comparison to be sensitive to data type mismatches in compared signals. When you specify this name-value pair, the algorithm compares the data types for aligned signals before synchronizing and comparing the signal data.

The Simulink.sdi.compareRuns function does not compare the data types of aligned signals unless you specify this name-value pair. The comparison algorithm can compare signals with different data types.

When signal data types do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to DataTypeMismatch.

When you specify that data types must match and configure the comparison to stop on the first mismatch, a data type mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Specify the name-value pair 'Time','MustMatch' when you want the comparison to be sensitive to mismatches in the time vectors of compared signals. When you specify this name-value pair, the algorithm compares the time vectors of aligned signals before synchronizing and comparing the signal data.

Comparisons are not sensitive to differences in signal time vectors unless you specify this name-value pair. For comparisons that are not sensitive to differences in the time vectors, the comparison algorithm synchronizes the signals prior to the comparison. For more information about how synchronization works, see How the Simulation Data Inspector Compares Data.

When the time vectors for signals do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to TimeMismatch.

When you specify that time vectors must match and configure the comparison to stop on the first mismatch, a time vector mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Specify the name-value pair 'StartStop','MustMatch' when you want the comparison to be sensitive to mismatches in signal start and stop times. When you specify this name-value pair, the algorithm compares the start and stop times for aligned signals before synchronizing and comparing the signal data.

When the start times and stop times do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to StartStopMismatch.

When you specify that start and stop times must match and configure the comparison to stop on the first mismatch, a start or stop time mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Whether the comparison stops without comparing remaining signals on the first detected mismatch, specified as the comma-separated pair consisting of 'StopOnFirstMismatch' and 'Metadata' or 'Any'. A stopped comparison may not compute results for all signals, and can return a mismatched result more quickly.

  • Metadata — A mismatch in metadata for aligned signals causes the comparison to stop. Metadata comparisons happen before comparing signal data.

    The Simulation Data Inspector always aligns signals and compares signal units. When you configure the comparison to stop on the first mismatch, an unaligned signal or mismatched units always causes the comparison to stop. You can specify additional name-value pairs to configure the comparison to check and stop on the first mismatch for additional metadata, such as signal data type, start and stop times, and time vectors.

  • Any — A mismatch in metadata or signal data for aligned signals causes the comparison to stop.

Output Arguments

collapse all

Comparison results, returned as a Simulink.sdi.DiffRunResult object.

Limitations

The Simulation Data Inspector does not support comparing:

  • Signals of data types int64 or uint64.

  • Variable-size signals.

Introduced in R2011b