SystemTest helps you develop and run your test and analyze results from the MATLAB command line or the SystemTest desktop. From the SystemTest desktop, you can:
SystemTest tests consist of elements, test vectors, and test variables. You can use each of these entities to create a variety of test scenarios ranging from a simple test that runs a series of elements once to a full parameter sweep that iterates over the values of test vectors that you define. Test vectors and test variables map data between SystemTest and the model or unit under test. You can optionally map test variables to results and perform analysis using the Test Results Viewer.
Test elements are discrete actions that run during test execution. You can use the test elements to integrate capabilities from other MathWorks products into SystemTest. SystemTest includes the following test elements:
Detailed information about additional test elements for SystemTest is available.
Test vectors specify the values that vary during each test iteration. SystemTest lets you use multiple test vectors whose values are arguments (for algorithms) or parameters and input signals (for models). You can define test vectors using any MATLAB expression or generate random values using probability distributions.
SystemTest includes normal (Gaussian) and uniform probability distribution functions.When used with Statistics and Machine Learning Toolbox (available separately), SystemTest lets you also specify exponential, gamma, lognormal, Student's t , or Weibull probability distributions.
Test variables are used to store data for use by test elements during execution. You can initialize variables in the Pre Test or Main Test with any MATLAB expression.
When you run a SystemTest TEST-file from the SystemTest desktop or using the
strun MATLAB command, it executes using the test structure you defined using test elements. The test structure is defined in three sections.
Pre Test runs once prior to Main Test and can be used for:
Main Test runs one or more times based on the number of iterations defined by the test vectors. SystemTest will execute each test element in Main Test in sequence once during each iteration.
Post Test runs once after all Main Test iterations are complete or when a test stops during execution. It can be used for:
The Run Status pane indicates which section is executing and how long the test has been running, and provides cumulative Pass/Fail status for the test iterations. After the test is complete, the pane displays the location of any generated files, any error messages generated during the test, and the start and stop time.
When used with Simulink Verification and Validation™ (available separately), SystemTest lets you view model coverage reports. These reports let you see how multiple iterations increase model coverage. Results are reported in HTML documents using industry-standard metrics for structural coverage.
SystemTest combined with Parallel Computing Toolbox™ allows you to distribute your test to multiple processors. These processors can reside in one multiprocessor computer or, when the toolbox is used with MATLAB® Distributed Computing Server™ (available separately), on a computer cluster. You can distribute Main Test iterations as tasks, which are performed simultaneously by different processors.
To distribute a test, you select a check box in the Distributed tab of the Test Properties pane. You can also define any files that your test depends upon using the File or Path Dependencies tables. No changes to the existing TEST-file or additional lines of code are necessary. When you run a distributed test, the test executes in this way:
Distributing a test to multiple processors or machines can speed up the total time a test takes to execute. This can enable you to run more test cases in the same amount of time as a test run on one processor.
The Test Results Viewer helps you manage, visualize, inspect, and analyze the data in your test results using multiple plot types, including line, surface, scatter, time series, waterfall and image.
For post-test analysis, you can:
Within these plots, you can distinguish the effects of test vectors on the result set using color, data markers, and subplots. Plots let you graphically select data associated with test iterations. Details associated with the selection display in the Current Iteration pane.
The Test Results Viewer lets you use multiple plots simultaneously to examine different views of data generated from the same test.