Fail Reasons exportable in the UnitTest-Framework?

Hi,

when I run unit tests with the unittest framework.

The result of a test run is a vector of matlab.unittest.TestResult objects.

The objects include an information, if a test fails, but do not contain the information about the failure.

If there is a failure, I want to give this failure information to another application.

Therefore my question: Are the fail reasons exportable in the UnitTest-Framework?

thanks

[Andy Campbell]

When you say the fail reasons, do you mean the "Framework Diagnostics" and "Test Diagnostics"? Alternatively, do you mean the "Reason(s)" that are shown in the failure summary at the end of the run?

[Till Rahlf]

Hi,

when the tests are finished, I get a vector of matlab.unittest.TestResult objects.

Like:

myTestResults =

1x4 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration

Totals: 2 Passed, 2 Failed, 1 Incomplete. 2.5783 seconds testing time.

Where can I see the "Reason(s)" that are shown in the failure summary at the end of the run?

I just see reasons during the test process, do I?

During the test process of the whole test suite, all problems are displayed, like: ================================================================================ Uncaught error occurred in myFirstTest/Test1.

    --------------
    Error Details:
    --------------
    Undefined function or variable 'undefExample'.

I can see the error text in the command window, but the resulting vector of matlab.unittest.TestResult objects does not contain this information. It shows only that a test fails, but not why.

I want to collect all warning and errors during a test suite run. After that I want to export it together with the test results to another software.

Thanks

Answer 1

[Sean de Wolski]

Now with R2014a, you can use custom test runner plugins such as the Failure Diagnostics Plugin and the Output Stream or To File plugin to gather the failure reasons and export them.

http://www.mathworks.com/help/releases/R2014a/matlab/ref/matlab.unittest.plugins.html

Answer 2

[Swati Tiwari]

Now with R2016a, you can use DiagnosticsRecordingPlugin to add diagnostics on test results.

http://www.mathworks.com/help/matlab/ref/matlab.unittest.plugins.diagnosticsrecordingplugin-class.html

If you run tests with the runtests function or the run method of TestSuite or TestCase, the test framework uses a DiagnosticsRecordingPlugin plugin that records diagnostics on test results.

After you run tests, you can access recorded diagnostics via the DiagnosticRecord field in the Details property on TestResult. For example, if your test results are stored in the variable results, find the recorded diagnostics for the second test in the suite by invoking records = result(2).Details.DiagnosticRecord.

Answer 3

[Andy Campbell]

This is related to your other question here. Basically, the core test result does not have any access or remember the reasons for the failure. Once you can extend the runner by implementing a custom plugin, that plugin can access such information. As mentioned in the linked answer though, this is not yet ready for prime time and is currently not a documented interface. It is very likely to change in a future release.

Answer 4

[per isakson]

This line picks up most of the output from the test

    msg_str = evalc('results = run( test_runner, test_suite );');