How to create a plugin that stores the exception from a failing test

Hi,

I am running MATLAB R2014a and I'm running single tests with the command:

result = run(NameOfTestClass, 'NameOfTestMethod')

I can then find out if the test passed or failed by looking at the value of:

result.Passed

and

result.Failed

Is it possible that I can store in the result the reason that the test failed, and the stacktrace?

Would this be via writing a plugin?

Thanks

Answer 1

[Andy Campbell]

Hi Daniel,

There is currently no plugin included with the framework that directly stores this information on its properties. However, this information is accessible if you write your own plugin. What you need to do when writing your plugin is:

1. Implement the createTestMethodInstance and/or createTestClassInstance method on the plugin in order to get a hold of the TestCase instances passed to the test
2. Add listeners to these instances to listen to the possible failure events
3. When the events are triggered, the listeners are either passed a QualificationEventData or an ExceptionEventData which contains this information and can be stored away and referenced later.

Also, it sounds like you really want programmatic access to this information, but just so you are aware you can leverage a different OutputStream to send the textual output from something like the FailureDiagnosticsPlugin. That is you can:

   import matlab.unittest.TestSuite;
   import matlab.unittest.TestRunner;
   import matlab.unittest.plugins.ToFile;
   import matlab.unittest.plugins.FailureDiagnosticsPlugin;

   suite = TestSuite.fromClass(?NameOfClass);
   plugin = FailureDiagnosticsPlugin(ToFile(some_file_on_disk));
   runner = TestRunner.withNoPlugins;
   runner.addPlugin(plugin);
   result = runner.run(suite);

You can also create your own OutputStream such as a stream which just holds all of the output in a property of a class, but it will all be textual information, whereas if you use the listener approach you will be able to access the data as MATLAB objects and data structures.

Hope that helps!

Andy

Answer 2

[per isakson]

See the answers to the question Fail Reasons exportable in the UnitTest-Framework?

[Daniel Wallis]

Thanks for your reply.

I tried the following:

 import matlab.unittest.TestSuite;
 import matlab.unittest.TestRunner;
 suite = TestSuite.fromClass(?NameOfClass);
 plugin = matlab.unittest.plugins.FailureDiagnosticsPlugin;
 runner = TestRunner.withNoPlugins;
 runner.addPlugin(plugin);
 result = runner.run(suite);

which only outputs the stack trace and the reason that the test failed to the Command Window, but doesn't store it in the plugin, result, or runner itself, as documented in the documentation of the FailureDiagnosticsPlugin.

Is there a plugin that exists that can store the stacktrace and reason of the test failing as properties of an object (the plugin perhaps)?

Thanks

[per isakson]

I still run R2013a and I use

    msg_str = evalc('results = run( test_runner, test_suite );');

Andy's words should be final!

Answer 3

[Andy Campbell]

This is probably a dup of this question as per isakson points out. Be sure to check there to see some more up to date answers that incorporate new features of the framework, like the DiagnosticsRecordingPlugin.