Produce a diagnistic once per failed test method?

Is it possible to produce a diagnostic once per failed test method (as opposed to once per verify/assert, which is what TestCase.onFailure does)?

A common scenario in my test code is that I read some input data, process it and make several testCase.verify... on the processed data. If one or more verifications fail, I wish to produce a diagnostic message with the input data, but only once per test method to not pollute the log.

[Adam]

Would this not just be a case of using regular Matlab conditional checks for all your intermediate cases and then using a single verify on an ANDed combination of these? Possibly with shortcuts in place if you want it to happen as soon as you hit a failure rather than running all your verifications.

Answer 1

[Alexey Bakhirkin]

I think I found a way to do it using existing callbacks. The idea is to create a base class for the test case with a flag telling the test case status and update this flag in the onFailure callback. Then during the run of the test I can add messages or callbacks to a collection and display/call them in the addTeardown callback.

classdef TestCaseBase < matlab.unittest.TestCase
    properties
        Failed = false
        MethodFailFns = {} 
    end
    
    methods
        function setFailed(testCase)
            testCase.Failed = true;
        end
        
        function runMethodFailFns(testCase)
            if testCase.Failed                
                for i = 1:numel(testCase.MethodFailFns)
                    fn = testCase.MethodFailFns{i};
                    fn();
                end
            end
        end
        
        function addOnMethodFail(testCase, fn)
            testCase.MethodFailFns{end + 1} = fn;
        end
    end   
    

    methods(TestMethodSetup)
        function setup(testCase)
            testCase.onFailure(@()setFailed(testCase));
            testCase.addTeardown(@runMethodFailFns, testCase);
        end
    end
end

Answer 2

[Steven Lord]

If you want to store your data, why not use a FileArtifact diagnostic? You could put it in a temporary folder created by a TemporaryFolderFixture (with 'PreservingOnFailure' set to true, so the folder is not deleted if a failure occurs) so you'll have access to it after the test finishes executing.

Or just skip the diagnostic and write your files to a TemporaryFolderFixture's folder, or a WorkingFolderFixture's folder, etc. If the test passes, that data is no longer needed and gets cleared when the fixture is destroyed. If it doesn't, and you've told the fixture to spare the folder on failure, the test should show you exactly where the data is located.