Unit Testing : Figures

My use case. I have some digraph plots, I'd like to verify the values on certain edges, hold the right values. What I'm doing, is plotting one figure per test, by calling

h = digraph.plot
verifySubstring(testCase, h.EdgeLabel{1}, '1,000,000') 

However, what I'd also like, is then when all the test have fun, to get an overview, of each.

Normally, I'd try something like this;

alignFigures(figs)

Where align figures comes from the matlab file exchange, and figs is a list of the figures created through each of the independant unit tests.

What I can't figure out... is how to persist the figure references, through the test cases...

Any ideas? I realise this sort of defies the point of independant unit tests... but it would be quite helpful, in this specific use case.

[Adam]

How are you creating your test cases? It's a while since I've written unit tests in Matlab, but when I did I always used the class method where each test for a given object was a function of a test class object. If you do this then anything you store as a property of the test class object will persist throughout all its tests until the test object is destroyed.

[Simon Parten]

That's actually what I've been trying... can't get it to work :-( ...

    properties
       figs        
   end
       
    methods(TestMethodSetup)
        function createFigure(testCase)
            f = figure();
            testCase.figs(end+1) = f;
        end       
    end              

But for the life of me, I cannot get this to work.

[Alex Kashuba]

@Simon, this will never work. The reason is the mechnism of the TestRunner.

If you have 5 test in a test class, then TestRunner creates 7 instances of the Test class and in each instances runs only one test. This is done so in order that tests (that can run in random order would not interact)

Therefore all class properties are 'bare'.

Answer 1

[Andy Campbell]

Does the approach outlined here work for your case?

https://blogs.mathworks.com/developer/2018/07/30/semi-automated-testing/

[Simon Parten]

For those interested, here is the skeleton of the solution I ended up with, based on Andy's documentation. For my use case, this is fantastic. Test class sketch

    properties       
        testName        
    end
    
    methods(TestMethodSetup)
        function createFigure(testCase)
            figure('Renderer', 'painters', 'Position', [50 50 1800 900]);
        end
    end
    
    methods(TestMethodTeardown)
        function sortFigure(testCase) 
           set(gcf, 'Name', testCase.testName);
             testCase.addTeardown(@(testCase) testCase.log(3, ...
                matlab.unittest.diagnostics.Diagnostic.join(testCase.testName, ...
                matlab.unittest.diagnostics.FigureDiagnostic(gcf))), testCase);            
            close(gcf)
        end
    end
            
    methods(Test)
        function test_SingleLossNATC_Ins(testCase)

            h = aModel.plotMean;
            verifySubstring(testCase, h.EdgeLabel{1}, '1,000,000');            
            testCase.testName = "NA TC Single loss of 51m";

        end
    end
    

And then the 'runner' setup ...

    
import matlab.unittest.plugins.TestReportPlugin;

runner = matlab.unittest.TestRunner.withTextOutput;
mypath = strcat(pwd, '\structureReport');
runner.ArtifactsRootFolder = mypath;
runner.addPlugin(TestReportPlugin.producingHTML(mypath, 'Verbosity',3));
runner.run(testsuite('testStructure'));
        

Answer 2

[Steven Lord]

I believe the unit testing infrastructure tries not to let you do that sort of thing, as it means that you don't really have a four phase test. You're performing Setup, Exercise, and Verify but skipping Teardown. That could lead the software under test to be in a different state than it should be at the start of the next test, and could allow information to "leak" from one test method to another as you already mentioned.

If you were doing this to capture the figure when the Verify phase of your test method fails (to aid in debugging) you could use a FigureDiagnostic. If you wanted to capture all the figures in case any failure occurs then delete the saved figures if all the test methods passed, you could use a TemporaryFolderFixture with 'PreservingOnFailure' set to true then save the figures as .fig files in that temporary folder during each test method. If any failure occurs, the temporary folder will be spared; otherwise it and its contents will be deleted when the test finishes execution.

There are a couple other potential approaches I can think of, but before I mention them I'd like to know a little more about how you're planning to use all those figures.

[Simon Parten]

I think this is the answer I'm looking for - basically in this case, it appears I want to violate one of the principles of unit testing... Probably, that means I'm trying to do the wrong thing!

Basically, I wanted a quick visual (non persistent) summary (otherwise I have to clear it all up!) of each test at the end of the run. I think I can live without this for the time being.