Filtering test cases out of a test suite

Hello,

I am writing a script that takes in a matlab.unittest.TestSuite object that contains several test cases from tests (call this testSuiteBeforeFiltering). I am then building a second TestSuite object of test cases that I would like to filter out of the original testSuite (call this invalidTests).

What would be desirable here is if there was an easy way to compare the two TestSuite objects, and get a new TestSuite of only the tests between them that are different. I came up with one way to do this, but it seems to break if the TestSuite that is passed in has had any paramters overwritten. This is close to what I think would work:

% Valid tests are tests within testSuiteBeforeFiltering, but not within invalidTests

% testSuiteBeforeFiltering and invalidTests are both TestSuites

suiteAllTests = {testSuiteBeforeFiltering.Name}';

suiteFilteredTests = {invalidTests.Name}';

% Find the test names that are different, then create the test suite

goodTests = setdiff(suiteAllTests, suiteFilteredTests);

testSuite = testsuite(goodTests);

However, the testsuite() function errors out if any parameters have been given a new value. This error message gets generated:

"

Incorrect number or types of inputs or outputs for function createSuiteFromName.

Error using matlab.unittest.internal.createTestSuite (line 21)

"ExampleDirectory.ExampleDirectory2.ExampleDirectory3.UnitTestName[ClassParameter=ClassParameter#ext]/TestMethodName(TestParameter1=1,TestParameter2=1)"

is not recognized as a valid test entity.

"

It seems like the part it does not like is the "ClassParameter=ClassParameter#ext". I was wondering if there is a way to create a TestSuite that contains only the elements that are different between two other TestSuites. Thanks!

Answer 1

[Jacob Mathew]

Hi Liam,

You can create two testsuites and then use the setdiff function to identify the indices of the testpoints that you and indexing them alone to create the filtered testsuite. Here is an example:

% addNumbers.m is the function we are testing      -|
% TestAddNumbers.m contains all the tests          -|->Attached with answer
% unwanted.m contains the tests that we don't need -|

% Creating the test suite for TestAddNumbers & unwanted
suiteAddNumbers = matlab.unittest.TestSuite.fromFile('TestAddNumbers.m');
suiteUnwanted = matlab.unittest.TestSuite.fromFile('unwanted.m');

% Extract the names of the test cases in each suite
% The names are extracted as 'testFileName/testPoint' 
addNumbersTestNames = extractAfter({suiteAddNumbers.Name}, '/')

addNumbersTestNames = 1x3 cell array
    {'testPositiveNumbers'}    {'testNegativeNumbers'}    {'testMixedSignNumbers'}

unwantedTestNames = extractAfter({suiteUnwanted.Name}, '/')

unwantedTestNames = 1x1 cell array
    {'testPositiveNumbers'}

% Find the indices of tests in suiteAddNumbers that are not in suiteUnwanted
[~, uniqueIndices] = setdiff(addNumbersTestNames, unwantedTestNames, 'stable')

uniqueIndices = 2×1

2 3

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

% Create the modified test suite using the unique indices
modifiedSuite = suiteAddNumbers(uniqueIndices);

filteredTestNames = extractAfter({modifiedSuite.Name}, '/')

filteredTestNames = 1x2 cell array
    {'testNegativeNumbers'}    {'testMixedSignNumbers'}

You can refer to the documentation of setdiff in the following link:

https://www.mathworks.com/help/releases/R2024a/matlab/ref/double.setdiff.html

[Liam]

This is what I was looking for, thank you very much!

Answer 2

[Steven Lord]

I am writing a script that takes in a matlab.unittest.TestSuite object that contains several test cases from tests (call this testSuiteBeforeFiltering). I am then building a second TestSuite object of test cases that I would like to filter out of the original testSuite (call this invalidTests).

What are you trying to do when performing this filtering? Depending on what exactly you're trying to do the selectIf method for matlab.unittest.TestSuite objects may be of use. For example, if the test cases you want to filter out all have a specific test tag that none of the tests you want to keep have, you could use the negation of matlab.unittest.selectors.HasTag to filter out just those test cases. See the third example on the selectIf documentation page for an example of this usage.

[Liam]

Thank you for the reply. I do utilize the selectIf method to build up the Test Suite of tests to filter out (invalidTests). It's a bit of an awkward usage, because the design that these tests run against has different configurations, so the tests to filter out are not always the same. Essentially, the filtering I am trying to do is a blacklist approach of selecting tests I do not want (based on what is available in the design to test), but it seems like the selectIf method is more of a whitelist approach that allows you to choose which test(s) you want.