Folder structure for matlab unit tests

This question is more of an opinion, than an actual question.

I have a library of a lot of different functions that have been developed in matlab over the years; some are implemented as a class and some are just normal matlab functions. They are arranged in a highly hierarchical folder structure.

I am trying to write test code using the munit framework in 2013+; what is the the best way to arrange the test functions for the unit test in the folder structure?, the most logical construct to have a sub directory or file in the directory with the low level function, but if that low level function is a class (with '@FolderName'); but you can't put the test code in that directory (it give an error when calling runtests).I could put a parallel folder structure with the unit tests, but this is a bit confusing.

So the real question is how to organize your unit tests (where to put, the unit tests relative to the implementation code)?

Any thoughts?

[Thorsten]

Your question reminds me of one of my favourite Knuth quotes:

Beware of bugs in the above code; I have only proved it correct, not tried it.

Donald Knuth

[Adam]

I use a parallel folder structure personally and a function I created that creates a template unit test file for me in the right location when I give it the filename of the file (class or function) I want to test. Usually I delete most of the template stuff I added in, but it at least saves me having to find the right location myself and remember the syntax for creating a unit test!

It also means I can more easily run the full suite of unit tests by just recursively building the test suite from sub-folders. Unfortunately for my true unit tests (i.e. fast tests) it takes longer to recursively build the test suite (which is not all that large) than it does to run it which takes away a bit from the idea of running unit tests regularly because they are so fast! I can't remember off-hand how I dealt with packages in my parallel folder hierarchy. I have been rather lazy of late in not really writing many tests around my recent work.

[Andy Campbell]

Nice thoughts @Adam. Note that if you organize your tests into packages it should not take as much time to build recursive suites. Let me know how you fare!

Answer 1

[Andy Campbell]

This is something that is highly subjective and many people will have many varied opinions. I have found over the years that no matter how much thought I put into folder structure, it always ends up not being ideal for some circumstances. Indeed, leveraging TestTags you might be able to get away from a hierarchical folder structure and tag certain tests so that you can select and filter them as desired, whether that be by feature or by type of test (unit, system, etc).

However, if you are just concerned with your development workflows (i.e. keeping the tests in some close or known location relative to the source), here are a couple options, I'd say choose which one works best for you.

1. Put the test in a parallel folder structure to your source. In this sense they are "far" from the source but the tests are easily separable and still can be found easily.
2. Put the tests in a subfolder of the source. This keeps the tests close to your source but if you ever want to get "just" the source you'll probably want to remove the test folders which would be a little work. As you mentioned, if something is in an @-folder, I would consider the containing folder of the @-folder the place to put the tests.

Actually, you may want to consider putting your tests into a package. This way, they are still separate from the source (they always must be prefixed with "test" for example) and you can always choose to move them wherever you want as long as the package base folder is on the path. For example, if your source file foo is on the path, if you put your test for foo into a "test package then you can run it by saying:

runtests test.TestForFoo

which will work regardless of where you choose to move the test package and the TestForFoo test within that package. You can also say:

runtests test

To run all the tests in that package. Finally, organizing tests into packages that are on the path gives you a performance boost since if they are organized in folders those folders are cd'ed into and added to the path at test runtime, and both of these operations are very expensive from a language execution perspective.

Hope that helps!