MATLAB Coder: C Version Much Slower Than Mex Version

I used MATLAB Coder to turn my MATLAB code into C code and expected a great decrease in run time, but instead found that my C code runs much slower than both the MATLAB code and the mex code. Over 5 tests, I get the following:

• Tthe MATLAB code took 77.48 seconds on average.
• The mex version running in MATLAB took 46.27 seconds on average.
• The generated C code took 262.35 seconds on average.

All three versions of the code produces the exact same output, so I am confident that each version is working properly. I am running the MATLAB and mex versions in R2023a and the C version in Debian 11 (bullseye) in a docker container. The cmake command I am running is

cmake -DCMAKE_BUILD_TYPE=Release ..

and my understanding is that this enables the C compiler optimisations, but I'm unsure if there are further optimisations I can enable. Before specifying the build type as release, my C code ran about 90 seconds slower still (~350s), which is horrendous compared to the MATLAB/mex version.

These are the settings I am generating my code with:

The full settings I used are attached in config.mat but they are almost all just the default settings.

My understanding is that MATLAB does some computations in parallel automatically, while the generated C code doesn't, so I tried turning on automatic parallelisation in the generated code. This only made the code run slower, however. I'm not sure how the automatic paralellisation works, but when I manually tried to parallelise parts of my code, it also did not run faster, so I did not really expect this to improve anything.

I'm just really unsure why the C version runs so much slower than even the mex version, which I expected to be similar or worse than the C version. Are there settings I can tweak in Coder? Are there further optimisations I can enable on the Linux side of things? If this is normal behaviour, what is the explanation for the discrepency between C and mex?

[dpb]

Builtin MEX functions are handcoded and have been optimized/worked on for years; the Coder is machine-generated code that is portable and generic; it's not terribly surprising it isn't as efficient.

The optimizations object basically turns on specific compiler flags, but those do nothing to markedly affect the code generator itself that converts the MATLAB m-code files to C.

What, if anything, you could do to cause the Coder to generate better, faster code would pretty-much be dependent upon whether you could find ways to improve the m-file code itself to produce a better translation -- but there are no real rules for there other than the generic guidelines in the documentation about optimization that I'm sure you'll have already looked at.

While sometimes one can make things markedly faster by converting m-files to mex; it isn't always the case and when it is possible it generally is the result of knowing how to write efficient C code that bypasses higher-level MATLAB constructs, not just that C is faster than MATLAB. That used to be true, but no longer with all the advances in the JIT compiler; if one is doing mostly numeric operations and the code is vectorized, the base MATLAB m-file code may well run essentially just as fast as optimized C code--it does, after all, boil down to eventually calling the highly-optimized BLAS or LAPACK libraries for basic operations and unless you have such code and help the Coder along in calling the standalone libraries, it will generate base transportable C code instead.

[Nyx]

Thanks! I'll look into writing my m-code in ways that will generate better C code.

[Naman Saraf]

Hi,

Is it possible for you to share your MATLAB code to understand the performance issues.

Thank you,

[Nyx]

Unfortunately, I can't share any code as it is confidential company property. I think linking BLAS and LAPACK into the code generation is a worthwile lead and I will report back with the results.

[dpb]

Note that LAPACK/BLAS won't be magic bullets unless your code/problem is such that it boils down to linear algebraic operations that match up to what routines are in the libraries.

Probably the next place to go would be to examine the performance of the C code itself; "pre-optimizing" before knowing just where the bottlenecks are is likely to be nonproductive. See <Profiiling C Code>

[Nyx]

I think BLAS/LAPACK may be beneficial considering I am using matrix decompositions and computing products with large matrices, which coder seems to have generated C code for instead of calling BLAS/LAPACK.

The original thought was that a compiled language would see performance improvement regardless, hence making code suitable for coder and optimising that code before turning it into C code. Since that doesn't seem to be the case from what you have said, I may have to profile the C code and not just the m-code. The C code is still running at an acceptable speed for my purposes, but it would have been nice to have it run faster than the m-code and mex code.

[Nyx]

I would like to update this post with my results from attempting to implement BLAS. In particular, I installed OpenBLAS. I was also able to set up my code on an Ubuntu machine to compare with the results from running in a Debian Docker container. The following results are the mean run time across 5 tests:

• MATLAB, Windows: 77.48s
• MATLAB, Ubuntu: 78.58s
• Mex, Windows: 46.27s
• C, sans BLAS, Docker: 262.35s
• C, with BLAS, Docker: 311.81s
• C, sans BLAS, Ubuntu: 66.78s
• C, with BLAS, Ubuntu: 101.37s

BLAS was not a silver bullet to solve my problems and instead managed to slow down the program further across both operating environments. To be completely honest, I am not sure why this is the case. I expected that, at worst, BLAS would do nothing, rather than slow the program down further. I am planning on profiling the C code as another user suggested to see if I can pin down where the C bottlenecks are. The results also suggest that running the C code in a Docker container is somehow slowing down the program immensely. The MATLAB code had similar performance on both Windows and Ubuntu, while the C code runs considerably faster on Ubuntu - even beating the MATLAB performance when BLAS is not enabled. No version beats the generated mex code yet - however, I am planning on running the mex version on Ubuntu to add to the collection of results.

Thank you for your suggestions so far and I will update with further results if I have anything interesting to share.

[Christine Tobler]

You may want to try a different version of the BLAS (and possibly also of LAPACK). You can find out the versions that are used by MATLAB on your installation by calling

>> version -lapack

>> version -blas

There are many build options that could affect the performance (e.g., is threading enabled?).

Answer 1

[Harsh]

Hi Nyx,

There can be several reasons for the C code to be slower than MEX version or the MATLAB code depending on your exact code. MATLAB tends to outperform the generated C code when an application predominantly relies on pre-compiled binaries in MATLAB or on tasks that the JIT (Just-In-Time) compiler optimizes effectively.

When generating MEX code, Coder has access to the high-performance libraries used by MATLAB. You mentioned in the comments that you are using matrix decompositions and computing products with large matrices; it explains the discrepancy between C and MEX. When using MEX files, MATLAB can directly leverage highly optimized libraries like BLAS, LAPACK, or Intel's MKL for matrix operations. Also, MATLAB handles memory allocation and deallocation efficiently, whereas C requires manual management, which can introduce overhead and inefficiencies.

You mentioned that you tried to parallelize parts of your code but that only worsened the run time. This could be because the number of calculations within each parallel region are too small. Parallelization introduces overheads that can cancel the benefits of a parallel execution in such cases.

Finally, to improve the generated C code run time you can link the C code to optimized BLAS and LAPACK libraries. This can significantly speed up matrix operations and decompositions. Also minimize dynamic memory allocation and deallocation within performance-critical sections. Use stack allocation where feasible and ensure data locality to improve cache usage. You may check the following page for more information - https://www.mathworks.com/help/coder/ug/optimize-generated-code.html#:~:text=MATLAB%20Coder.-,Strategies%20to%20Optimize%20Speed,-Use%20this%20table

I hope this answers your question. Happy coding !

[Nyx]

Thank you. This answers the original question of "why is the mex version faster than the C version", so I will take it as the accepted answer. However, linking BLAS did not improve performance in this case. I wrote a more detailed comment under my main post.

[Brian]

I know this is late, but the reason may be tied to Row-Major versus Column-Major Ordering. I see a 4x speed improvement in MATLAB code when I use the default Column-Major ordering. However, C/C++ uses Row-Major ordering default. The C Coder and GPU Coder by default generate Column-Major ordered code. This could be the reason for your decreased performance. You can tell C & GPU Coder to use Row-Major ordered output. I would try that and see if that fixes your problem.