Internal error with symbolic engine. Quit and restart MATLAB.

Hi guys,

I am using symbolic math toolbox in Matlab 2018b in Win10 64-bit.

The problem is

Error mupadmex

Internal error with symbolic engine. Quit and restart MATLAB.

Error in sym/matlabFunction>mup2mat (line 404)

res = mupadmex('symobj::generateMATLAB',r.s,ano,spa,0);

Error in sym/matlabFunction>writeOutput (line 540)

body = mup2mat(y,false,sparseMat);

Error in sym/matlabFunction>writeMATLAB (line 453)

writeOutput(fid,outputs,f,1,sparseMat);

Error in sym/matlabFunction (line 183)

g = writeMATLAB(funs,file,varnames,outputs,body, opts.Optimize, opts.Sparse, opts.Comments);

Error in sym/odeFunction (line 135)

f = matlabFunction(expr, 'Vars', varsAndParams, matlabFunOptions{:});

Error in the_method_7 (line 180)

combined_mass_matrix_handle1 = odeFunction( Mrr , p_inde( 1 : 16 ) , 'Sparse' , true ,

'Optimize', false , 'File' , 'mass_matrix_file') ;

Are there any solutions to this? Many thanks!

[Walter Roberson]

Experiment with having it not write to a file.

[Tony Cheng]

Hi Walter, could you pls offer me some solutions to fix it?

[Walter Roberson]

I offered a possibility already.

All you have given us is that this involves odeFunction applied to a sparse system. You did not give your code and you did not post your operating system or your MATLAB release. I really do not have time to run through 30 different versions of MATLAB on at least 6 operating systems in hopes that I will happen to stumble on a system of variables that reproduces the problem.

[Tony Cheng]

Dear Walter,

Thanks very much for your detailed explanation!

Actually, the codes are run in Matlab 2018b on Win 10. In my project, the codes are very long ( around 300 rows ) , and the error always occurs nearly at the end of the codes, wasting the long computational time before arriving at the row where errors occur.

I think the codes are all right according to the physical model in the project, that is to say, errors takes place due to the codes themselves.

If possible, could you pls offer me your email or any other channels, via which I can send you the codes?

Many thanks for your help!

[Walter Roberson]

You can post a dropbox or google drive link.

I do not have Windows 10 active at the moment. I keep meaning to link my virtual machine back in but have not gotten around to it.

[Tony Cheng]

Dear Walter,

Thanks so much for your immediate reply! I did not respond to you until now because I was on business on a factory on site for the job. Fininsh it now and very very tired.

In here, we can not get access to dropbox or google. They are not allowed in here...

So could you give me some other channels such as emailbox to send the files to you?

1000 thanks for your help!

[Walter Roberson]

I would suggest to you that you open a support case.

Would it be possible for you to save Mrr and p_inde( 1 : 16 ) to a .mat and attach that? We probably do not need your code, just those, since the problem looks like it is within the odeFunction call.

[Tony Cheng]

Hi Walter,

Thanks very much for your suggestion! Actually, during the time for my business, I have replaced odeFunction by matlabFunction, but it sees that it is very time-consuming! the codes have been running for over 72 hours on matlabFunction!!

For your suggestion about attachment about Mrr and p_inde( 1 : 16 ) to a .mat , I will try it after the computation on matlabFunction!

1000 thanks for your help!

[Tony Cheng]

Hi Walter,

The use of matlabFunction is shown in the attachement.

Many thanks!

[Tony Cheng]

Hi Walter,

The attachments are about Mrr and p_inde(1:16).

Also, the code where the error occurs is:

combined_mass = odeFunction( Mrr , p_inde( 1 : 16 ) , 'Sparse' , true , 'Optimize', false , 'File' , 'mass_matrix_file') ;

Thanks very very much!

Regards

Tony

[Walter Roberson]

That Mrr is taking a quite long time to load the .mat !

[Tony Cheng]

Hi Walter, yes it is. I also spent over 20+ hours to load it in my workstation...... THANKS very much for ur patience!

[Walter Roberson]

If what is happening on my system is anything to go by, I suspect you ran out of swap space and MATLAB aborted because of that. I interrupted mine when it reached 70 gigabytes of memory used.

[Tony Cheng]

Hi Walter, you mentioned swap space. Does that indicate that you are using Linux?

Actually, I am using Win 10. Will the system make an influence?

If not, I think for my Win 10 system, the virtual space on C disk in my workstation will be increased to hold anything needed in the annoying computation...

[Walter Roberson]

I am using Mac.

MS Windows uses a page file, which is configured for a particular size rather than grown indefinitely.

https://www.geeksinphoenix.com/blog/post/2016/05/10/how-to-manage-windows-10-virtual-memory.aspx

With the memory exhaustion I was getting, I figured that the problem might just be too big to do all at once, and that plausibly you could run one conversion at a time, or perhaps a few together. I saved the elements of the 32 x 32 array into separate .mat files so I could reload them individually. I then told it to start converting the largest single one of them by itself.

The conversion ran for about 24 hours, without using excess memory during that time. However, a short time ago, my system started to slow down, and when I investigated I found that the mupad process was using little memory but that MATLAB was suddenly growing quickly in memory. When it got up to about 40-ish gigabytes, it crashed.

It is not expected that the mupad process would use so little memory, so I suspect I might have been looking at the mupad process for a different instance of MATLAB. I suspect that mupad crashed and that triggered MATLAB chewing up a bunch of memory and crashing. I will try converting a smaller one and see what happens.

[Tony Cheng]

I cross the fingers for your trail and I will seek a larger station for the computaion.

40GB thanks for your great help and valuable suggestions!

[Walter Roberson]

Mrr(30,30) was successfully sent to file. It took about 16 1/2 hours of computation. I do not know what the peak memory was; when I went to bed about 5 1/2 hours in, it was using about 10.48 gigabyte and had barely grown in the previous 2 hours (after a period of growth just before that.)

[Walter Roberson]

The conversion of Mrr(30,30) by itself is over 300 megabytes even though the .mat to store it is less than 31 kilobytes.

I was able to get the symbolic engine to write it out in a different format in "only" 50 minutes. It needs a bit of post-processing to use, but at the moment it looks like some auxillary functions might be enough, such as

function etc(various_arguments, qrf_dof_d3_in, qrf_dof_d4_in, more_arguments)
Im = @(v) imag(v);
conjugate = @(v) conj(v);
qrf_dof_d3 = @(t) qrf_dof_d3_in;
qrf_dof_d4 = @(t) qrf_dof_d4_in;

It is clear, however, that each evaluation of the system would take a fair bit of time, as there is probably a few gigabytes of expressions there. There are a lot of redundant calculations and the efficiency would probably be greatly improved by using the expression optimizer -- but the expression optimizer is very slow. It might be easiest to do the expression optimization in post-processing using text substitutions, such as calculating sin(qrf_dof_d4(t)) once and storing it in a variable and substituting that variable into the text of the expression.

[Walter Roberson]

By the way: would it be reasonable to expect that all of the inputs are real-valued? That could cut down on computations.

[Tony Cheng]

Dear Walter,

Thanks very very much for your great painstaking efforts, long time and patience for my problem! I really do not know how to express my appreciations to you in English or Chinese. Really beyong my speaking language...

Well, for this lenghty computation, I firmly believe that:

1. The computational procedure must be improved and simplified to save time, otherwise, the results can not be applied in the next stage in the project which needs realtime implementation.
2. Or, if what we want at the current stage is just the numerical values, we need a more powerful PC to do it.

Also, all our inputs are real-valued, we use " .' " to avoid the production of "conj" when transpose of a matrix is computed. Are there any other ways to avoid the unnecessary, annoying and time-consuming "conj" in our project?

BTW, I notice that in https://au.mathworks.com/help/symbolic/massmatrixform.html?s_tid=srchtitle

about massMatrixForm, matlabFunction is applied to compute ODEs.

Also, odeFunction is used to solve ODEs.

Here I am very curious, what the difference is using matlabFunction and odeFunction to reach the numerical solutions of ODEs? Is there any difference in accuracy, computational cost( e.g., time, RAM etc. ), or anything else?

In the end, 300 GB thanks to you, your Mac and your words!

[Walter Roberson]

Although the inputs might be real-valued, I see that the calculations involve complex values, and include taking real and imaginary parts of various expressions. It might be easier to leave the conj() in place.

I was able to write all of Mrr(30,30) to file in optimized form in slightly under 5 hours. The result was a much more reasonable 128 kilobytes instead of 325 megabytes. Execution time was about 0.0002 seconds per invocation. Without the optimization, execution time is about ... ummm, pretty long, I will update when I can manage to get a timing.

Your expressions have a mix of sizes, with some being constants (.mat file about 720 bytes), and some being small calculations (.mat file about 5 to 8 kilobytes) and some being large calculations (.mat file between 30 kilobytes and 60 kilobytes.) Mrr(30,30) is one of the smaller large calculations, with .mat file about 30 kilobytes.

This hints that although it might be time consuming, that it might be practical to write out the various entries one by one in optimized form.

However, when I looked at the generated code, I was reminded of a bug that I found in generating optimized files with matlabFunction: it does a whole bunch of calculations and messes up the final result, returning one of the intermediate results instead. At least as of R2019a, the result is not usable, at least not for long expressions such as these.

The difference between matlabFunction and odeFunction for this kind of input is that odeFunction converts function references such as qrf_dof_d3(t) into variables such as qrf_dof_d3 . After doing that, it passes the resulting expression into matlabFunction to do the code generation.

Would it be practical for you to upgrade to R2019a? There are some additional symbolic functions in R2019a that can make it easier to preprocess your functions for work-arounds.

[Walter Roberson]

The failures on writing optimized files are case 03593372

[Tony Cheng]

Hi Walter, today I have an idea in this trouble.

Because the symbolic expression of this problem is extraordinarily lengthy and complicated, and we have the initial values of the symbolic variables, Could we transfer them from symbolic expressions into numercial expressions before we use odeFunction, as shown in the attachment pdf file?

If yes, the computational burden can be alleviated a lot when sending relevant matrixes into odeFunction .

Many thanks for your great help!

[Tony Cheng]

For R2019a, I will try to find some access to it from my workmates. I will try this new version definitely.