FPRINTF making mistakes when inside a PARFOR loop

Hi there,

I am running a model that writes MATLAB code to a file every time it is run, and then runs that code. This model is run 5 times, separately, so I decided to replace the for-loop for this with a parfor-loop. To avoid mistakes/overlaps with writing to the file, I have made the model write to 5 different files. In summary, the pseudocode is this:

parfor i = 1:5
    fprintf("model"+i+".m", "lots_of_code_gets_written_in_here"); % write the code for the model
    
    if i == 1
        model1(arg1,arg2); % run the model
    elseif i == 2
        model2(arg1,arg2); % run the model
    elseif i == 3
        model3(arg1,arg2); % run the model
    elseif i == 4
        model4(arg1,arg2); % run the model
    elseif i == 5
        model5(arg1,arg2); % run the model
    end
end

This solution used to work perfectly. The parfor-loop would write code to 5 different files and run these 5 files separately and in parallel. Unfortunately, somehow it no longer works. What happens now is that when writing the code to the file, there are mistakes in the code - basic syntax mistakes like leaving out brackets, incomplete lines, etc. As a result, this written code no longer runs and I get an error.

I replaced the parfor-loop with a for-loop again, and everything works fine. There are no errors in the code written by MATLAB. But I would still like to use the parfor-loop because of the increased speed.

Any idea why having a parfor-loop might cause these errors and what I can do about this?

Would greatly appreciate any tips!

Best,

Alex

EDIT: this problem seems to be to do with the fact that these files are being written to a shared hard drive. At least, when I move my files back there, the problem comes back. Does this seem plausible and is there nonetheless a solution?

[Edric Ellis]

Writing to a separate file from each iteration of the parfor loop definitely seems like it should be a safe option, and your pseudocode looks reasonable. Perhaps you could try narrowing things down to the simplest reproduction that actually demonstrates the problem, otherwise all we can do is suggest guesses as to what's going wrong. I suppose the most likely culprit is simultaneous writes to the same file, but your pattern ought to preclude that. Maybe you could fread the file just after you've written it out to ensure it has the correct contents.

[Walter Roberson]

"clear" the name of the function or script to remove it from the parsing cache. Best do that just before you start writing to the file and then again after you finish writing

[Mohammad Sami]

If your code generation part is not taking a lot of time, you can consider moving out to a for loop. This will run first and create your code files. then you can run it in the parfor loop.

[Andrew Janke]

When you say "when I move my files back there, the problem comes back" does that mean the problem occurs even with a regular for loop (and not just in a parfor) when you are writing to files on the network drive?

Your pseudocode does look reasonable. Can you post actual code that will reproduce the issue, maybe from a cut-down version of your program?

What is the model built in? Is it just another piece of Matlab code?

Answer 1

[Alex van Tuyll]

Thanks for your replies!

I ended up using Mohammed's solution and that works.

However, yes, Andrew, the problem still occurs when writing files on the network drive, but only with parfor.

The whole model is purely MATLAB.