puzzling parfor error during large calculation

Hi there

I am running a large number of differential evolution optimization runs on a 4-parameter problem, for different input control parameters. I am using parfor to speed up things significantly. However I have experienced some weird error instances which, given that I have tested the whole thing as a normal for-loop, suggests that the parallel-toolbox is experiencing problems. The error in question is:

Caught "std::exception" Exception message is:
fl:filesystem:PathNotFound
Error using distcomp.remoteparfor/getCompleteIntervals (line 138)
An unexpected error occurred during PARFOR: Error in remote execution of
remoteParallelFunction : RUNTIME_ERROR

Error in runnested_eddy_Lee_DEparamopt_mainprog (line 23)
parfor ii=1:length(NPop)

Does anyone know what this means? I gather that a file associated with the parallel run is missing but I don't know how to fix that, if that is the case, and google gives me no clues as to the cause/solution of this problem.

Cheers

Jakob

Answer 1

[Jakob Sievers]

I think I may have solved the issue. I dug deep into the optimization code and discovered that it saves a temporary file, and deletes it again following the calculation. Throw that into a parfor loop and the file is created and deleted by separate processes and the whole thing goes wrong. I attached a time-signature to the filename during its creation and the problem seizes to be relevant.

[Mission]

I am stuck in the same problem. But the method you propose does not work. Could you give me more detailed information. Thank you!

[Rinu]

I had the same problem, and in my case it seems the problem was caused because I open different files using the same filename in every loop and did not close the file before exiting the loop. Using fclose within the loop solves the problem.