How to set prefdir for parpool workers?

I am running parpool jobs with 10 workers under 2016b on my Mac (Mac Pro “Twelve Core”; 3.06 GHz 6 Core Xeon X5675 x2).

Initial jobs run without error. Over time, however, workers begin to fail with the error:

Error using prefutils>loadPrefs (line 42)
    Unable to read MAT-file /Users/myname/Library/Application
    Support/MathWorks/MATLAB/R2016b/matlabprefs.mat. File might be corrupt.

After some searching, I found these links. They advise changing the Matlab prefdir prior to starting a parpool job so as to avoid write conflicts within the prefs file:

Case HPC Software guide for matlab parpool jobs

NYU HPC guide

This advice, however, seems to regard the situation in which multiple parpools are running concurrently.

More generally, I am unable to determine how I would arrange for each worker in a parpool to start with its own, separate prefdir, and if this is even needed to fix the error that I am encountering.

Thanks for any help.

Answer 1

[Faiz Gouri]

This error occurs when multiple MATLAB workers attempt to access the matlabprefs.mat file simultaneously. Here is a sample code to reproduce this issue:

function parTest

    if matlabpool('size')==0
      matlabpool('open');
    end

    parfor i=1:10

      pause(randn(1));
      setpref('parTest','test','test');

    end %parfor
  end %function parTest

To avoid this problem, set preferences using "pctRunOnAll" , before the parfor loop.

function parTest

    if matlabpool('size')==0
      matlabpool('open');
    end

    pctRunOnAll setpref('parTest','test','test');

    parfor i=1:10
      ...parfor statements go here...
    end

end %function parTest

Alternatively, the problem can be avoided by increasing the pause time from "pause(randn(1))" to "pause(randn(i/100))" , where 'i' is the parfor counter variable.