Can a parfor loop be restarted?

I am processing large amounts of files. Each file might take thirty minutes to an hour. Here's the code snippet on the high level:

parfor firstfile:lastfile
  [morestuff1, morestuff2] = runfunction(stuff1, stuff2);
end

Problem is that every night I leave it running on the server, something wrong happens and the parfor doesn't complete. I want to be able to catch a loop that fails for any reason and restart it later. Here's what I am proposing as the solution:

files = firstfile:lastfile;
while length(files) > 0
  parfor currentfile = files
    try
      [morestuff1, morestuff2] = runfunction(stuff1, stuff2);
      files(files == currentfile) = [];
    catch
      % not sure what to do here
    end
  end
end

Answer 1

[OCDER]

Perhaps you can save the input variables used for your runfunction into a separate cell array. Try this for example:

ErrorFile = cell(1, 1000);
parfor k = 1:1000
    stuff1 = rand(1);
    stuff2 = rand(1);
    try
        assert(stuff1 > stuff2, 'OOPS: Error in this function!'); %Pretend this is the "runfunction". If there's an error, record the input variables.
    catch Msg
        ErrorFile{k} = {Msg, k, stuff1, stuff2}; %Save the iteration #, variables, and error message 
    end
end
ErrorFile(cellfun(@isempty, ErrorFile)) = []; %This stores variables required to redo parfor with errors
cellfun(@(x) display(x{1}), ErrorFile); %Display all the error messages