Why is parfor repeating loops
조회 수: 6 (최근 30일)
이전 댓글 표시
Dear Matlab community,
I am running 500 simulations, and am using parfor to run several simulations (one simulation on each core) in parallel to finish running the simulations more quickly. However it seems parfor is repeating some of the simulations, unnecessarily increasing the total run time.
Currently I have a folder for each of the 500 simulations containing the inputs. The code navigates to each folder. Then it loads the inputs, and runs the simulation for that set of inputs. Finally it saves the outputs for that set of inputs to a folder unique to that set of inputs (I should end up with 500 output folders). Within each parfor loop there are three additional inputs that may change depending on what I want to test, I select the input I want to use with a for loop. At the moment I am keeping these inputs constant (I only specify one possible option for each of these inputs). I can tell that the code is repeating some of these simulations as for a specific set of inputs I am sometimes getting multiple output files. Specifically (if you look at the pseudo code) it is as if it is testing different weightings, however I have only specified 1 weighting. As far as I can tell repeatition of the simulations is random, of the 92 simulations that completed/started to run 48 of the simulations were repeated and I cannot see any particular pattern in the simulations that have been repeated.
I have written a pseudo code below to demostrate what I am doing.
model = {"model1"; "model2"; "model3", ...};
loadingConditions = "quasiStatic";
costFunction = "sumWeightedAbsoluteErrorSquared";
weights = weight1;
% Do things to check the models, loading conditions, cost functions,
% weights are correctly specified and specify simulation inputs that are
% constant.
% In this I use for loops which use gg, hh, ii, and jj as the index.
numModels = max(size(model));
numLoadingConditions = max(size(loadingConditions));
numCostFunctions = size(costFunction,2);
numWeights = size(weights,1);
% Clear indices used in for loops outside of parfor to clear temporary
% variables
clear hh ii jj
parfor gg = 1:numModels
% Specify inputs for simulations that are constant in this set but I
% might want to change in the future
% Load simulation inputs
for hh = 1:numLoadingConditions
% Import loading condition
for ii = 1:numCostFunctions
% Select cost function to use
for jj = 1:numWeights
% Specify the output folder, based on the input folder, the
% loading condition, cost function, and the weighting
% Select the loading condition, the cost function to use,
% and the weighting to use
% Run simulation
% Save outputs
end
end
end
end
Could this be due to a new parfor loop on one core starting before the previous loop had completed? I did not think this was possible. However as the code failed before completing all the iterations I was able to see the number of iterations currently ongoing when the code failed as each time a loop completed it had to write an output. Of the simulations run 10 had not written the output, suggesting 10 loops were being run at the time the code failed. However the computer has 8 cores, which to me suggests some of the cores were running multiple simulations at the same time.
If anyone has any suggestions as to why and as to how to stop this it would be very helpful.
Thanks,
Samuele
댓글 수: 6
Edric Ellis
2022년 10월 7일
There are certain conditions under which a parfor loop can repeat iterations, but these ought to be rather rare. They are to do with two specific forms of error handling:
- Dealing with missing files that need to be auto-attached
- Handling crashed workers
While running your code, do you see any messages in the MATLAB command window related to analysing and attaching files? If so, you can pre-empty this by using addAttachedFiles to send the code to the workers ahead of time. If the workers are crashing... well, that might be harder to avoid.
답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Parallel for-Loops (parfor)에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!