How to pass structure data into parfor loop

조회 수: 40(최근 30일)
Xiaohan Du
Xiaohan Du 2016년 11월 23일
편집: corentin 2021년 1월 8일
Hi all,
I have a simple parfor test code like this:
inpt.a1 = rand(100);
inpt.a2 = rand(100);
parfor i = 1:10000
inpt_pass = inpt;
otpt = sin(inpt_pass.a1);
end
I wrote inpt_pass in order to pass the structure inpt into the parfor loop. However, if inpt is a very large structure, passing it into every parfor loop would be cumbersome, and cost loads of storage.
If I do:
inpt.a1 = rand(100);
inpt.a2 = rand(100);
parfor i = 1:10000
inpt_pass = inpt.a1;
otpt = sin(inpt_pass.a1);
end
which only pass part of the structure into parfor loop (ideal), I got the broadcast variable warning.
So how can I do it? Many thanks!
  댓글 수: 1
corentin
corentin 2021년 1월 8일
편집: corentin 2021년 1월 8일
I am in a similar situation but I would like to pass to each worker a different part of the struct -- i.e. the content of a given field and only that content (not the whole struct). Following the example above, I'd like to write something like:
inpt.a1 = rand(100);
inpt.a2 = rand(100);
% ...
inpt.aN = rand(100);
parfor i = 1:N
fieldname = sprintf('a%i',i);
inpt_pass = inpt.(fieldname);
otpt = sin(inpt_pass);
end
Is it true that the entire struct is not passed into the parfor loop in that case, thus saving memory? If the answer is negative, what alternatives do I have?

댓글을 달려면 로그인하십시오.

채택된 답변

Brendan Hamm
Brendan Hamm 2016년 11월 23일
편집: Brendan Hamm 2016년 11월 23일
You can simply extract the data that you want to pass to the workers:
inpt.a1 = rand(100);
inpt.a2 = rand(100);
A1 = inpt.a1; % Only inpt.a1 is sent to the workers
parfor i = 1:10000
otpt = sin(A1);
end
Now there is no extra overhead of transferring data. This of course assumes a 1-by-1 vector. If you have more elements in your struct you could just make a copy of only the data that needs to be sent:
inpt.a1 = rand(100);
inpt.a2 = rand(100);
tempVar.a1 = inpt.a1;
parfor i = 1:10000
% You will still have a warning, but you can ignore it as you only pass the data you need.
otpt = sin(tempVar.a1); %#ok<PFBNS> <-- Ignore the warning
end
If you get an error about how MATLAB cannot classify the variable (common with structs), place the body of the parfor loop into a local function and this can overcome this issue easily.

추가 답변(1개)

Sean de Wolski
Sean de Wolski 2016년 11월 23일
편집: Sean de Wolski 2016년 11월 23일
Use a cell array instead of a structure - they can be easily sliced by index. Or, less preferably, in newer releases turn the structure into a parallel.pool.Constant so that it persists on the workers.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by