How to reduce the usage of "broadcast variables" when using parfor
조회 수: 63 (최근 30일)
이전 댓글 표시
Hi, in a "parfor" loop, a same warning message is put on three variables which I named "el", "ds", and "vmin". The message is: "The entire array or structure 'el' (or 'ds', 'vmin') is a broadcast variable. This might result in unnecessary communication overhead."
This "parfor" loop requires four variables which were calculated before the loop: Variable "el" is 4-by-378 double, "NElem"=27, "ds" is 27-by-27 sparse double, and "vmin" is 1-by-378 double.
Part of this loop is as follows:
el_row1=el(1,:);
parfor e=1:NElem
k=el_row1-e==0;
ind=el(4,k);
ds_e=ds(:,ind);
vs_e=ds_e*vmin(k)';
...
end
Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please? I read on the various types of variables in parfor-loops in Matlab documentation, I'm guessing the reason these three variables get the warning message is the form of indexing: "k" or "ind" is not a loop variable, so "el", "ds" or "vmin" cannot be sliced. But I don't know how to turn the variables "el", "ds", and "vmin" into slice variables. Any help is appreciated.
댓글 수: 0
채택된 답변
Walter Roberson
2022년 8월 10일
"Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please?"
No. You are accessing the values in random order.
k=el_row1-e==0;
ind=el(4,k);
You can build el4=el(4,:); and index that at el_row1==e so that el as a whole does not need to be sent.
You could findgroups el_row1 and splitapply or similar ahead of time to create cell ind{e} so that the index information could be sliced.
But ds and vmin are going to have to be sliced. Unless, that is, it turns out that the information needed for any given e turns out to be consecutive and can be split ahead of time.
Your arrays are not large at all. I would be concerned about whether they are doing enough work to make it worth using parfor.
댓글 수: 6
Bruno Luong
2022년 8월 10일
el_row1=[1 2 3 2 1 1];
[~,~,J] = unique(el_row1);
i = accumarray(J,(1:length(J))',[],@(x){x});
i{:}
추가 답변 (1개)
Bruno Luong
2022년 8월 10일
편집: Bruno Luong
2022년 8월 10일
You can use parallel constant to convert el, el_row1, ds, vmin to constant and reduce the data transfert, if they are not modified (the snipet of code you showed they are not).
댓글 수: 0
참고 항목
카테고리
Help Center 및 File Exchange에서 Parallel for-Loops (parfor)에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!