parfor error - variable cannot be classified

조회 수: 1 (최근 30일)
Varoujan
Varoujan 2015년 5월 12일
댓글: Walter Roberson 2015년 5월 15일
I have been using Matlab for a while but I am new to parallel computing. I have a set of .m files that take a long time and after profiling the code and seeing which sections suck of most of the time, I thought perpahs I can take advantage of parallel computing.
Here is one of those sections (sxyt is ~1million, numFrames = 10, numParams is ~ 18).
numParams = length(a);
sxyt = size(xytIn, 1);
DxDyOut = single(zeros(sxyt, numFrames, 2));
osc = single(zeros(sxyt, numFrames, numParams));
mag = single(zeros(sxyt, numFrames, numParams));
t = zeros(sxyt, numFrames);
randomPhase = single(zeros(1, numParams));
randomPhaseOffset = single(ones(1, numParams));
randomPhase = randomPhaseOffset .* rand(1,numParams) .* 2 .* pi;
for k = 1:numParams
osc(:,:,k) = exp((2.*pi.*f(k).*t(:,:) + randomPhase).* 1i);
mag(:,:,k) = a(k) .* (k1(k) + k2(k).*exp(-t(:,:)/(tau(k) + 0.0001)));
DxDyOut(:,:,1) = DxDyOut(:,:,1) + real( jonesx(k) .* osc(:,:,k) .* mag(:,:,k) );
DxDyOut(:,:,2) = DxDyOut(:,:,2) + real( jonesy(k) .* osc(:,:,k) .* mag(:,:,k) );
end
When I turn the for loop to parfor loop, I get "The variable DxDyOut in a parfor cannot be classified" error. I read the Matlab help on parfor and a number of the other submissions but still can't figure it out. In some earlier posts, there were double for loops which were solved by making the outer loop a parfor and leaving the inner one as a for loop to slice the data. I don't get any errors for osc and mag lines. So, this made me think that the error has to do with the fact that I am accumulating the results in the variable DxDyOut. So, I changed the for loop thusly:
A = single(zeros(sxyt, numFrames, numParams));
B = single(zeros(sxyt, numFrames, numParams));
DxDyOut2 = single(zeros(sxyt, numFrames, 2));
for k = 1:numParams
osc(:,:,k) = exp((2.*pi.*f(k).*t(:,:) + randomPhase).* 1i);
mag(:,:,k) = a(k) .* (k1(k) + k2(k).*exp(-t(:,:)/(tau(k) + 0.0001)));
A(:,:,k) = real( jonesx(k) .* osc(:,:,k) .* mag(:,:,k) );
B(:,:,k) = real( jonesy(k) .* osc(:,:,k) .* mag(:,:,k) );
end
DxDyOut2(:,:,1) = sum(A,3);
DxDyOut2(:,:,2) = sum(B,3);
I no longer get the error but this time the loop takes 10x longer to complete when using parfor. I noticed that Matlab worker used all of my 16GB RAM with parfor. A, B, mag arrays are about 750MB each and osc is 1.5 GB, with all the variables in the workspace adding up to ~4.2 GB.
So, I should have enough memory with two workers running. I don't understand why I am running out of memory. Either way, it seems like in cases where large data is manipulated, parallel computing won't help because one runs into memory limits. I would appreciate any help.
A quick update - when both forms of the for loop are run, the results are identical, i.e., isequal(DxDyOut, DxDyOut2) = 1. However when I convert the second form to parfor, isequal(DxDyOut, DxDyOut2) = 0. The first form won't work with parfor.

답변 (1개)

Walter Roberson
Walter Roberson 2015년 5월 12일
Try
A = single(zeros(sxyt, numFrames));
B = single(zeros(sxyt, numFrames));
DxDyOut2 = single(zeros(sxyt, numFrames, 2));
for k = 1:numParams
osc(:,:,k) = exp((2.*pi.*f(k).*t(:,:) + randomPhase).* 1i);
mag(:,:,k) = a(k) .* (k1(k) + k2(k).*exp(-t(:,:)/(tau(k) + 0.0001)));
A = A + real( jonesx(k) .* osc(:,:,k) .* mag(:,:,k) );
B = B + real( jonesy(k) .* osc(:,:,k) .* mag(:,:,k) );
end
DxDyOut2(:,:,1) = A;
DxDyOut2(:,:,2) = B;
  댓글 수: 5
Varoujan
Varoujan 2015년 5월 15일
Thank you ! I tried the "drange" suggestion. It helped but not enough. On my laptop I have a dual core i7 (4 processes with hyper-threading).
Below are the timing I get. Note I have two forms of the for loop in my example above - your suggestion of "A = A + ..." version is faster. The other form doesn't work with parfor. So, in each case I list timing for both forms.
for loop without parallel processing:
Elapsed time is 17.160106 seconds.
Elapsed time is 13.927942 seconds.
parfor loop w/ two workers:
Elapsed time is 16.593184 seconds.
Elapsed time is 29.215239 seconds.
for loop with "drange":
Elapsed time is 15.835708 seconds.
Elapsed time is 13.865574 seconds.
So, while "drange" helps, it doesn't do any better than the for loop without any parallel processing.
Walter Roberson
Walter Roberson 2015년 5월 15일
Could you try it with double precision and see how the speed changes?

댓글을 달려면 로그인하십시오.

카테고리

Help CenterFile Exchange에서 Loops and Conditional Statements에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by