Java Exception when running Parfor

Hi

I am trying to run a parallel job in Matlab using parfor on a machine with 12 nodes and 128 GB of memory, so resources should not be a problem. My program uses two very large matrices (communication of which may be the source of the problem). I get following error at the start of parfor loop:

Error using parallel_function (line 598) Java exception occurred: java.lang.IllegalArgumentException at java.util.concurrent.Semaphore.acquireUninterruptibly(Unknown Source) at com.mathworks.toolbox.distcomp.pmode.io.TransmissionChannel$MessageHolder.acquireNecessaryPermits(TransmissionChannel.java:611) at com.mathworks.toolbox.distcomp.pmode.io.TransmissionChannel.enqueueMessageForSending(TransmissionChannel.java:118) at com.mathworks.toolbox.distcomp.pmode.io.TransmissionChannel.enqueueMessageForSending(TransmissionChannel.java:511) at com.mathworks.toolbox.distcomp.pmode.io.CommunicationGroup.sendToChannel(CommunicationGroup.java:289) at com.mathworks.toolbox.distcomp.pmode.io.CommunicationGroup.sendTo(CommunicationGroup.java:253) at com.mathworks.toolbox.distcomp.pmode.ParforControllerImpl.sendInterval(ParforControllerImpl.java:343) at com.mathworks.toolbox.distcomp.pmode.ParforControllerImpl.addInterval(ParforControllerImpl.java:289) at com.mathworks.toolbox.distcomp.pmode.ParforControllerImpl.addInterval(ParforControllerImpl.java:244)

Following is a demo code to give you an idea. Any help would be greatly appreciated.

Best,

Vikrant.

code:

function obj = getobj(P, X, Wi)

% P = 117 x 39 matrix
% X = large matrix 
% Wi = Large sparse matrix

[I, J, V] = find(Wi);

obj = 0;
parfor k = size(I,1);

      i = I(k);
      j = J(k);
      if i ~= j
                  Xi = X(i,:);
                  Xj = X(j,:);
                  fi = sqrt(Xi*(P*P')*Xi');
                  fj = sqrt(Xj*(P*P')*Xj');
                  fij = Xi*(P*P')*Xj';
       obj = obj + (1 - fij/(fi*fj))*(-V(k));

      end

end % END OF PARFOR

end % END OF FUNCTION

PS: There is a reason I have to write the code in the crude way. For example, my matrix X is 1471326x117, that is too big to get Y = X*X';

[Martin]

Hi Vikrant. I'm getting the same error for some of my code. Did you manage to figure this out?

Thanks,

Martin

Answer 1

[John]

Hello Martin/Vikrant, Did you ever find a solution to this? Getting error on 2011b. Don't get the error when small arrays involved, only with large arrays being communicated.

[Walter Roberson]

Arrays to be communicated are limited to 2 gigabytes.