colon operation (:) causes parfor to fail on cell array

The following code causes Matlab to throw subscript assignment error:

s=cell(1,20);
parfor i = 1:20
    s{:,i}=1;
end

However, when the colon operator is replaced with a 1, there is no error:

s=cell(1,20);
parfor i = 1:20
    s{1,i}=1;
end

My guess is this involves a slicing issue somehow. Any insights?

[Sergio Santos]

There is a solution to this. Reshaping whatever you want to send to parfor and turning it into a single column or raw vector. Also make a vector to store the length of each vector you reshape. Then when the operation is finished reshape back into matrix form. Shaping and reshaping is very fast and can be fastly done by a single worker, i.e. no need parallel computing.

Answer 1

[Marshall]

Here's the response I got the other day from Mathworks:

--------------------------------------------------------------------------

Hello

First of all I would like to apologize for the delay in my response. It has been very busy over here.

I was researching on this issue and in order to find a valid explanation for this I collaborated with the experts.

In the following code,

s=cell(1,20);
parfor i = 1:20
     s{:,i}=1;
end

The way PARFOR interprets this is that it is expecting a vector of size 20 on the RHS which could be stored in the vector “s”. Hence, you get a “dimension mismatch” error.

However,

s=cell(1,20);
parfor i = 1:20
     s{1,i}=1;
end

This executes fine because only a scalar value is expected in the RHS for “s{1,i}”.

This is a bug in the execution of PARFOR about how it interprets the various variables and its assignments. I would like to thank you for bringing this to our notice. I have captured this in a bug report and forwarded it to the developers for future consideration.

If you have any other questions regarding MATLAB, please feel free to reply to this email and I would be happy to assist you.

Thanks Anuj

--------------------------------------------------------------------------

I'm a bit skeptical of their reason; I don't know why parfor would expect a vector of size 20, when the dimension along the colon operator is only length 1. Unless that is the bug that they're fixing.

Answer 2

[Cedric]

It has a priori nothing to do with PARFOR; the expression involving the colon on the left hand side is a comma separated list (CSL), so you cannot have the scalar 1 on the right hand side. Try it with a simple FOR loop and there will be no difference.

I don't know exactly what you want to do with that, but I guess that it is something like

 s(:,i) = {1} ;

[Jan]

It is at least strange, that the comma-separated list on the left side with only 1 element differs from s{1,i}.

[Cedric]

I don't find it too strange in the sense that the compiler must check whether the operation is legal type/structure-wise and not whether an illegal operation (type/structure-wise) could still be performed because a singleton dimension is involved and there is therefore no ambiguity.

[Marshall]

Hi Cedric,

I think you're on the right track, but Matlab is still inconsistent. The code runs perfectly fine using just a for loop. For some reason, Matlab resolves the colon (:) operator when the dimension being accessed is a singleton, but only in for loops, and not in parallel loops.

For example, the following code runs fine:

c=cell(1,10);
c{:,1}=1;

Places a 1 into the first element on c, because the CSL consists of a single element. The following, however, fails:

   c{1,:}=1;

The compiler understands during a normal for-loop that the former works, but a parfor does not.

[Cedric]

Hmm, you're right; yesterday I made a quick test on a distant machine before answering, which failed with the simple FOR loop, but I probably made a mistake because it's working on my laptop now (not the same setup though, so I should retry on the distant machine when I have time).

I would definitely be interested in the answer to this question then. It might be worth asking Mathworks directly if you get now clear answer here.

[Marshall]

Hi Cedric,

Thanks for your interest. So Matlab returned a response with a rather simple reason that I don't believe is the correct one (they claim that the problem lies in the fact that a parfor slices by row, and not by column, and in my example, Matlab could not verify independence of my columns).

The reason I don't believe this to be true is that the following code fails for seemingly the same reason:

y=zeros(10,10);
parfor i = 1:10
    y(i,:)=1;
end

In addition, many of the iterations of the loop do succeed, as evidenced by what happens when you print the iteration:

y=zeros(10,10);
parfor i = 1:10
    fprintf('%g\n',i);
    y(i,:)=1;
end

The result is the following (I have 4 matlab workers open):

>> tmp
6
5
2
1
4
3
Error using tmp>(parfor consume) (line 1)
Subscripted assignment dimension mismatch.

I'll be happy to keep you updated.

[Cedric]

Thank you for the update! I will think about it a bit more about it and perform a few tests on my side.

[Marshall]

No problem. I believe the issue is semaphore related, since I get an error from parfor's consume value. Check this out:

function tmp 
y=zeros(25,25);
parfor i = 1:25
    fprintf('%g ',i);
    y(i,:)=1;
end

(note that in a parallel loop, each worker is assigned its own stdout line, so each line of output comes from a single worker)

The result is this:

>> tmp
14 13 12 11 
10 9 8 7 6 
5 4 3 2 1 
17 16 15 

Error using tmp>(parfor consume) (line 1)
Subscripted assignment dimension mismatch.

Error in tmp (line 14)
parfor i = 1:25

Caused by:
    Subscripted assignment dimension mismatch.

So...it's not a syntax or logical issue, it's an issue where the one of the workers is running into a problem beginning one of the iterations. The Mathworks application support engineer is looking into it.

Answer 3

[Ben]

please tell us they working on a patch. i've come across this bug too now in 2013a