how to run Gpu and Cpu in parallel in for loop?

for i=1:10
    A(:,:,i)=rand(5,5);
    [q1,r1]=qr(A(:,:,i));
end

I wanted cpu and gpu's to communicate with itself about the loop index. For iteration 1, cpu is working, gpu1 should also simultaneously work on iteration 2. gpu 2 should also simultaneously work on iteration 2. If gpu1 completes its task early, it should start iteration 3. When cpu is done with iteration 1, it should proceed with iteration 4 and so on. How should we implement this. The cpu, gpu1,gpu2 should only communiate about the loop index 'i' and not the A matrix. Once cpu or either of gpu's completes its iteration, it should parallely update the output array. How should this be implemented ?

[OCDER]

Homework? Looks very similar to this post...

https://www.mathworks.com/matlabcentral/answers/439731-how-to-use-gpu-and-cpu-simultaneously-in-a-for-loop

[Walter Roberson]

https://www.mathworks.com/matlabcentral/answers/439731-how-to-use-gpu-and-cpu-simultaneously-in-a-for-loop

is this a homework assignment or did you repost?

[madhan ravi]

Similar to https://www.mathworks.com/matlabcentral/answers/439731-how-to-use-gpu-and-cpu-simultaneously-in-a-for-loop

[Naveen kumar Elumalai]

That question is talking about how each iteration can be run in asynchronous manner. Here, I needed a partial communication internally about the loop index but not A matrix. So this is a different question. No it's not a homework @walter roberson and @ocder. Yeah its a bit similar @madhan ravi.

[OCDER]

Just curious, why do you want to split the job as such? Do you want GPU and CPU to be treated as indistinguishable processors? I foresee an issue where you must continuously exchange an array from a normal array to gpuArray, which can create a lot of overhead and slow things down.

Example of job tasks you want?

Task#   CPU1   CPU2   GPU1   GPU2
1        X
2               X
3                      X
4                             X 
5               X
6        X
7               X
8        X
9                      X
10              X      
11                            X

[Naveen kumar Elumalai]

Yeah, the above example you referred, is what I want. The conversion of normalArray to gpuArray does bring a lot of overhead. Thats why, I wanted a work around for executing above task.

Answer 1

[Walter Roberson]

You cannot do that with parfor or spmd.

What you can perhaps do is use parfeval to submit tasks associated with various resources. Monitor task completion, and upon completion upon a resource, collect the results and start a new task on the same resource.