GPU + Parallel processing(par for loop)

Is it possible speed up the usual "for" loop in matlab code using "parfor" feature in "parallel processing toolkit" and a GPU system("Nvidia Tesla GPU C2050")? That is, can you create and deploy multiple matlab workers in GPU system just like in a machine with multiple CPU's?

Answer 1

[Jason Ross]

There are a few ways of doing this, explained as follows:

Subject:

Is it possible to use MATLAB GPU functionality on a compute cluster with NVIDIA Tesla cards on each node in MATLAB Distributed Compute Server 5.0 (R2010b)?

Problem Description:

I have a cluster with NVIDIA Tesla GPUs on each node. I would like to know if it is possible use MATLAB GPU functionality with MATLAB Distribtued Compute Server and Parallel Computing Toolbox.

Solution:

The functionality to use GPU Computing with MATLAB on a cluster is supported in MATLAB 7.11 (R2010b). The only limitation is, that each MATLAB worker can at most use one GPU at a time.

There are three use cases possible:

1. Each nodes has only a single GPU and is of a single core processor. In this configuration there will be no issues.

2. Each node has a multi-core processor and a single GPU per node. In this configuration two uses are possible:

• run only one MATLAB worker per each node at a time.
• allow multiple workers to run on the same node at the same time and share the GPU. However, then GPU Computation will be serialized across workers.

3. Each nodes has a multi-core processor and has multiple GPUs per node. In this situation you will need to manually assign GPUs to workers on each node to ensure that each worker gets an independent GPU card. An example is shown below:

matlabpool open 8 
spmd 
gpuDevice( labindex ); 
% customer GPU code goes here 
end 
matlabpool close

Reference: http://www.mathworks.com/support/solutions/en/data/1-DOIZVT/index.html?product=DM

Answer 2

[Walter Roberson]

No, GPU processing has to be done differently than parfor processing.

[Walter Roberson]

Also, if you have multiple GPU, it is only possible to bind one GPU per worker. I would suspect that this implies that different workers cannot access the same GPU, but I am not confident of that being a limitation.