Assigning GPUs to workers in local cluster

I have a machine with several NVidia TESLA GPUs. I want to run a parfor loop, using a local cluster. Each worker may need to use a GPU. I would like to assign GPUs to workers in such a way that each workers gets exclusive access for one GPU when it needs it, or if there are no free GPUs, the worker waits until a GPU becomes available. It should be something like this:

parfor k=1:N
    b = func1(a{k});  % Perform computations that do not need GPU

    G = getFreeGPU()  % Get a free GPU (wait until available). The GPU is reserved for this worker.
    c = func2(b,G);   % Run some computations involving GPU, using GPU G.
    releaseGPU(G);    % Release the GPU G, making it available for other workers

    a{k} = func3(c);  % Perform more computations that do not need GPU
end|

getFreeGPU and releaseGPU are place holders for the desired capability. They should act like semaphores.

My version of Matlab is R2014b. I run it on Windows 64 bit and Linux 64 bit.

Thank you very much. Razi

Answer 1

[Edric Ellis]

While it might be possible to do what you're asking, it's definitely tricky since PARFOR does not allow direct communication between the workers. I would try simply allowing multiple workers to use each GPU, and to see whether that works well - modern GPUs often work remarkably well when oversubscribed in this way. By default, the workers will select different GPUs to use, so this oversubscription happens automatically (see help selectGPU for more details).