Why should I start a parallel pool manually?

I want to run a parfor-loop. I can start a parallel pool "manually" with the command

parpool('local', 5);

but I can also directly start the loop with

parfor (i = 1:10, 5)

It is less code (and just as understandable) to use parfor directly. Are there advantages to starting (and then closing) the pool manually? If so, which?

Answer 1

[Walter Roberson]

Are you trying to measure how long the parallel portion of the code takes? Is it fair that the timing for the same code might be very different if the user just happens to run the code within half an hour of the previous parallel run, before the pool timed out?

When you create the pool manually (or use gcp to find a current pool if it exists) then you can be consistent about what you are measuring.

You also need the pool handle for some parallel constructs, such as parfeval to run a future on a cluster or to use background pool

Now suppose you want to retest with 6 pool members instead of 5. If you specified the size of the parfor call then you have to edit every such place.

[Felix Müller]

Ah, so if I were to run two of such loops back to back probably the second would be faster, because the pool is still open from the first loop? Good point!

parfeval is not used here, only one loop. Otherwise I do see the difference to be in the whole "parallel environment".

And finally I do use a variable for the number of cores to use anyway and just made it simpler for this example.

I'll wait a little if other answers come up but otherwise will accept your. Thanks either way!

[Raymond Norris]

The way you're calling parfor, parpool doesn't honor the value 5. For example, I have a 4 core laptop. Notice, in both cases a pool of 4 workers start, regardless of the size of workers I want dedicated to the parfor-loop.

>> parfor (i = 1:10,2), rand; end
Starting parallel pool (parpool) using the 'Processes' profile ...
    Connected to the parallel pool (number of workers: 4).
>> delete(gcp)
Parallel pool using the 'Processes' profile is shutting down.
>> parfor (i = 1:10,5), rand; end
Starting parallel pool (parpool) using the 'Processes' profile ...
    Connected to the parallel pool (number of workers: 4).

[Felix Müller]

Hmm, okay. I have always opened a pool manually. I just found out about this other way and then wondered about the applications. Because I am only ever running one parfor loop only I thought it might be a better way to handle it (if there are no benefit opening a pool manually hence the question).

I got this info from the matlab documentation where it says

"parfor (loopvar = initval:endval, M); statements; end executes statements in a loop using a maximum of M workers or threads, where M is a nonnegative integer."

from https://de.mathworks.com/help/matlab/ref/parfor.html