how I can run my CPU at 100% usage with parfor loop?

Hello,

I have a Matlab program where I use a parallel for-Loops (6 workers). The Computation time is fast compared to using for loop, but during the calculation, when I verify the performance I found that the program only uses about 60% -70 % of CPU. How to fully use the CPU of my PC?

Thank you

Answer 1

[Walter Roberson]

To fully use the CPUs with parfor

1. open the parpool explicitly instead of waiting for it to be open automatically
2. write your code so that it does not pass any variables into or out of the parfor loop, so that no memory needs to be transfered between processes
3. write your parfor code so that it does not do any file I/O or interaction with any device

In other words, to get 100% CPU, it will be necessary for you to write code that does nothing useful, wasting its time doing calculations that it never reports the results of in any way.

If you want to get results back from the parfor workers, then you cannot get 100% CPU.

The less data that you transfer, for the longer computation per iteration, the closer you can get to 100% CPU.

[K. Taieb]

Thank you very much

Indeed, in each iteration, the program allows to obtain a matrix and save it as an image. There are not memory needs to be transferred between processes.

How I can open the parpool explicitly?

like this ?

delete(gcp('nocreate'))
parpool('local',6)

[Walter Roberson]

For closer to 100% CPU stop saving the images.

[Walter Roberson]

At the very least, send as little to disk as you can. That can include using compression methods in memory and writing out the compressed version. The data you write out in the parfor loop does not need to be in image format: you can run a cleanup loop afterwards.

Oh yes: for maximum efficiency, have each worker output to a different disk controller (not just a different disk for the same controller -- that that is better than using the same disk for all output files.) Disk I/O is relatively slow, and you are going to encounter disk contention.