필터 지우기
필터 지우기

parfor load balancing chunksize

조회 수: 6 (최근 30일)
arvid Martens
arvid Martens 2018년 3월 21일
편집: William Smith 2018년 3월 26일
I have function that uses a parfor (~100 iteration) evaluating another function. However one of the workers is two times faster than the other two (it uses a GPU that is two times faster, than the ones used by the other workers). Suddenly the usage of worker one (the fast one) stops, while the other ones are still calculating a lot of iterations (say 5-10 each). I suspect that the worker one is out of available chunks of the parfor load balancing whilst the other ones are still busy with one of the larger chunks.
Is there a way to change the maximal chunksize to for instance 2 or 3 such that the problem of unexploited resources is circumvented?

채택된 답변

Edric Ellis
Edric Ellis 2018년 3월 22일
parfor offers no means of controlling the chunk size. parfeval allows full control over how you split work up - perhaps you can use that instead (unfortunately, this will require a bit of restructuring of your code).

추가 답변 (1개)

William Smith
William Smith 2018년 3월 26일
편집: William Smith 2018년 3월 26일
This issue of heterogenous workloads (or in your case heterogenous workers) and parfor's lack of proper support for them has come up a number of times over the years.
I solved this in my own domain because I know roughly how long it will process each piece of work, in advance. I then used the 'Longest Processing Time' scheduling algorithm, described at https://en.wikipedia.org/wiki/Multiprocessor_scheduling , to preallocate an array of length parpool().NumWorkers . Each element of the array has multiple pieces of work, which is what my scheduling algorithm optimises.

카테고리

Help CenterFile Exchange에서 Parallel for-Loops (parfor)에 대해 자세히 알아보기

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by