Multiple Parallel Jobs on Server

조회 수: 5 (최근 30일)
Shumao Zhang
Shumao Zhang 2022년 2월 10일
답변: Edric Ellis 2022년 2월 11일
Hi,
Let's say I have a script "main.m" which uses the parallel computing toolbox for parallelization. I have a server (Mac) with 20+ cores of cpus, and would like to run multiple "main.m" jobs at the same time for parameter searching. Each "main.m" job only needs 4 cpus. I opened multiple instances of Matlab and ran one job in each instance. However, I found that the speed of each job slows done significantly (compared to just run one single job). I guess it might because some cores are used by more than one job. Suppose 4*number of jobs < 20, what would be a suggested way to achieve the goal? Is there a way to ensure that each job will use different cores?
As a remark, "main.m" is a script and is not a function.
Thanks!
  댓글 수: 2
Benjamin Thompson
Benjamin Thompson 2022년 2월 10일
I don't think you can assign jobs to cores like that. Have you tried reducing the size of your parallel pool to maybe 5 instead of 20?
Shumao Zhang
Shumao Zhang 2022년 2월 10일
Thanks for the reply. However, for each job I only need 4 cores so the size of my parallel pool is 4.
I need to, for example, run two jobs at one time, which means ideally I just need 8 cores and assign them to these two jobs without sharing.
When I tried openning two Matlab instances, I feel that some cores are shared by two jobs so the speed for each job slows down significantly.

댓글을 달려면 로그인하십시오.

답변 (1개)

Edric Ellis
Edric Ellis 2022년 2월 11일
You say that "main.m" script uses 4 cores to do parallel computing. I presume this means that it contains a parfor loop. One way you could launch multiple different copies of "main.m" is by using the batch command. Each time you call batch, you get back a parallel.Job object which lets you see how the computation is going, and retrieve the results at the end. By using the 'Pool' parameter to the batch command, you can make the job have a parallel pool available for running the parfor loops. You might do something like this:
for idx = 1:3
param = idx; % param is a variable read by your script "main.m"
job(idx) = batch('main', 'Pool', 4); % Launches "main.m" with a parallel pool of 4 workers
end
% Now that all jobs are launched, we can wait for them to complete and
% collect the result
for idx = 1:3
wait(job(idx));
results{idx} = load(job(idx)); % This will retrieve variables created by "main.m"
end

카테고리

Help CenterFile Exchange에서 MATLAB Parallel Server에 대해 자세히 알아보기

제품


릴리스

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by