run background function on subset of allocated parpool nodes on cluster?

조회 수: 2 (최근 30일)
Science Machine
Science Machine 2022년 7월 23일
댓글: Edric Ellis 2022년 7월 25일
Is it possible to start a parpool, then run a background function on a subset of the parpool's allocated nodes (on a cluster)?
Suppose I allocate (total Procs) = (48 procs) x ( 6 nodes) using
parpool(48*6);
Then can I then invoke some command such as
myNum=[1e6,2e6]
Result1 = parfeval(backgroundPool_1,@rand,1,myNum(1)); % suppose bgPool_1 is a subset of parpool somehow
Result2 = parfeval(backgroundPool_2,@rand,1,myNum(2)); % suppose bgPool_1 is a subset of parpool somehow
where backgroundPool_1 would be like 48*6/2, half of the number of allocated nodes, and same for bgP_2.
  • I would not want to allocate two different parpools, because I am running interactively on a cluster and want all nodes to be pre-allocated on the slurm queue
  댓글 수: 1
Edric Ellis
Edric Ellis 2022년 7월 25일
This isn't possible today. Can you explain more about what you want to do on the pool subset if you could create one? What do you want to run there, and how would it use multiple workers of the subset? One thing I can imagine is doing something like this:
p = parpool(..);
[subPool1, subPool2] = somehowDividePool(p);
fut = parfeval(subPool1, @myFunctionThatCallsParfor, numOuts, in1, in2);
Here, myFunctionThatCallsParfor has a parfor loop inside, and it gets to use the workers from subPool1 to run that.
Is that the sort of thing you're interested in? If not, it would be useful if you could explain what you would like to be able to do.

댓글을 달려면 로그인하십시오.

답변 (0개)

카테고리

Help CenterFile Exchange에서 Parallel Computing Fundamentals에 대해 자세히 알아보기

제품


릴리스

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by