run background function on subset of allocated parpool nodes on cluster?
조회 수: 2 (최근 30일)
이전 댓글 표시
Is it possible to start a parpool, then run a background function on a subset of the parpool's allocated nodes (on a cluster)?
Suppose I allocate (total Procs) = (48 procs) x ( 6 nodes) using
parpool(48*6);
Then can I then invoke some command such as
myNum=[1e6,2e6]
Result1 = parfeval(backgroundPool_1,@rand,1,myNum(1)); % suppose bgPool_1 is a subset of parpool somehow
Result2 = parfeval(backgroundPool_2,@rand,1,myNum(2)); % suppose bgPool_1 is a subset of parpool somehow
where backgroundPool_1 would be like 48*6/2, half of the number of allocated nodes, and same for bgP_2.
- I would not want to allocate two different parpools, because I am running interactively on a cluster and want all nodes to be pre-allocated on the slurm queue
댓글 수: 1
Edric Ellis
2022년 7월 25일
This isn't possible today. Can you explain more about what you want to do on the pool subset if you could create one? What do you want to run there, and how would it use multiple workers of the subset? One thing I can imagine is doing something like this:
p = parpool(..);
[subPool1, subPool2] = somehowDividePool(p);
fut = parfeval(subPool1, @myFunctionThatCallsParfor, numOuts, in1, in2);
Here, myFunctionThatCallsParfor has a parfor loop inside, and it gets to use the workers from subPool1 to run that.
Is that the sort of thing you're interested in? If not, it would be useful if you could explain what you would like to be able to do.
답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Parallel Computing Fundamentals에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!