reentering a parallel pool
조회 수: 1 (최근 30일)
이전 댓글 표시
Hi everyone, I try to parallel function evaluation on workers. Suppose I have 10 different vectors that will be evaluated through same function. I want to retrieve outputs when they become ready. Also, when the certain number of outputs is reached, I want to evaluate the retrieved outputs through a different function without waiting to fetch all of the results. Moreover, when the unretrived data of the agents become ready, I want to use it on the same fc2 function. I want to incomplete parfeval futures to use when they finished on the same fc2 function. Also, It will be a continuing process. Whenever 5 of the retrieved results become ready, they will be used as an input for fc2 function and output of the fc2 function will be input for parfeval function again and again. I wonder is it possible or not? Here is my code.
while itr<itrmax
for j=1:nbagents
f(j) =parfeval(p,@evaluate,1,agent(j));
end
counter=0;
finishedjobs=[];
for idx = 1:nbagents
[completedIdx,value] = fetchNext(f);
finishedjobs=[finishedjobs completedIdx];
agent(completedIdx)=value;
counter=counter+1;
if counter==5
break;
end
end
for i=1:counter
output(i)=agent(finishedjobs(i))
end
target=fc2(output(i));
itr=itr+1;
end
댓글 수: 0
답변 (1개)
Sean de Wolski
2020년 6월 10일
Look at using a DataQueue to transmit intermediate data from one parfeval future that's running continuously.
web(fullfile(docroot, 'parallel-computing/parallel.pool.dataqueue.html'))
댓글 수: 5
Sean de Wolski
2020년 6월 11일
I don't think that's possible with parfeval, unfortunately. It can be done with spmd, though that's synchronous. Maybe one of the Parallel Devs will chime in.
참고 항목
카테고리
Help Center 및 File Exchange에서 Parallel Computing Fundamentals에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!