Retrieving data from createtask is very slow
조회 수: 4 (최근 30일)
이전 댓글 표시
I used createJob and createTask to execute a program on 16 notes, then retrived my data from each tasks. The result is very large, I can not use fetchOutputs or getAllOutputArgument.My network is a Gigabit LAN. However, I found that the speed of retrieving data is only 3M/s.It cost too much time is data transmission. This is my code:
Job=createJob(sched); set(Job,'PathDependencies',PathDependency},'FileDependencies','Reconstruction_MLOS_SMART_Parallel.m','IntegratingLOS.m','MARTUpdate.m','MARTInitialize.m'});
for ii=1:cpucorenum
curTask=newtask{ii,1};
PartStart=curTask(1);
PartEnd=curTask(end);
TaskObj(ii,1)=createTask(Job,@MARTUpdate,1,{VolumeSpace(:,:,PartStart:PartEnd), MFx, MFy, Xmesh, Ymesh, xmesh_jiu, ymesh_jiu, m, oriI, newI, PartStart, PartEnd, ParaSet});
end
submit(Job);
wait(Job,'finished');
for ii=1:cpucorenum
curTask=newtask{ii,1};
PartStart=curTask(1);
PartEnd=curTask(end);
results_t = get(TaskObj(ii,1), 'OutputArguments');
VolumeSpace(:,:,PartStart:PartEnd)=results_t{1,1};
destroy(TaskObj(ii,1));
clear results_t
end
%%clear the temporary value
clear results_t TaskObj
destroy(Job);
clear Job
The problem arised in ‘results_t = get(TaskObj(ii,1), 'OutputArguments')’. Too much time expended in this step.
I holp some one can help me. Thank you!
댓글 수: 0
답변 (1개)
Edric Ellis
2014년 1월 23일
The time taken for fetching outputs is driven by the disk access speed, and the speed of MATLAB's own SAVE command. I ran the following test:
c = parcluster('local');
numMBytes = floor(logspace(1, 3, 10));
times = zeros(size(numMBytes));
for idx = 1:numel(numMBytes)
j = batch(c, @randi, 1, {[-10 10], numMBytes(idx), 1024^2, 'int8'});
wait(j);
tic
x = fetchOutputs(j);
times(idx) = toc;
delete(j);
end
plot(numMBytes, numMBytes ./ times);
xlabel 'Number of megabytes'
ylabel 'MB / s'
and saw on my system approximately 140 MB/s. This is R2013b on GLNXA64 using the local cluster which stores the outputs on a network disk.
참고 항목
카테고리
Help Center 및 File Exchange에서 MATLAB Parallel Server에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!