Parfor waitbar : How to do this more cleanly?

조회 수: 84 (최근 30일)
Simon
Simon 2019년 6월 6일
편집: Edric Ellis 2019년 6월 11일
I'm a frequent casual user of parfor loops. Getting a waitbar (progress meter) from parallel code used to involve having to have each thread append to a file to record progress, and there are a number of contributions on the file exchange that do this. It was kinda silly, but it was necessary.
Now that the DataQueue system exists, it should be possible to do this more sensibly. So I went looking for a good waitbar on the File Exchange. And I couldn't find one, so I tried to make my own.
The code below works nicely, but it depends on using three globals - which isn't great, because if those variable names are being used by something else, it's gonna break stuff. Can anybody suggest a better way to do this, preferably not involving globals? Persistent variables should work within update.m, but it's getting the info (waitbar handle and expected number of interations) from create.m to update.m that's stumped me.
There are three functions, all part of the ParWaitbar package. Usage is explained in the first one:
create.m:
function [ q ] = create( n, txt )
%FNPARWAITBAR Produces a waitbar object that works with parfor.
% Relies on globals. Not sure how to avoid this.
% Input:
% n: Number of iterations expected
% txt: Text for the waitbar dialog.
% Output: q: handle for the dataqueue
%
% Usage: [ q ] = ParWaitbar.create( 10, "Predicting earthquakes..." );
% parfor i=1:10
% pause(rand*5); (or do useful stuff)
% send( q, i ); % doesn't matter what's sent (i here). We're just
% counting the items rcvd.
% end
% ParWaitbar.cleanup(q);
global wb; %waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected
q = parallel.pool.DataQueue;
wb = waitbar( 0, txt );
wb_max = n;
wb_N = 0;
afterEach( q, @ParWaitbar.update );
end
update.m:
function update(~)
global wb; %shoudl have waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected
wb_N = wb_N + 1;
waitbar( wb_N / wb_max, wb );
end
cleanup.m
function cleanup( q )
% FIXME we're left with q afterwards. Presumably a fn can't delete
% something outside its own scope
global wb; %shoudl have waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected
close(wb);
delete(q);
clear wb wb_N wb_max q
end
(as an aside, the fact that we're left with q leftover as a "handle to deleted DataQueue" isn't ideal. But that's far less of a problem than the use of globals with fixed names.)

채택된 답변

Edric Ellis
Edric Ellis 2019년 6월 10일
Hm, I've been meaning to tidy up my work-in-progress parallel.pool.DataQueue pool waitbar for quite a while. Here's roughly what it would look like. This uses a handle object that gets copied to the workers, so there's no need for global state.
classdef PoolWaitbar < handle
properties (SetAccess = immutable, GetAccess = private)
Queue
N
end
properties (Access = private, Transient)
ClientHandle = []
Count = 0
end
properties (SetAccess = immutable, GetAccess = private, Transient)
Listener = []
end
methods (Access = private)
function localIncrement(obj)
obj.Count = 1 + obj.Count;
waitbar(obj.Count / obj.N, obj.ClientHandle);
end
end
methods
function obj = PoolWaitbar(N, message)
if nargin < 2
message = 'PoolWaitbar';
end
obj.N = N;
obj.ClientHandle = waitbar(0, message);
obj.Queue = parallel.pool.DataQueue;
obj.Listener = afterEach(obj.Queue, @(~) localIncrement(obj));
end
function increment(obj)
send(obj.Queue, true);
end
function delete(obj)
delete(obj.ClientHandle);
delete(obj.Queue);
end
end
end
This works with parfor, spmd, and parfeval, like this:
pw = PoolWaitbar(100, 'Example');
parfor ii = 1:20
increment(pw)
end
spmd
for ii = 21:40
if labindex == 1
increment(pw);
end
end
end
for ii = 41:100
parfeval(@() increment(pw), 0);
end
  댓글 수: 2
Simon
Simon 2019년 6월 10일
Fabulous, thank you! I don't fully understand Matlab's OO facilities, but this seems to do nicely.
Is there any reason to do increment(pw) instead of pw.increment? The latter feels more intuitive to me for using an object's method, but I don't know what normal is in Matlab for this
Are you planning to put this on File Exchange?
Edric Ellis
Edric Ellis 2019년 6월 11일
편집: Edric Ellis 2019년 6월 11일
This PoolWaitbar does use a slightly subtle trick to make things work - the use of Transient properties stops the client-side handle being sent to the workers.
There's no real difference between increment(pw) and pw.increment - but the former is the usual function-call style that tends to be used by MathWorks documentation and example code.
I would like to put this on File Exchange - but it's part of a larger suite of similar utilities, and there's some polishing required...

댓글을 달려면 로그인하십시오.

추가 답변 (0개)

카테고리

Help CenterFile Exchange에서 Environment and Settings에 대해 자세히 알아보기

제품


릴리스

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by