Memory issues using parfeval

조회 수: 13 (최근 30일)
Andre Zeug
Andre Zeug 2020년 4월 22일
답변: Swastik Sarkar 2024년 10월 16일
Hi everyone!
Since years I am using the nice function export_fig to save figures, which I create with my scripts and functions. To speed this up I tried to do this in background using 'parfeval'. Unfortunately, for some reason workers pile up memory consumption which can be monitored e.g. with the ProcessExplorer.
function export_figP(fname, varargin)
% same input as export_fig, but first input must be the filename, last
% input must be the figure handle (could be improved)
tParFeval = tic;
tt = []; fig_1 = [];
set(varargin{end},'Visible','off')
myEvalStr = 'tt = parfeval(@export_fig, 0, fname';
for n=1:nargin-2
myEvalStr = sprintf('%s, ''%s''',myEvalStr, varargin{n});
end
myEvalStr = sprintf('%s, fig_1);',myEvalStr);
n = nargin-1;
eval(sprintf('fig_1 = varargin{%d};',n))
eval(myEvalStr)
afterEach(tt, @(~)evalAfterAll(tt,fig_1,fname,tParFeval), 0);
function evalAfterAll(tt,hh,saveStr,tParFeval)
if ~isempty(tt.Error)
fprintf(2,'!!! Error saving Figure %s !!! \n',saveStr)
else
fprintf(2,'Figure ''%s'' saved within %.3fs (total time %.3fs)\n',saveStr,(datenum(tt.FinishDateTime)-datenum(tt.CreateDateTime))*24*60*60, toc(tParFeval))
close(hh)
tt = [];
clear all
end
end
end
The function can be tested with this lines (requires export_fig)
%% Test export_figP
if ~isfolder(fullfile(pwd, 'figs')); mkdir('figs');end
tic
for n = 1:1000
% create simple figure
fig1 = figure('Color','w','Visible','on');
imagesc(rand(1000, 1000))
colorbar
% save figure in Background
saveStr = sprintf('%04d_TestFig.png',n);
export_figP(sprintf('figs/%s',saveStr),'-q100','-nocrop','-r300',fig1);
fprintf('saving of %s started (%.3f)\n', saveStr, toc)
end
What do I do wrong?
How to clear the workers memory after the function call (except shuting down parallel pool)?
Probably my function 'export_figP' does something substantially wrong.
  댓글 수: 1
Andre Zeug
Andre Zeug 2020년 5월 1일
Is there an option to clear workers memory when no further job is processed?
p = gcp('nocreate')
p =
ProcessPool with properties:
Connected: true
NumWorkers: 16
Cluster: local
AttachedFiles: {}
AutoAddClientPath: true
IdleTimeout: Inf (no automatic shut down)
SpmdEnabled: true
q = p.FevalQueue
q =
FevalQueue with properties:
Number Queued: 0
Number Running: 0
But memory consumtion is still big after several calls of parfeval:

댓글을 달려면 로그인하십시오.

답변 (1개)

Swastik Sarkar
Swastik Sarkar 2024년 10월 16일
The export_figP and evalAfterAll functions can be enhanced for readability and effective management of figures throughout their lifecycle. Below is the improved implementation:
function export_figP(fname, varargin)
figHandle = varargin{end};
set(figHandle, 'Visible', 'off');
tt = parfeval(@export_fig, 0, fname, varargin{1:end-1}, figHandle);
afterEach(tt, @(~) evalAfterAll(tt, figHandle, fname), 0);
end
function evalAfterAll(tt, figHandle, saveStr)
if ~isempty(tt.Error)
fprintf(2, '!!! Error saving Figure %s !!! \n', saveStr);
else
fprintf('Figure ''%s'' saved successfully.\n', saveStr);
end
% Close the figure and clear the task
close(figHandle);
delete(tt);
end
Clearing the output data of the parallel.FevalFuture object can further enhance performance. More information can be found in the following resource:
Hope this helps.

카테고리

Help CenterFile Exchange에서 Text Data Preparation에 대해 자세히 알아보기

제품


릴리스

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by