Parfor loop, how to check if it is actually working?

조회 수: 14 (최근 30일)
Orestis Stylianou
Orestis Stylianou 2019년 9월 22일
댓글: Orestis Stylianou 2019년 9월 23일
Hello peopel, the following is my code
load surrogate_data.mat
scales = [16 18 20 22 24 27 30 34 37 41 46 51 57 ...
63 70 78 87 97 107 119 133 147 164 182 202 225 ...
250 278 309 343 382 424 472 524 583 648 720 800 ...
889 989 1099 1221 1358 1509 1677 1864 2072 2303 ...
2560 2846 3163 3516 3908 4344 4828 5367 5965 6631 7370 8192];
combos = nchoosek(1:62,2);
parfor iteration = 1:10
all_scales_avg_MI = {};
for subjects = 1:12
names = ["Subject01","Subject02","Subject03","Subject04","Subject05","Subject06","Subject07","Subject08","Subject09","Subject10","Subject11","Subject13"];
name = names(subjects);
iteration_number = "iteration" + convertCharsToStrings(num2str(subjects));
data = all_Subjects.(name).(iteration_number);
[rows, columns] = size(data);
scale_counter = 1;
scale_avg_MI = {};
for scale = scales
window_counter = 0;
i = 1;
number = floor(rows/scale);
all_windows_mutual_information = {};
for windows = 1 : number
% Fixes the occasional problem with the last mattrix
% if it exceeds datapoints
if (window_counter+1)*scale+1 > rows
matrix =data(window_counter*scale+1:(window_counter+1)*scale,:);
else
matrix = data(window_counter*scale+1:(window_counter+1)*scale+1,:);
end
for j = 1:length(combos)
ch1 = combos(j,1);
ch2 = combos(j,2);
comboName = convertCharsToStrings(num2str(ch1)) + '-'+ convertCharsToStrings(num2str(ch2));
ts1 = matrix(:,ch1);
ts2 = matrix(:,ch2);
[ MI ] = mutual_information( ts1, ts2, 0, 2, 0 );
all_windows_mutual_information{j,1,windows} = comboName;
all_windows_mutual_information{j,2,windows} = MI;
end
window_counter = window_counter + 1;
i = i + 1;
end
% Averaging every window for each scale
a = cell2mat(all_windows_mutual_information(:,2,:));
b = mean(a,3);
scale_avg_MI(:,1,scale_counter) = all_windows_mutual_information(:,1,1);
scale_avg_MI(:,2,scale_counter) = num2cell(b);
scale_counter = scale_counter + 1;
end
% Averaging every scale for each subject
a = cell2mat(scale_avg_MI(:,2,:));
b = mean(a,3);
all_scales_avg_MI(:,2,subjects) = num2cell(b);
figure
end
final(:,2,:,iteration) = all_scales_avg_MI(:,2,:);
end
for iteration = 1:40
for subjects = 1:12
for i = 1:length(combos)
ch1 = combos(i,1);
ch2 = combos(i,2);
comboName = convertCharsToStrings(num2str(ch1)) + '-'+ convertCharsToStrings(num2str(ch2));
final{i,1,subjects,iteration} = comboName;
end
end
end
clearvars -except final
save 1-10.mat
So.... Initially I tried to run this thing with 40 and 20 iterations, in both cases 5-10 min later I will get an error that matlab is out of memory (I was checking the dask manager and matlab was using 127/128 of my RAM). I decided to run the same script but for only 10 iterations. I let the script run for two hours and stopped it (ctrl+c) just to see what variables are in the workspace. I could see only the variables defined before the parfor loop. In regular loops I would actually get some variables on the workspace depending on the exact line the script stopped. Is that a normal thing? Or am I stuck in an infinity loop for some reason? I ve also included "figure" in line 59 just to have a pop-up and be sure that everythging is fine, it's been 4.5 hours ( i started the script again) and yet I had no results.
P.S.1 I run the same script for only one iteration with a parfor loop at line 10 (parfor subjects = 1:12 and so on) and it took me 2 hours to complete.
P.S.2 I have 24 cores avaliable (2x12core CPUs)
Thanks for the help
  댓글 수: 2
Edric Ellis
Edric Ellis 2019년 9월 23일
Because the parfor loop iterations are running in separate MATLAB worker processes, and these worker processes effectively run in "-nodisplay" mode, your figure calls will have no (visible) effect. I would recommend using disp instead.
Also, it is normal for parfor loops not to produce partial results when interrupted by CTRL-C.
Orestis Stylianou
Orestis Stylianou 2019년 9월 23일
Thanks for the answer. As a matter of fact the loop finished 10 min ago and everything was fine. The only problem I am having right now is that I run ouf of memory when I try to run all 40 iterations. Any suggestions on how to deal with it, except of decreasing the iteration number?

댓글을 달려면 로그인하십시오.

답변 (0개)

카테고리

Help CenterFile Exchange에서 Parallel for-Loops (parfor)에 대해 자세히 알아보기

태그

제품


릴리스

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by