how can I use the parallel computing for lsquarefit function.

I run a image processing program and it has a data of matrix with size of (254*318*31). Each element represents a pixel and it takes approximately 0.5-1 second for one pixel so it takes around 3 days to get my results any faster method. I tried with the parallel computing but when I start the parallel computing it runs only for 30 mins and stops automatically. How do i make it run throughout the program.

Answer 1

[Raymond Norris]

Hi Sasidharan,

Could you provide a bit more context, at a high level, how you're running your code. That is

• on your machine or a cluster
• if on a cluster, using MJS or a scheduler (e.g. PBS)
• how are you starting the parallel pool, explicitly (i.e. calling parpool) or automatically (e.g. by calling parfor)
• is the code a combination of a lot of serial code (i.e. outside of a parfor) and parallel code or just parallel code (i.e. the bulk of the time is running in a parfor/spmd block)?

For starters, be aware that, by default, a parellel pool will be deleted after 30 minutes of inactivity. I could imagine that you've run your parallel code, but your (serial) code still runs in access of 30 minutes after the last parfor/spmd call so that MATLAB automatically shuts down your parallel pool, but continues to run your code.

If you want to disable/extend the time out, go to the MATLAB Preferences > Parallel Computing Toolbox. At the bottom is a checkbox to disable automatic timeout as well as an edit box to change the default if you do want it.

Thanks,

Raymond

[Sasidharan Subramani]

I run my code in my computer and I start the parallel pool by calling parpool explicitily.

%July25 2020. AM. p-type two-defect model

tauptt0 = 5.58; %start point, us

tau2p1tt0 = 2.48;%start point, us

tau2p2tt0 = 14.9;%start point, us

tau3p1tt0 = 1.81;%start point, us

tau3p2tt0 = 8.94;%start point, us;

NT10tt0 = 1.26e7; %2.8e17;

NT20tt0 = 7.75e6; %1.6e16;

Coef0 = 7e-4; %start point

tauptt_bound = [1,10]; %low limit, high limit. us

tau2p1tt_bound = [1,10]; %low limit, high limit. us

tau2p2tt_bound = [1,50]; %low limit, high limit. us

tau3p1tt_bound = [1,40]; %low limit, high limit. us

tau3p2tt_bound = [0.2,10]; %low limit, high limit. us

NT10tt_bound = [1e7,1e8]; %NT10t = zeros(128,160);

NT20tt_bound = [1e6,1e7]; %NT20t = zeros(128,160);

Coef_bound = [1e-5,1e-3]; %low limit, high limit.

tauptt = zeros(254,318); %taupt = zeros(128,160);

tau2p1tt = zeros(254,318); %tau2p1t = zeros(128,160);

tau2p2tt = zeros(254,318); %tau2p2t = zeros(128,160);

tau3p1tt = zeros(254,318); %tau3p1t = zeros(128,160);

tau3p2tt = zeros(254,318); %tau3p2t = zeros(128,160);

NT10tt = zeros(254,318); %NT10t = zeros(128,160);

NT20tt = zeros(254,318); %NT20t = zeros(128,160);

Coef = zeros(254,318); %Coef = zeros(128,160);

Pexp = zeros(1,length(f));

for i1 = 1:254 %i1 = 1:254 Y direction

for i2 = 1:318 %i2 = 1:318 X direction

for i3 = 1:length(f)

if Data(i1,i2,1) > 0.1;

%if PData(i1,i2,1) < -200;

Pexp(i3) = Data(i1,i2,i3);%change May8 2020

%end %changed May15

Param = lsqcurvefit(@layer_substrate,[tauptt0,tau2p1tt0,tau2p2tt0,tau3p1tt0,tau3p2tt0,NT10tt0,NT20tt0,Coef0],f,Pexp,[tauptt_bound(1),tau2p1tt_bound(1),tau2p2tt_bound(1),tau3p1tt_bound(1),tau3p2tt_bound(1),NT10tt_bound(1),NT20tt_bound(1),Coef_bound(1)],[tauptt_bound(2),tau2p1tt_bound(2),tau2p2tt_bound(2),tau3p1tt_bound(2),tau3p2tt_bound(2),NT10tt_bound(2),NT20tt_bound(2),Coef_bound(2)]); %change May8 2020

%Param = lsqcurvefit(@layer_substrate,[tauptt0,tau2p1tt0,tau2p2tt0,tau3p1tt0,tau3p2tt0,Coef0],f,Pexp,[tauptt_bound(1),tau2p1tt_bound(1),tau2p2tt_bound(1),tau3p1tt_bound(1),tau3p2tt_bound(1),Coef_bound(1)],[tauptt_bound(2),tau2p1tt_bound(2),tau2p2tt_bound(2),tau3p1tt_bound(2),tau3p2tt_bound(2),Coef_bound(2)]); %change May20 2020

% Param = lsqcurvefit(@layer_substrate,[tauptt0,tau2p1tt0,tau2p2tt0,tau3p1tt0,tau3p2tt0],f,Pexp,[tauptt_bound(1),tau2p1tt_bound(1),tau2p2tt_bound(1),tau3p1tt_bound(1),tau3p2tt_bound(1)],[tauptt_bound(2),tau2p1tt_bound(2),tau2p2tt_bound(2),tau3p1tt_bound(2),tau3p2tt_bound(2)]); %change May8 2020

%Param = lsqcurvefit(@layer_substrate,[tauptt0,tau2p1tt0,tau2p2tt0],f,Pexp,[tauptt_bound(1),tau2p1tt_bound(1),tau2p2tt_bound(1)],[tauptt_bound(2),tau2p1tt_bound(2),tau2p2tt_bound(2)]); %3 parameters. change May16 2020

tauptt(i1,i2) = Param(1);

tau2p1tt(i1,i2) = Param(2);

tau2p2tt(i1,i2) = Param(3);

tau3p1tt(i1,i2) = Param(4);

tau3p2tt(i1,i2) = Param(5);

NT10tt(i1,i2) = Param(6);

NT20tt(i1,i2) = Param(7);

Coef(i1,i2) = Param(8);

%end

end

end

end

end

figure(1)

imagesc(tauptt);axis image;colorbar;

hold on;

title ('taup');

figure(2)

imagesc(tau2p1tt);axis image;colorbar;

hold on;

title ('tau2p1');

figure(3)

imagesc(tau2p2tt);axis image;colorbar;

hold on;

title ('tau2p2');

figure(4)

imagesc(tau3p1tt);axis image;colorbar;

hold on;

title ('tau3p1');

figure(5)

imagesc(tau3p2tt);axis image;colorbar;

hold on;

title ('tau3p2');

figure(6)

imagesc(Coef);axis image;colorbar;

hold on;

title ('Coef x 1e25');

figure(7)

imagesc(NT10tt);axis image;colorbar;

hold on;

title ('NT10 x 1e10');

figure(8)

imagesc(NT20tt);axis image;colorbar;

hold on;

title ('NT20 x 1e10');

This is my program for the image fitting.

[Raymond Norris]

And where do you call any parallel construct (e.g. parfor)? Are you trying to re-write any of the for loops as parfor?

parpool starts a pool of workers, which parfor, spmd, etc. can make use of, but parpool alone doesn't parallelize your code. You have several options, here are two:

• Rewrite one of your nested (most likely the outer) for loop as a parfor. At first glance, Pexp might cause some issues.
• When calling lsqcurvefit, there's an optional 'UseParallel' switch to help the solver. Try the following:

Param = lsqcurvefit(@layer_substrate, ... ,'UseParallel',true);    

If you're not calling any parallel constructs, then that would explain why 30 minutes, the parallel pool was deleted (since it timed out from not running any parallel code).

Does that sound like what is happening?

Raymond