When using spmd, the network communication speed is too slow.

Hello I am testing the distributed arrays using spmd with 14 computers.

The test code is as follows:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

delete(gcp('nocreate'));

c=parcluster;

delete(c.Jobs);

%%

spmd

end

N = 8;

time_parallel = zeros(N,1);

% for nn = 1 : N

for nn = N

tic

spmd

nx = nn;

ny = nn;

NBx = 2*nx+1;

NBy = 2*ny+1;

L = NBx*NBy;

end

spmd

A_ = rand(2*L,2*L,codistributor())+1i*rand(2*L,2*L,codistributor());

[Vp_,Dp_] = eig(A_,'nobalance');

spmdBarrier;

end

time_parallel(nn)=toc

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

I monitored the cpu usage of all nodes and the network details of the headnode, and it was as follows.

spmd started at 17:50 and ended at 21:30.

In the graph on the left, I think number 1 is starting parallel pool, and number 2 is an eig function calculation.

How can I reduce the network communication time from 18:00 to 21:30?

Changing the number of workers per node did not change much.

The purple color from 19:30 to 21:30 on the cpu usage graph is due to the monitoring program.

I look forward to hearing from you.

Sincerely

Answer 1

[akshatsood]

Hi 명규,

I understand that you are testing distributed arrays using spmd with 14 computers and experiencing too slow nework communications speed. In reference to the cpu usage plots attached, you wish to reduce the network connection time to 18:00 to 21:30 from the present 17:50 to 21:30 . It is evident that 'spmd' body executes code in parallel on workers of parallel pool where each worker can operate on different portion of distributed data, and can communicate with other participating workers while performing the parallel computations.

In order to acheive the desired network communication speed, I suggest the following workarounds

• Ensure that your are sending data to the workers outside the loop, so that the workers need not read the data during every iteration and network communication time does not gets compromised.
• In reference to the code script attached, variables 'nx' and 'ny' seem to be used only while computing 'NBx' and 'NBy'. You can get rid of themeselves and directly use the loop index variable 'nn' for determining 'NBx' and 'NBy'. I ran your code on my system with the above stated changes and observed an improvement in speed.
• You can leverage zeros function to create codistributed array of all zeros and use getLocalPart and gather function to extract the local portion of the codistributed array followed by gathering the results back to the host.

Keeping in mind the above modifications, the code can be updated as follows for optimum performance

delete(gcp('nocreate'));
c=parcluster;
delete(c.Jobs);

spmd
end
N = 8;
time_parallel = codistributed.zeros(N,1); % create codistributed array of all zeros

for nn = N
    
    tic
    spmd
        NBx = 2*nn+1;
        NBy = 2*nn+1;
        L = NBx*NBy;
    end
    
    spmd
        A_ = rand(2*L,2*L,codistributor())+1i*rand(2*L,2*L,codistributor());
        [Vp_,Dp_] = eig(A_,'nobalance');
        spmdBarrier;
    end
    
    % get the local part out of time_parallel so that we can operate on it directly
    
    tp_local = getLocalPart(time_parallel); 
    tp_codist = getCodistributor(time_parallel);
    
    tp_local(nn)=toc;

    % put time_parallel back together
    
    time_parallel = codistributed.build(tp_local, tp_codist);

end

% gather the results back to the host

time_parallel = gather(time_parallel)

Have a look at below references for better understanding

Benchmarking A\b - MATLAB & Simulink Example - MathWorks India

I hope this helps.