Why did spmd create a pool of 12 workers on a cluster with 32 cores?

I am experiencing a problem with running Matlab code with spmd blocks on the cedar cluster on Compute Canada. I asked for 32 cores and 1 node being allocated to me. The system can detect 32 cores, but spmd can generate a pool of only 12 workers.

Can someone explain why? I am attaching both the Matlab file and the output file here.

Answer 1

[Kojiro Saito]

In Parallel Preferences "Preferred number of workers" is set to 12 by default.

You can change this value to 100 (more than 32) in Preferences menu.

Alternatively, you can specify workers in parpool.

You can add parpool(num) before spmd blocks.

parpool(num)
spmd(num)
    ...
end

[Claire Fang]

I added parpool(num) before the spmd(num) end block, but I received the following error message:

Starting parallel pool (parpool) using the 'local' profile ...

Error using parpool (line 145)
Parallel pool failed to start with the following error. For more detailed
information, validate the profile 'local' in the Cluster Profile Manager.

Error in Prepare_data (line 175)
parpool(num)

Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    670)
    Failed to start pool.
        Error using parallel.Job/attachRequiredFiles (line 913)
        Could not open file
        "/home/fanglimi/.matlab/local_cluster_jobs/R2020a/Job4.in.mat", the
        error returned was: Unable to read file
        '/home/fanglimi/.matlab/local_cluster_jobs/R2020a/Job4.in.mat'. No such
        file or directory..

Should I use parpool('local', num) instead?

[Kojiro Saito]

You might need to delete previous job files in "/home/fanglimi/.matlab/local_cluster_jobs/R2020a/".

[Claire Fang]

@Kojiro Saito, you are right. It is related. Apparently I needed to prepare a .matlab folder on the server. Instructions are shown on https://docs.computecanada.ca/wiki/MATLAB. Below is the code:

[name@cluster ~]$ cd $HOME
[name@cluster ~]$ if [ -d ".matlab" ]; then
  mv .matlab scratch/
else
  mkdir -p scratch/.matlab
fi && ln -sn scratch/.matlab .matlab 

Once I did that, everything worked. Thanks!

[Kojiro Saito]

@Claire Fang

Usually, parpool will create a job directory, but in your universities clusters, home directory was set to read-only. I didn't know that, but I'm glad the issue is solved now.