Configure Advanced Options for MATLAB Job Scheduler Integration

Follow these instructions to configure advanced options during integration of MATLAB^® Job Scheduler with your cluster.

Note

If this is the first time you integrate MATLAB Job Scheduler, see the following for the most common configuration options: Install and Configure MATLAB Parallel Server for MATLAB Job Scheduler and Network License Manager.

In the following instructions, matlabroot refers to the location of your installed MATLAB Parallel Server™ software. Where you see this term used in the instructions that follow, substitute the path to your location.

Run Multiple MATLAB Parallel Server Versions

You can upgrade your MATLAB Job Scheduler clusters and continue to use the R2016a release onwards of Parallel Computing Toolbox on your MATLAB desktop client to connect to it. To take advantage of this backward compatibility feature:

Install the latest version of MATLAB Parallel Server on your cluster. You must use this version to run MATLAB Job Scheduler on your cluster.
Install MATLAB Parallel Server for each release that you want to support in the cluster. For example, to use R2016a and R2016b with your cluster, install both the R2016a and R2016b releases of MATLAB Parallel Server.
Configure MATLAB Job Scheduler with the location of these installations. In the mjs_def configuration file, specify the location of each installation of MATLAB Parallel Server in the MJS_ADDITIONAL_MATLABROOTS variable. You can find this file in matlabroot/toolbox/parallel/bin for Linux (mjs_def.sh) and Windows (mjs_def.bat). For more information, see mjs.

With this configuration, the MATLAB Job Scheduler allows MATLAB clients from the installed releases to submit jobs to the cluster. The MATLAB Job Scheduler dynamically starts the right version of the MATLAB worker to run the job.

Set Up Windows Cluster Hosts

If this is the first installation of MATLAB Parallel Server on a cluster of Windows machines, you need to configure these hosts for job communications.

Note

If you do not have a Windows cluster, or if you have already installed a previous version of MATLAB Parallel Server on your Windows cluster, you can skip this step.

Configure Windows Firewalls on the Cluster Nodes

If you are using Windows^® firewalls on your cluster nodes,

Log in as a user with administrator privileges.
Execute the following in a Windows command prompt.
```
matlabroot\toolbox\parallel\bin\addMatlabToWindowsFirewall.bat
```
This command adds MATLAB as an allowed program. If you are using other firewalls, you must configure them for similar accommodation.

Configure Windows User Access for mjs

The user that mjs runs as requires access to the cluster MATLAB installation location. By default, mjs runs as the user LocalSystem. If your network allows LocalSystem to access the install location, you can skip this step. (If you are not sure of your network configuration and the access provided for LocalSystem, contact the MathWorks install support team.)

Note

If LocalSystem cannot access the install location, you must run mjs as a different user.

You can set a different user with these steps:

With any standard text editor (such as WordPad) open the mjs_def file found at:
```
matlabroot\toolbox\parallel\bin\mjs_def.bat
```
Find the line for setting the MJSUSER parameter, and provide a value in the form domain\username:
```
set MJSUSER=mydomain\myusername
```
Provide the user password by setting the MJSPASS parameter:
```
set MJSPASS=password
```
Save the file.

Open Required Ports on Server

The mjs service uses as many ports as required, starting with BASE_PORT. By default, BASE_PORT is 27350.

If you use a host that runs a total of nJ job managers and nW workers, the mjs service reserves a total of 6+2*nJ+4*nW consecutive ports for its own use. All job managers and workers, even those on different hosts, that are going to work together must use the same base port. Otherwise the job managers and workers will not be able to contact each other. In addition, MPI communication occurs on ports starting at BASE_PORT+1000 and use 2*nW consecutive ports.

For example, if you use a host with 1 job manager and 16 workers, then you need the following ranges of ports to be open:

27350 – 27422 for the mjs service.
28350 – 28382 for MPI communication.

To connect from MATLAB to a cluster with a non-default BASE_PORT, you must append the value of BASE_PORT to the 'Host' property in the MATLAB Job Scheduler cluster profile. You must do this in the form Hostname:BASE_PORT, for example myMJSHost:44001.

Stop mjs Services of Old Installation

If you have an older version of MATLAB Parallel Server running on your cluster nodes, you should stop the mjs services before starting the services of the new installation.

Stop mjs on Windows

Open a Windowscommand window with administrator privileges.
In the command window, navigate to the folder of the old installation that contains the control scripts.
```
cd oldmatlabroot\toolbox\parallel\bin
```
Stop and uninstall the old service and remove its associated files by typing the following command.
```
mjs uninstall -clean
```
In releases before R2019a, the service is called mdce. Type the following commands instead.
```
cd oldmatlabroot\toolbox\distcomp\bin
mdce uninstall -clean
```
Note
Using the -clean flag permanently removes all existing job data. Be sure this data is no longer needed before removing it.
Repeat the instructions of this step on all worker nodes.

Stop mjs on UNIX

Log in as root. If you cannot log in as root, you must alter the following parameters in the oldmatlabroot/toolbox/parallel/bin/mjs_def.sh file to point to a folder for which you have write privileges: CHECKPOINTBASE, LOGBASE, PIDBASE, and LOCKBASE if applicable. In releases before R2019a, this file is oldmatlabroot/toolbox/distcomp/bin/mdce_def.sh instead.
On each cluster node, stop the mjs service and remove its associated files by typing the commands:
```
cd oldmatlabroot/toolbox/parallel/bin
./mjs stop -clean
```
In releases before R2019a, the service is called mdce. Type the following command instead.
```
cd oldmatlabroot/toolbox/distcomp/bin
./mdce stop -clean
```
Note
Using the -clean flag permanently removes all existing job data. Be sure this data is no longer needed before removing it.

Set the MATLAB Job Scheduler Security Level

Before starting the mjs service on your cluster nodes, set a security level. For instructions, see Set Security Level. For additional security considerations, see Set MATLAB Job Scheduler Cluster Security.

Start `mjs` Service, MATLAB Job Scheduler, and Workers (Command-Line)

You can start MATLAB Job Scheduler using a graphical interface or the command line. For instructions on how to use the graphical interface, see Configure the MATLAB Job Scheduler. To use the graphical interface, Admin Center, you must run it on a computer that has direct network connectivity to all the nodes of your cluster. If you cannot run Admin Center on such a computer, you must use the command-line interface. For instructions on how to use the command-line interface, follow the next steps.

Use the Command-Line Interface (Windows)

Start the mjs Service
You must install the mjs service on all nodes (head node and worker nodes). Begin on the head node.
1. Open a Windowscommand window with administrator privileges.
2. In the Windows command window, navigate to the folder with the control scripts:
```
cd matlabroot\toolbox\parallel\bin
```
3. Install the mjs service by typing the command:
```
mjs install
```
4. Start the mjs service by typing the command:
```
mjs start
```
5. Repeat the instructions of this step on all worker nodes.
As an alternative to items 3–5, you can install and start the mjs service on nodes remotely from one machine by typing:
```
cd matlabroot\toolbox\parallel\bin
ssh hostA mjs install
ssh hostA mjs start
```
where hostA refers to a remote host. For more information on your ssh utility, see the usage reminder by typing:
```
ssh
```
Once installed, the mjs service starts running each time the machine reboots. The mjs service continues to run until explicitly stopped or uninstalled, regardless of whether a MATLAB Job Scheduler or worker session is running.
Start the MATLAB Job Scheduler
To start the MATLAB Job Scheduler, enter the following commands in a Windows command prompt. You do not have to be at the machine on which the MATLAB Job Scheduler runs, as long as you have access to the MATLAB Parallel Server installation.
1. In your Windows command prompt, navigate to the folder with the startup scripts:
```
cd matlabroot\toolbox\parallel\bin
```
2. Start the MATLAB Job Scheduler, using any unique text you want for the name <MyMJS>:
```
startjobmanager -name <MyMJS> -remotehost <MATLAB Job Scheduler host name> -v
```
3. Verify that the MATLAB Job Scheduler is running on the intended host.
```
nodestatus -remotehost <MATLAB Job Scheduler host name>
```
  Note
  If you are executing startjobmanager on the host where the MATLAB Job Scheduler runs, you do not need to specify the -remotehost flag.
  If you have more than one MATLAB Job Scheduler on your cluster, each must have a unique name.
Start the Workers
Note
Before you can start a worker on a machine, the mjs service must already be running on that machine. If you are using the network license manager, it must be running on the network.
For each node used as a worker, enter the following commands in a Windows command prompt. You do not have to be at the machines where the MATLAB workers will run, as long as you have access to the MATLAB Parallel Server installation.
1. Navigate to the folder with the startup scripts:
```
cd matlabroot\toolbox\parallel\bin
```
2. Start the workers on each node, using the text for <MyMJS> that identifies the name of the MATLAB Job Scheduler you want this worker registered with. Enter this text on a single line:
```
startworker -jobmanagerhost <MATLAB Job Scheduler host name>
    -jobmanager <MyMJS> -remotehost <worker host name> -v
```
  To run more than one worker session on the same node, give each worker a unique name by including the -name option on the startworker command, and run it for each worker on that node:
```
startworker ... -name <worker1 name>
startworker ... -name <worker2 name>
```
3. Verify that the workers are running.
```
nodestatus -remotehost <worker host name>
```
4. Repeat items 2–3 for all worker nodes.
For more information about mjs, MATLAB Job Scheduler, and worker processes, such as how to shut them down or customize them, see MATLAB Job Scheduler Cluster Customization.

Use the Command-Line Interface (UNIX)

Start the mjs Service
On each cluster node, start the mjs service by typing the commands:
```
cd matlabroot/toolbox/parallel/bin
./mjs start
```
Alternatively, you can start the mjs service on nodes remotely from one machine by typing
```
cd matlabroot/toolbox/parallel/bin
ssh hostA matlabroot/toolbox/parallel/bin/mjs start
ssh hostB matlabroot/toolbox/parallel/bin/mjs start
ssh hostC matlabroot/toolbox/parallel/bin/mjs start
```
where hostA, hostB, and hostC refers to your remote host names. For a long list of host names, you can start the mjs service on several nodes remotely from one machine by typing:
```
for host in hostA hostB hostC;
 do ssh $host matlabroot/toolbox/parallel/bin/mjs start;
done 
```
For more information on your ssh utility, see the usage reminder by typing:
```
ssh
```
Alternatively, you can access the system reference manual by typing:
```
man ssh
```
Start the MATLAB Job Scheduler
To start the MATLAB Job Scheduler, enter the following commands. You do not have to be at the machine on which the MATLAB Job Scheduler runs, as long as you have access to the MATLAB Parallel Server installation.
1. Navigate to the folder with the startup scripts:
```
cd matlabroot/toolbox/parallel/bin
```
2. Start the MATLAB Job Scheduler, using any unique text you want for the name <MyMJS>. Enter this text on a single line.
```
./startjobmanager -name <MyMJS> -remotehost <MATLAB Job Scheduler host name> -v
```
3. Verify that the MATLAB Job Scheduler is running on the intended host:
```
./nodestatus -remotehost <MATLAB Job Scheduler host name>
```
  Note
  If you have more than one MATLAB Job Scheduler on your cluster, each must have a unique name.
Start the Workers
Note
Before you can start a worker on a machine, the mjs service must already be running on that machine. If you are using the network license manager, it must be running on the network.
For each computer hosting a MATLAB worker, enter the following commands. You do not have to be at the machines where the MATLAB workers run, as long as you have access to the MATLAB Parallel Server installation.
1. Navigate to the folder with the startup scripts:
```
cd matlabroot/toolbox/parallel/bin
```
2. Start the workers on each node, using the text for <MyMJS> that identifies the name of the MATLAB Job Scheduler you want this worker registered with. Enter this text on a single line:
```
./startworker -jobmanagerhost <MATLAB Job Scheduler host name>
   -jobmanager <MyMJS> -remotehost <worker host name> -v
```
  To run more than one worker session on the same machine, give each worker a unique name with the -name option:
```
./startworker ... -name <worker1>
./startworker ... -name <worker2>
```
3. Verify that the workers are running. Repeat this command for each worker node:
```
./nodestatus -remotehost <worker host name>
```
For more information about mjs, MATLAB Job Scheduler, and worker processes, such as how to shut them down or customize them, see MATLAB Job Scheduler Cluster Customization.

Install the mjs Service to Start Automatically at Boot Time (UNIX)

Although this step is not required, it is helpful in case of a system crash. Once configured for this, the mjs service starts running each time the machine reboots. The mjs service continues to run until explicitly stopped, regardless of whether a MATLAB Job Scheduler or worker session is running.

You must have root privileges to do this step.

Debian, Fedora, SUSE, and Red Hat (non-Fedora) Platforms

On each cluster node, register the mjs service as a known service and configure it to start automatically at system boot time by following these steps:

Create the following link, if it does not already exist:
```
ln -s matlabroot/toolbox/parallel/bin/mjs /etc/mjs
```
Create the following link to the boot script file:
```
ln -s matlabroot/toolbox/parallel/bin/mjs /etc/init.d/mjs
```
Set the boot script file permissions:
```
chmod 555 /etc/init.d/mjs
```
Find your default run level. If you have a SysV Linux^® machine, you can determine the default run level by booting your machine and immediately executing the $runlevel command. The second number output is the default run level of your system. If your Linux machine does not support SysV, look in /etc/inittab for the default run level.
When you have determined the run level, create a link in the rc folder associated with that run level. For example, if the run level is 5, execute one of the following sets of platform-specific commands.
- Debian and Fedora platforms:
```
cd /etc/rc5.d;
ln -s ../init.d/mjs S99MJS
```
- SUSE platform:
```
cd /etc/init.d/rc5.d;
ln -s ../mjs S99MJS
```
- Red Hat platform (non-Fedora):
```
cd /etc/rc.d/rc5.d;
ln -s ../../init.d/mjs S99MJS
```

Validate Installation with MATLAB Job Scheduler

To verify that your MATLAB Parallel Server products are installed and configured correctly, create a cluster profile and validate it. For instructions, see Connect MATLAB Client to MATLAB Parallel Server Cluster. You can specify the number of workers to use when validating your profile, to avoid occupying the whole cluster. If your validation does not pass, contact the MathWorks Install Support Team, or see Troubleshoot Common Problems.

After you create a cluster profile, you can make any modifications appropriate for your applications, such as NumWorkersRange, AttachedFiles, or AdditionalPaths. To save your profile for other users, in the Cluster Profile Manager, select the profile and click Export, then save your profile to a file in a convenient location. Later, when running the Cluster Profile Manager, other users can import your profile by clicking Import. For more information about cluster profiles, see Discover Clusters and Use Cluster Profiles (Parallel Computing Toolbox).