Configure Environment for Interactive Debugging
Supported Platform: Linux® only.
Before you can interactively debug your applications using the MATLAB® API for Spark™, you must configure your MATLAB environment by adding the location of the Spark assembly jar to the front of MATLAB's static Java class path.
A Spark assembly JAR file includes all the Spark dependencies, including some internal Hadoop® dependencies. This JAR file is created during Spark installation and is specific to a particular combination of
Spark and Hadoop versions. For example, using Spark version 1.4.0 and Hadoop version 2.4.0 creates the file
To add the location of the Spark assembly jar to the front of MATLAB’s static Java® class path, use the following procedure:
Open a Linux terminal.
Navigate to the directory
is the location where MATLAB is installed.
Set the environment variable
SPARK_HOMEto point to the Spark installation directory. For example:
Shell Command zsh / csh / tcsh
% setenv SPARK_HOME /share/spark
$ export SPARK_HOME=/share/spark
From the directory
, execute the shell script
generatesparkjcpath. For example:
This will create a file called
directory. The contents of a sample
javaclasspath.txtfile look like this:
<before> /share/spark/conf/ /share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jar /share/spark/lib/datanucleus-rdbms-3.2.9.jar /share/spark/lib/datanucleus-api-jdo-3.2.6.jar /share/spark/lib/datanucleus-core-3.2.10.jar
<before>tag ensures that
spark-assembly-1.4.0-hadoop2.4.0.jaris added to the front of the Java class path.
Start MATLAB from the
You can create the
javaclasspath.txt file in a directory
-o option. The file must be named
$ ./generatesparkjcpath -o /home/user/javaclasspath.txt
You can verify whether
added to the front of the Java class path by typing the following at the MATLAB command
>> spath = javaclasspath('-static') ; >> spath(1:5)
/share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jaron the top of the list.
You need to complete this configuration just once. Subsequent usage only requires
that you start MATLAB from the folder
You may instead place the generated
prefdir. If you do so, MATLAB adds the necessary libraries to the static path at launch.