matlab.mapreduce.DeployHadoopMapReducer Class

R2026b

Namespace: matlab.mapreduce

(To be removed) Configure a MapReduce application for deployment against Hadoop

expand all in page

matlab.mapreduce.DeployHadoopMapReducer will be removed in a future release. There is no replacement functionality. For more information, see Version History.

Description

A DeployHadoopMapReducer object represents executing MapReduce on a Hadoop® cluster with MATLAB® Runtime.

Construction

config = matlab.mapreduce.DeployHadoopMapReducer creates a matlab.mapreduce.DeployHadoopMapReducer object that specifies the default properties for Hadoop execution.

Use the resulting object as input to the mapreducer function to specify the configuration properties for Hadoop execution. For deploying a standalone application, pass the matlab.mapreduce.DeployHadoopMapReducer object as input to mapreduce.

config = matlab.mapreduce.DeployHadoopMapReducer(Name=Value) creates a matlab.mapreduce.DeployHadoopMapReducer object with properties specified by one or more name-value pair arguments.

Name-Value Arguments

expand all

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Path to Hadoop installation, specified as the comma-separated pair consisting of the HadoopInstallFolder and a character vector or a string scalar.

The default value of Hadoop install folder is specified by the environment variables in the order of precedence of MATLAB_HADOOP_INSTALL, HADOOP_PREFIX, and HADOOP_HOME.

Path to Hadoop application configuration files, specified as the comma-separated pair consisting of the HadoopConfigurationFile and a character vector or a string scalar.

MATLAB Runtime installation folder for Hadoop cluster, specified as the comma-separated pair consisting of the MCRRoot and a character vector or a string scalar.

MCRRoot specifies the MATLAB Runtime installation folder used by Hadoop when executing mapreduce tasks in Hadoop.

Example: MCRRoot='/hd-shared/hadoop-2.2.0/MCR/v84'

A containers.Map object of name-value arguments that specify Hadoop configuration properties for a specific job or application. Name-value pairs must be specified as character vectors.

The properties specified in the containers.Map object are passed as a [GENERIC_OPTION] consisting of name-value arguments signaled by a -D flag to the hadoop shell command.

Example:

setenv('HADOOP_PREFIX', '/usr/lib/hadoop') % replace with your Hadoop install location
name = {'mapreduce.map.maxattempts','mapreduce.job.reduces'};
value = {'2','1'};
prop = containers.Map(name,value);
obj = matlab.mapreduce.DeployHadoopMapReducer('HadoopProperties', prop)

Examples

collapse all

Create and use a matlab.mapreduce.DeployHadoopMapReducer object to deploy into a standalone application, and deploy against Hadoop.

config = matlab.mapreduce.DeployHadoopMapReducer(...
    MCRRoot='/hd-shared/hadoop-2.2.0/MCR/v84');
mr = mapreducer(config);

Version History

collapse all

See Also

|

Topics