com.mapr.db.mapreduce

Class MapRDBMapReduceUtil

java.lang.Object

com.mapr.db.mapreduce.MapRDBMapReduceUtil

public class MapRDBMapReduceUtil
extends java.lang.Object

Utility routines to set up M/R specific job details for Map/Reduce steps.

Constructor Summary

Constructors

Constructor and Description
MapRDBMapReduceUtil()

Method Summary

Modifier and Type	Method and Description
static void	configureMutationOutputFormat(org.apache.hadoop.mapreduce.Job job, java.lang.String destTable) Utility function to set up TableMutationOutputFormat if user wants to send DocumentMutation to update existing documents in a MapRDB table.
static int	configurePartitioner(org.apache.hadoop.mapreduce.Job job, java.lang.String destPath) Utility function that is used to set up custom partitioner for MapR-DB destination tables.
static void	configureTableInputFormat(org.apache.hadoop.mapreduce.Job job, java.lang.String srcTable) This function is called to configure table input format to read data from MapR-DB table into a MapReduce application.
static void	configureTableOutputFormat(org.apache.hadoop.mapreduce.Job job, java.lang.String destTable) This function is invoked to configure the table output format for MapReduce application.
static void	setMapOutputKeyValueClass(org.apache.hadoop.mapreduce.Job job) Function to be used to set Map Output Key class if an intermediate stage (Map/Reduce) will emit value (as key) and Document (as value).
static void	setOutputKeyValueClass(org.apache.hadoop.mapreduce.Job job) Function to be used to set Output Key class if an intermediate stage (Map/Reduce) will emit value (as key) and Document (as value).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRDBMapReduceUtil

public MapRDBMapReduceUtil()

Method Detail

configureTableInputFormat

public static void configureTableInputFormat(org.apache.hadoop.mapreduce.Job job,
                                             java.lang.String srcTable)
                                      throws java.io.IOException

This function is called to configure table input format to read data from MapR-DB table into a MapReduce application.

Parameters:

job - MapReduce job object.

srcTable - Name of the source table.

Throws:

java.io.IOException

configureTableOutputFormat

public static void configureTableOutputFormat(org.apache.hadoop.mapreduce.Job job,
                                              java.lang.String destTable)

This function is invoked to configure the table output format for MapReduce application.

Parameters:

job - MapReduce job object.

destTable - Name of the destination table.

configureMutationOutputFormat

public static void configureMutationOutputFormat(org.apache.hadoop.mapreduce.Job job,
                                                 java.lang.String destTable)

Utility function to set up TableMutationOutputFormat if user wants to send DocumentMutation to update existing documents in a MapRDB table.

Parameters:

job - MapReduce job object.

destTable - Name of the destination table.

setMapOutputKeyValueClass

public static void setMapOutputKeyValueClass(org.apache.hadoop.mapreduce.Job job)

Function to be used to set Map Output Key class if an intermediate stage (Map/Reduce) will emit value (as key) and Document (as value).

Parameters:

job -

setOutputKeyValueClass

public static void setOutputKeyValueClass(org.apache.hadoop.mapreduce.Job job)

Function to be used to set Output Key class if an intermediate stage (Map/Reduce) will emit value (as key) and Document (as value).

Parameters:

job -

configurePartitioner

public static int configurePartitioner(org.apache.hadoop.mapreduce.Job job,
                                       java.lang.String destPath)
                                throws java.io.IOException

Utility function that is used to set up custom partitioner for MapR-DB destination tables. This function assumes that the destination table has already been created with partitions created around tablet boundaries.

Parameters:

job - MapReduce job object.

destPath - Destination table for M/R job.

Returns:

Number of partition splits which will determine number of reducers.

Throws:

java.io.IOException