Recompiling MapReduce V1 Applications

Existing compiled MapReduce V1 applications may need to be recompiled before they can be run as MapReduce V1 applications in MapR Version 5.x. The small number of API changes that have been made, including removal of classes and methods and conversion of classes to interfaces, are documented here. If your application does not use any of the changes listed in this document, you do not need to recompile the application.

When an application has been compiled against MapReduce V1 or MapReduce V2 (YARN), the application can be run in either mode.

The following list of changes is grouped by package name.

org.apache.hadoop.mapred.jobcontrol

• extends ControlledJob
• getMapredJobID: return type changed from String to JobID

• extends org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl
• String addJob(ControlledJob) instead of String addJob(Job), where Job extends ControlledJob

org.apache.hadoop.mapred

• Changed from class to interface

• All methods removed. Counter class remains for backward compatibility.

• Removed methods:
  • void registerNotification(JobConf, JobStatus)
  • void startNotifier()
  • void stopNotifier()

• QueueACL qACLNeeded: Change in type from org.apache.hadoop.mapred.QueueManager.QueueACL to org.apache.hadoop.mapred.QueueACL

• Changed from class to interface
• Progress method removed

• Extends AbstractCounters, and some methods have been type parameterized, using generic. This change breaks binary compatibility.

• Removed methods: captureDebugOut, captureOutAndError, getJobDir, getTaskLogFile, getUserLogDir

• Removed methods: boolean getIncludeCounters(), TaskLogFS getTaskLogFs(), void setIncludeCounters(boolean), void setTaskLogFs(TaskLogFS)

• Signature change for several methods where the JvmContext argument has been removed.

• getHttpScheme removed

• getMaxPrefetchMapTasks() removed

• New exception thrown

• Change in exceptions: java.lang.ClassNotFoundException was removed. java.lang.InterruptedException was added.
• Change of visibility from protected to public.

org.apache.hadoop.mapreduce

• Changed from class to interface
• Removed:
  • boolean equals(Object)
  • int hashCode()
  • void readFields(DataInput): read the binary representation of the counter
  • void write(DataOutput)

• Changed from class to interface
• Removed:
  • boolean equals(Object)
  • Counter findCounter(String, String): internal to find a counter in a group
  • Counter findCounter(String)
  • String getDisplayName(): get the display name of the group
  • String getName(): get the internal name of the group
  • int hashCode()
  • void incrAllCounters(CounterGroup)
  • Iterator<Counter> iterator()
  • void readFields(DataInput)
  • int size(): return the number of counters in this group
  • void write(DataOutput)

• Extends AbstractCounters, and some methods have been type parameterized, using generic. This breaks binary compatibility.

• Removed: TaskCompletionEventList getTaskCompletionEventList(int), void setUserClassesTakesPrecedence(boolean)

• Changed from class to interface
• Removed: boolean userClassesTakesPrecedence()

• getStagingDir: Change in signature from (JobClient, Configuration) to (Cluster, Configuration)

• Changed from class to interface

• Changed from non-abstract to abstract

• Changed from class to interface

• Changed from class to interface

• Changed from non-abstract to abstract

• Changed from class to interface
• Removed: progress

• Changed from class to interface

org.apache.hadoop.mapreduce.lib.output

• abortTask: Change in exceptions thrown from no exceptions to java.io.IOException.

org.apache.hadoop.mapreduce.security.token

• Removed class

org.apache.hadoop.mapreduce.util

• Change in signature for some methods (emoved arg - signal; now just takes pid): killProcess, killProcessGroup, etc.