Hadoop Client Compatibility
Describes compatibility between Hadoop 2.x and Hadoop 3.x clients and servers.
Hadoop Client-Server Compatibility Matrix
IMPORTANT Release 6.x user documentation is no longer being updated. As a result,
this page might be out of date. For the most current compatibility information, see the
corresponding page under Interoperability Matrices.
As indicated in Hadoop Protocol Versions, EEP 9.0.0 includes Hadoop 3, while previous EEP versions used Hadoop 2. The following table
summarizes Hadoop 2 and Hadoop 3 client-server compatibility:
1Some configuration is required. Use the following step.
| Hadoop 3 Server | Hadoop 2 Server | |
|---|---|---|
| Hadoop 3 Client | Compatible | Not Compatible |
| Hadoop 2 Client | Compatible1 | Compatible |
Using a Hadoop 2 Client with a Hadoop 3 Server
To use a Hadoop 2 client with a Hadoop 3 server, add the following properties to the
<HADOOP_HOME>/etc/hadoop/mapred-site.xml
file:<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/var/mapr/cluster/yarn/rm/staging</value>
</property>
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>The first property change is needed for the Hadoop 2 client contained in EEP 8.1.0. In EEP 8.1.0, the default path was changed to /var/mapr/cluster/yarn/hs. In
EEP
9.0.0, the default was changed back to
/var/mapr/cluster/yarn/rm/staging.