List of YARN Enhancements

The following bug fixes were backported from Hadoop 2.7.1 to Hadoop 2.7.0:
  • HADOOP-11872: On Windows clients, the hadoop fs command prints an incorrect message that says to use yarn jar instead of hadoop fs.
  • HADOOP-12473: DistCp's ignoring failures option should be mutually exclusive with the atomic option.
  • MAPREDUCE-6238: MR2 can't run local jobs with -libjars command options which is a regression from MR1
  • MAPREDUCE-6324: Uber jobs fail to update AMRM token when it rolls over.
  • MAPREDUCE-6334: Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler
  • MAPREDUCE-6339: Job history file is not flushed correctly because isTimerActive flag is not set true when flushTimerTask is scheduled.
  • MAPREDUCE-6353: Divide by zero error in MR AM when calculating available containers
  • MAPREDUCE-6361: MR2 cannot run local jobs with -libjars command options which is a regression from MR1.
  • MAPREDUCE-6410: Aggregated log deletion does not work after refreshing Log Retention Settings in a secure cluster.
  • YARN-3351: AppMaster tracking URL is broken in HA
  • YARN-3385: KeeperException$NoNodeException causes ResourceManager shutdown during ZooKeeper node deletion.
  • YARN-3485: FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies
  • YARN-3487: CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
  • YARN-3493: After memory settings are changed in the yarn-site.xml file, the ResourceManager fails to start.
  • YARN-3522: DistributedShell uses the wrong user to put timeline data. Now DistributedShell instantiates TimeLineClient as the correct user.
  • YARN-3544: AM logs link missing in the RM UI for a completed app
  • YARN-3585: NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled.
  • YARN-3601: Fix the UT TestRMFailover.testRMWebAppRedirect test case.
  • YARN-3614: ResourceManager crashes when the FileSystemRMStateStore fails to remove an application.
  • YARN-3675: FairScheduler: RM quits when node removal races with continuous-scheduling on the same node.
  • YARN-3681: Yarn cmd says "could not find main class 'queue'" in windows.
  • YARN-3686: CapacityScheduler should trim default_node_label_expression.
  • YARN-3701: Isolating the error of generating a single app report when getting all apps from generic history service.
  • YARN-3723: Need to clearly document primaryFilter and otherInfo value type
  • YARN-3725: App submission via REST API is broken in secure mode due to Timeline DT service address is empty.
  • YARN-3733: Fix DominantRC#compare() does not work as expected if cluster resource is empty.
  • YARN-3753: RM failed to come up with "java.io.IOException: Wait for ZKClient creation timed out."
  • YARN-3764: CapacityScheduler should forbid moving LeafQueue from one parent to another.
  • YARN-3804: Both RM are on standBy state when kerberos user not in yarn.admin.acl
  • YARN-3832: Resource Localization fails on a cluster due to existing cache directories.
  • YARN-3840: Resource Manager web ui issue when sorting application by id (with application having id > 9999).
  • YARN-3842: NodeManager proxy should retry on NMNotYetReadyException.
  • YARN-3843: Fair Scheduler should not accept applications with spaces in the queue name.
  • YARN-3850: NodeMamager fails to read files from full disks which can lead to container logs being lost and other issues.
  • YARN-4256: YARN fair scheduler vcores with decimal values.