List of YARN Enhancements
The following bug fixes were backported from Hadoop 2.7.1 to Hadoop 2.7.0:
- HADOOP-11872: On Windows clients, the hadoop fs command prints an incorrect message that says to use yarn jar instead of hadoop fs.
- HADOOP-12473: DistCp's ignoring failures option should be mutually exclusive with the atomic option.
- MAPREDUCE-6238: MR2 can't run local jobs with -libjars command options which is a regression from MR1
- MAPREDUCE-6324: Uber jobs fail to update AMRM token when it rolls over.
- MAPREDUCE-6334: Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler
- MAPREDUCE-6339: Job history file is not flushed correctly because isTimerActive flag is not set true when flushTimerTask is scheduled.
- MAPREDUCE-6353: Divide by zero error in MR AM when calculating available containers
- MAPREDUCE-6361: MR2 cannot run local jobs with -libjars command options which is a regression from MR1.
- MAPREDUCE-6410: Aggregated log deletion does not work after refreshing Log Retention Settings in a secure cluster.
- YARN-3351: AppMaster tracking URL is broken in HA
- YARN-3385: KeeperException$NoNodeException causes ResourceManager shutdown during ZooKeeper node deletion.
- YARN-3485: FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies
- YARN-3487: CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
- YARN-3493: After memory settings are changed in the yarn-site.xml file, the ResourceManager fails to start.
- YARN-3522: DistributedShell uses the wrong user to put timeline data. Now DistributedShell instantiates TimeLineClient as the correct user.
- YARN-3544: AM logs link missing in the RM UI for a completed app
- YARN-3585: NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled.
- YARN-3601: Fix the UT TestRMFailover.testRMWebAppRedirect test case.
- YARN-3614: ResourceManager crashes when the FileSystemRMStateStore fails to remove an application.
- YARN-3675: FairScheduler: RM quits when node removal races with continuous-scheduling on the same node.
- YARN-3681: Yarn cmd says "could not find main class 'queue'" in windows.
- YARN-3686: CapacityScheduler should trim default_node_label_expression.
- YARN-3701: Isolating the error of generating a single app report when getting all apps from generic history service.
- YARN-3723: Need to clearly document primaryFilter and otherInfo value type
- YARN-3725: App submission via REST API is broken in secure mode due to Timeline DT service address is empty.
- YARN-3733: Fix DominantRC#compare() does not work as expected if cluster resource is empty.
- YARN-3753: RM failed to come up with "java.io.IOException: Wait for ZKClient creation timed out."
- YARN-3764: CapacityScheduler should forbid moving LeafQueue from one parent to another.
- YARN-3804: Both RM are on standBy state when kerberos user not in yarn.admin.acl
- YARN-3832: Resource Localization fails on a cluster due to existing cache directories.
- YARN-3840: Resource Manager web ui issue when sorting application by id (with application having id > 9999).
- YARN-3842: NodeManager proxy should retry on NMNotYetReadyException.
- YARN-3843: Fair Scheduler should not accept applications with spaces in the queue name.
- YARN-3850: NodeMamager fails to read files from full disks which can lead to container logs being lost and other issues.
- YARN-4256: YARN fair scheduler vcores with decimal values.