List of YARN Enhancements in Version 5.2.0
MapR 5.2 runs the 2.7.0 version of Hadoop. In the MapR 5.2.0 release, a number of bug fixes from various Hadoop versions were backported into 2.7.0.
Bug fixes backported in 5.2.0
-
YARN-4452: NPE when submit Unmanaged application
-
YARN-4422: Generic AHS sometimes doesn't show started, node, or logs on App page
-
YARN-3849: Too much of preemption activity causing continuos killing of containers across queues
-
YARN-4794: Deadlock in NMClientImpl
-
YARN-4546: ResourceManager crash due to scheduling opportunity overflow
-
MAPREDUCE-6622: Add capability to set JHS job cache to a task-based limit
-
MAPREDUCE-6635: Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException
-
MAPREDUCE-6454: MapReduce doesn't set the HADOOP_CLASSPATH for jar lib in distributed cache.
-
MAPREDUCE-6376: Add avro binary support for jhist files
-
MAPREDUCE-6554: MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
-
MAPREDUCE-6302: Preempt reducers after a configurable timeout irrespective of headroom
-
MAPREDUCE-6514: Job hangs as ask is not updated after ramping down of all reducers
-
MAPREDUCE-6689: MapReduce job can infinitely increase number of reducer resource requests
-
MAPREDUCE-5485: Allow repeating job commit by extending OutputCommitter API
-
MAPREDUCE-6555: TestMRAppMaster fails on trunk
-
MAPREDUCE-6595: Fix findbugs warnings in OutputCommitter and FileOutputCommitter
-
MAPREDUCE-6478: Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob().
-
YARN-4424: Fix deadlock in RMAppImpl
-
YARN-4354: Public resource localization fails with NPE
-
YARN-2902: Killing a container that is localizing can orphan resources in the DOWNLOADING state
-
YARN-4180: AMLauncher does not retry on failures when talking to NM
-
YARN-3697: FairScheduler: ContinuousSchedulingThread can fail to shutdown
-
YARN-3535: Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED
-
YARN-3857: Memory leak in ResourceManager with SIMPLE mode
-
YARN-3878: AsyncDispatcher can hang while stopping if it is configured for draining events on stop
-
YARN-3990: AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected
-
YARN-3793: Several NPEs when deleting local files on NM recovery
-
YARN-4047: ClientRMService getApplications has high scheduler lock contention
-
HADOOP-12304: Applications using FileContext fail with the default file system configured to be wasb/s3/etc.
-
MAPREDUCE-5948: org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well
-
MAPREDUCE-6481: LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes.
-
MAPREDUCE-6492: AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask
-
MAPREDUCE-5982: Task attempts that fail from the ASSIGNED state can disappear
-
MAPREDUCE-5649: Reduce cannot use more than 2G memory for the final merge
-
MAPREDUCE-6549: Multibyte delimiters with LineRecordReader cause duplicate records
-
YARN-2336: Fair scheduler REST api returns a missing '[' bracket JSON for deep queue tree
-
YARN-1050: Document the Fair Scheduler REST API
-
YARN-3464: Race condition in LocalizerRunner kills localizer before localizing all resources
-
YARN-4041: Slow delegation token renewal can severely prolong RM recovery
-
YARN-4347: Resource manager fails with Null pointer exception
-
MAPREDUCE-6284: Add Task Attempt State API to MapReduce Application Master REST API
-
MAPREDUCE-6316: Task Attempt List entries should link to the task overview
-
MAPREDUCE-6364: Add a "Kill" link to Task Attempts pageAdd a "Kill" link to Task Attempts page
-
MAPREDUCE-6405: NullPointerException in App Attempts page
-
MAPREDUCE-6419: JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999
-
MAPREDUCE-5649: Reduce cannot use more than 2G memory for the final merge