Spark 2.2.1 - 2101 (EEP 5.0.6) Release Notes

The notes below relate specifically to the HPE Ezmeral Data Fabric distribution for Apache Hadoop. You may also be interested in the open source Spark 2.2.1 Release Notes.

Spark Version 2.2.1
Release Date January, 2021
MapR Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub https://github.com/mapr/spark
GitHub Release Tag 2.2.1-mapr-2101
Maven Artifacts https://repository.mapr.com/maven/
Package Names Package Names for MapR Ecosystem Packs (MEPs)

Important Notes

  • Although Spark 2.2 can connect to Hive Metastore 2.1, features of Hive that were added after Hive 1.2 are not supported by Spark.

    As of Spark 2.2.1 and EEP 5.0, Spark uses Kafka-1.0.1.

  • Spark Yarn and standalone modes are only supported on clusters in MRv2 (YARN) mode. Spark Yarn and standalone modes not supported on clusters in MRv1 (classic) mode.

  • Core 6.0 and EEP 5.0 introduce "Simplified Security." If you are using these versions and enable security in your cluster, scripts automatically configure Spark security features.

Hive Support

This version of Spark supports integration with Hive. However, note the following exceptions:

Fixes

This release includes the following new patches since the latest HPE Spark 2.2.1 release. For complete details, refer to the commit log for this project in GitHub.
GitHub Commit Date (YYYY-MM-DD) Comment
5c6b8d6 2020/11/04 MapR [SPARK-804] sparkContext.loadFromMapRDB throwing RuntimeException in scala code
aecb512 2020/11/23 MapR [SPARK-798] Structured Streaming rewinds offset to 0 upon re-subscription
db7098c 2021/01/11 MapR [SPARK-834] Backport SPARK-795 into Spark 2.2.1 (MEP 5.x)

Known Issues and Limitations

None.

Resolved Issues

None.