Spark 2.2.1 - 2101 (EEP 5.0.6) Release Notes
The notes below relate specifically to the HPE Ezmeral Data Fabric distribution for Apache Hadoop. You may also be interested in the open source Spark 2.2.1 Release Notes.
Spark Version | 2.2.1 |
Release Date | January, 2021 |
MapR Version Interoperability | See Component Versions for Released EEPs and EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/spark |
GitHub Release Tag | 2.2.1-mapr-2101 |
Maven Artifacts | https://repository.mapr.com/maven/ |
Package Names | Package Names for MapR Ecosystem Packs (MEPs) |
Important Notes
-
Although Spark 2.2 can connect to Hive Metastore 2.1, features of Hive that were added after Hive 1.2 are not supported by Spark.
As of Spark 2.2.1 and EEP 5.0, Spark uses Kafka-1.0.1.
-
Spark Yarn and standalone modes are only supported on clusters in MRv2 (YARN) mode. Spark Yarn and standalone modes not supported on clusters in MRv1 (classic) mode.
-
Core 6.0 and EEP 5.0 introduce "Simplified Security." If you are using these versions and enable security in your cluster, scripts automatically configure Spark security features.
Hive Support
- Hive-on-Spark is not supported.
- Spark-SQL is supported, but not fully compatible with Hive. See the Apache Spark documentation and the HPE Spark documentation for details.
Fixes
GitHub Commit | Date (YYYY-MM-DD) | Comment |
5c6b8d6 | 2020/11/04 | MapR [SPARK-804] sparkContext.loadFromMapRDB throwing RuntimeException in scala code |
aecb512 | 2020/11/23 | MapR [SPARK-798] Structured Streaming rewinds offset to 0 upon re-subscription |
db7098c | 2021/01/11 | MapR [SPARK-834] Backport SPARK-795 into Spark 2.2.1 (MEP 5.x) |
Known Issues and Limitations
None.
Resolved Issues
None.