Spark 2.2.1-1904 (EEP 5.0.3) Release Notes
This section provides reference information, including new features, patches, and known issues for Spark 2.2.1.
The notes below relate specifically to the MapR Distribution for Apache Hadoop. This release of Spark has backward-compatibility changes, see the open-source Spark 2.2.1 Release Notes for more information.
These release notes contain only MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.
Spark Version | 2.3.3 |
Release Date | April 2019 |
MapR Version Interoperability | See EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/spark |
GitHub Release Tag | 2.2.1-mapr-1904 |
Maven Artifacts | https://repository.mapr.com/maven/ |
Package Names | Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names. |
- Starting with EEP 6.0.0,
keyStore
andtrustStore
passwords can be removed from thespark-defaults.conf
file and can be set in the/opt/mapr/conf/ssl-client.xml
file. - Starting with EEP 6.0.0, after an upgrade,
configuration files of previous versions are saved in the
/opt/mapr/spark
directory. - The MapR 6.1 and EEP 6.0.0 release introduces "Simplified Security". If you are using these versions and enable security on your MapR cluster, MapR scripts automatically configure Spark security features.
Hive Support
This version of Spark supports integration with Hive. However, note the following exceptions:
- Hive-on-Spark is not supported.
- Spark-SQL is supported, but it is not fully compatible with Hive. For details, see the Apache Spark documentation and the MapR Spark documentation.
New in This Release
- For a complete list of all new features, refer to the open source documentation.
Fixes
This MapR release includes the following new fixes since the latest MapR Spark 2.3.1 release. For details, refer to the commit log for this project in GitHub.
GitHub Commit | Date (YYYY-MM-DD) | Comment |
---|---|---|
41854fb | 2019/01/23 | MapR [SPARK-396] Interface change of sendToKafka |
d6c12f5 | 2019/02/05 | [SPARK-383] Avoid using 'SimpleChannelInboundHandler' to prevent 'TypeParameterMatcher' generation |
26388c1 | 2019/02/07 | [SPARK-10878][CORE] Fix race condition when multiple clients resolvesartifacts at the same time |
a2f38c9 | 2019/02/15 | [SPARK-449] Kafka offset commit issue fixed |
ef8f5cd | 2019/03/07 | CVE-2018-1320 vulnerability in Apache Thrift |
314e8e4 | 2019/03/12 | MapR [SPARK-445] Messages loss |
4a9322d | 2019/03/15 | MapR [SPARK-514] Recovery from checkpoint is broken |
14618bb | 2019/03/19 | SPARK-463] MAPR_MAVEN_REPO variable for specifying mapR repository |
b9e7086 | 2019/03/26 | [SPARK-508] MapR-DB OJAI Connector for Spark isNull condition returns incorrect result |
924aa51 | 2019/03/28 | MapR [SPARK-510] nonmapr "admin" users not able to view other user logs in SHS |
2ca0af1 | 2019/03/28 | MapR [SPARK-461] Stop graph after jobs completion to prevent 'java.lang.IllegalStateException: No active subscriptions' |
0607a96 | 2019/04/04 | [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS instead of ip address |
ff48dbc | 2019/04/11 | MapR [SPARK-444] Fix of hive version for spark dev branches |
Known Issues
- None.
Resolved Issues
- None.