Spark 1.6.1-1608 Release Notes
The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.6.1 Release Notes
Spark Version | 1.6.1 |
Release Date | September 1, 2016 |
MapR Version Interoperability | See the Interoperability Matrices, Ecosystem Support Matrix (Pre-5.2 releases), and Spark Support Matrix. |
Source on GitHub | https://github.com/mapr/spark/tree/1.6.1-mapr-1608 |
Package Names | The following packages are associated with this release:
|
Important Notes
If you want to integrate Spark 1.6.1-1608 with HPE Ezmeral Data Fabric Streams, you must install the Kafka 0.9.0-1607 package.
Hive Support
This version of Spark supports integration with Hive. However, note the following exceptions:
- Hive on Spark is not supported.
- Spark-SQL is supported but it is not fully compatible with Hive; see the Apache Spark documentation and MapR's Spark documentation for details.
Fixes
This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.
GitHub Commit | Date (YYYY-MM-DD) | Comments |
---|---|---|
3024b71 | 2016-08-01 | MAPR-24129: Backported SPARK-16796 so that passwords are now masked on the Environment page of the Spark History Server UI. |
7755848 | 2016-08-16 | [MARP-24264] The following fixes have been backported to resolve the Spark
vulnerability to code injection in the toCommentSafeString method:
|
04c451a | 2016-08-18 | [MAPR-24292] HPE Ezmeral Data Fabric Streams consumer no longer fails to read the first message when the message contains a single string. |
12df59b | 2016-08-25 | [MAPR-24263] The Spark assembly and example JAR files no longer include a dependency on MapR-FS libraries. |
Known Issues
- MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
- MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
- Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability