Spark 1.5.2-1608 Release Notes
The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.5.2 Release Notes.
Spark Version | 1.5.2 |
Release Date | September 1, 2016 |
MapR Version Interoperability | See the Ecosystem Support Matrix (Pre-5.2 releases) and Spark Support Matrix. |
Source on GitHub | https://github.com/mapr/spark/tree/1.5.2-mapr-1608 |
Package Names | The following packages are associated with this release:
|
Hive Support
This version of Spark supports integration with Hive. However, note the following exceptions:
- Hive on Spark is not supported.
- Spark-SQL is supported but it is not fully compatible with Hive; see the Apache Spark documentation and MapR's Spark documentation for details.
Important Notes
This release of Apache Spark for MapR includes the following shared library
dependencies:
- Mesos version: 0.21.1
- HBase version: 0.98.12-mapr-1506
- Flume version 1.6.0
- Zookeeper version: 3.4.5-mapr-1503
- Hive version: 1.2.0-mapr-1603
- Hive parquet version: 1.6.0
- Kafka version: 0.9.0.0
- Kafka clients version: 0.9.0.0-mapr-1602
Fixes
This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.
GitHub Commit | Date (YYYY-MM-DD) | Comments |
---|---|---|
ac11283 | 2016-08-02 | MAPR-24129: Backported SPARK-16796 so that passwords are now masked on the Environment page of the Spark History Server UI. |
0bda0b9 | 2016-08-18 | MARP-24264: The following fixes have been backported to resolve the Spark
vulnerability to code injection in the toCommentSafeString method:
|
fc72384 | 2016-08-22 | MAPR-24318: Backported SPARK -1191 and SPARK-11311 so that Spark is now able to create UDF's using the Hive Thrift Service. |
681dcf4 | 2016-08-24 | MAPR-24263: The Spark assembly and example JAR files no longer include a dependency on MapR-FS libraries. |
Known Issues
- MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
- MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
- MAPR-22940/SPARK-11851: On clusters that use Kerberos authentication, Spark Thrift Server is unable to connect to beeline.
- Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability