Spark 1.6.1-1611 Release Notes
The notes below relate specifically to the MapR Distribution for Apache Hadoop. You may also be interested in the open-source Spark 1.6.1 Release Notes.
Spark Version | 1.6.1 |
Release Date | December 9, 2016 |
MapR Version Interoperability | See EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/spark |
GitHub Release Tag | 1.6.1-mapr-1611 |
Package Names | The following packages are associated with this release:
|
New in This Release
This version of Spark supports integration with Hive and MapR Event Store For Apache Kafka. However, note the following exceptions:
- Hive on Spark is not supported.
- Spark-SQL is supported, but it is not fully compatible with Hive; for details, see the Apache Spark documentation and the MapR Spark documentation.
- If you want to integrate Spark 1.6.1 with MapR Event Store For Apache Kafka, you must install the latest Kafka 0.9.0.0 package.
- Full support of MapR Event Store For Apache Kafka is available only on MapR 5.2 clusters.
Fixes
This MapR release includes the following new fixes since the previous release of MapR Spark 1.6.1. For details, refer to the commit log for this project in GitHub.
GitHub Commit Number | Data (YYYY-MM-DD) | MapR Fix Number and Description |
---|---|---|
50fbc6b | 2016-09-23 | [SPARK-17210][SPARKR] The sparkr.zip archive is not distributed to executors when running sparkr in RStudio. |
bba4dfa | 2016-09-29 | MAPR-24602: Spark RStudio jobs failed with a java.net.SocketTimeoutException. |
e837cec | 2016-10-10 | [MAPR-24635] The Spark History Server takes a long time to display the stage details. |
847bc6f | 2016-10-24 | [MAPR-24870] Reverted changes that were made in MAPR-23243. |
3edeef6 | 2016-10-28 | [MAPR-24415] SPARK_JAVA_OPTS is deprecated. |
175c9b5 | 2016-11-04 | [MAPR-13522] [CORE] Executor should kill itself when it is unable to send a heartbeat to the driver more than n times. |
92e042e | 2016-11-04 | [MAPR-13522] [CORE] Fixed the exit log place for heartbeat. |
26990db | 2016-11-11 | [MAPR-25223] SparkR did not work. |
Known Issues and Limitations
- MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark Master and Spark History Server.
- MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift Server will not start.
- Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability
Resolved Issues
None.