Spark 1.6.1-1703 Release Notes
The notes below relate specifically to the MapR Distribution for Apache Hadoop. You may also be interested in the open-source Spark 1.6.1 Release Notes.
| Spark Version | 1.6.1 |
| Release Date | April 2017 |
| MapR Version Interoperability | See EEP Components and OS Support. |
| Source on GitHub | https://github.com/mapr/spark |
| GitHub Release Tag | 1.6.1-mapr-1703 |
| Maven Artifacts | https://repository.mapr.com/maven/ |
| Package Names | See Package Names for Ezmeral Ecosystem Packs (EEPs) |
NOTE For some important Spark limitations, See "Known Issues and Limitations" later in this
release note.
New in This Release
This version of Spark supports integration with Hive. However, note the following exceptions:
- Hive-on-Spark is not supported.
- Spark-SQL is supported, but it is not fully compatible with Hive. For details, see the Apache Spark documentation and the MapR Spark documentation.
Fixes
This MapR release includes the following new fixes since the latest MapR Spark 1.6.1 release. For details, refer to the commit log for this project in GitHub.
| GitHub Commit | Date (YYYY-MM-DD) | MapR Fix Number and Description |
|---|---|---|
| 9275605 | 2017/03/16 | Fixed gap handling. |
| dbf3b16 | 2017/03/10 | Removed overridden methods count, countApprox, isEmpty from kafkaRDD. |
| 3014abc | 2017/03/07 | [MAPR-26040] Kafka streaming module was refactored (#96). |
| 62def95 | 2017/03/06 | [BUILD][1.6] Fix compilation. |
| e092b35 | 2017/02/23 | Merge pull request #90 from mapr/spark-1.6.3-porting. |
| 04cbcb0 | 2017/02/23 | [SPARK-17696][SPARK-12330][CORE] Partial backport of to branch-1.6. |
| 9be6167 | 2017/02/13 | [SPARK-14357][CORE] Properly handle the root cause being a commit denied exception. |
| 566a5a9 | 2017/02/13 | [SPARK-16182][CORE] Utils.scala -- terminateProcess() should call Process.destroyForcibly() if and only if Process.destroy() fails. |
| 31843fc | 2017/02/13 | [SPARK-17618] Fix invalid comparisons between UnsafeRow and other row formats. |
| 2781e0b | 2017/02/13 | [SPARK-8428][SPARK-13850] Fix integer overflows in TimSort. |
Known Issues and Limitations
- To integrate Spark 1.6.1 with MapR Event Store For Apache Kafka, you must install the latest Kafka 0.9.0.0 package.
- Full support of MapR Event Store For Apache Kafka is available only on clusters with MapR 5.2 and later.
- MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
- MAPR-19761: On a secure cluster, MapR software does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
- Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability
Resolved Issues
None.