Hive 2.3.6-1912 (EEP 6.3.0) Release Notes
This section provides reference information, including new features, fixes, known issues, and limitations for Hive 2.3.6-1912 for EEP 6.3.0.
Below are release notes for the Hive component included in the MapR Converged Data Platform.
You may also be interested in the following:
These release notes contain only MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.
| Hive Version | 2.3.6 |
| Release Date | December 2019 |
| MapR Version Interoperability | See Hive and HCatalog Support Matrix and Ecosystem Support Matrix (Pre-5.2 releases) and EEP Components and OS Support. |
| Source on GitHub | https://github.com/mapr/hive |
| GitHub Release Tag |
2.3.6-mapr-1912 |
| Maven Artifacts | See Maven Artifacts for MapR. |
| Package Names | Navigate to https://package.ezmeral.hpe.com/releases/MEP/, and select your EEP and OS to view the list of package names. |
| ODBC / JDBC Drivers | Hive 2.3.6 works with the following MapR Hive ODBC drivers:For additional driver information, see Connecting to HiveServer2. |
Feature Support
- MapR supports Hive-2.3.6 on Tez-0.9.1. For more information, see Tez 0.9.1-1912 (EEP 6.3.0) Release Notes.
- MapR does not support Hive on Spark, so you cannot use Spark as an execution engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
- MapR does not support HDFS encryption in Hive tables.
- MapR supports HBase using Hive 2.3.6 starting with MEP 6.3.0 and MapR Core 6.1.0.
- MapR does not support LLAP with Hive 2.3.6, as Apache Slider is not a MapR ecosystem component.
- Starting from Hive 2.1, Hive needs to run the
schematoolcommand as an initialization step.
New Features
- None.
Known Issues
- HIVE-655 - Failed to launch Hive WebHCat job after update. See the workaround in Troubleshooting Hive and Tez.
- HIVE-19502 Unable to insert values into table stored by
JdbcStorageHandler - HIVE-19286 NPE in
MERGEoperator on MR mode - PIG-25 – Pig 0.16 failed to use ORC Storage. Pig - Hive dependency issue – MapR Pig does not support MapR Hive ORC integration.
- HIVE-522 – "INSERT INTO" overwrites old data when the destination table is
encapsulated by a backquote.
After HQL harmonization with requirements of the SQL standard, queries with table names or database names that contain a dot (
.) can fail. For detailed information, see Troubleshooting Hive and Tez.
Some select queries can be converted to single FETCH task minimizing latency. Currently,
the query should be single sourced not having any subquery and should not have any
aggregations or distincts (which incur RS), lateral views, and joins:
none:disable hive.fetch.task.conversionminimal: selectstar, filter on partition columns,LIMITonlymore: SELECT, filter,LIMITonly (support TABLESAMPLE and virtual columns)
IMPORTANT
Hive-2.3.6 MEP-6.3.0 requires Pig-0.16 MEP-6.3.0 and will not support Pig-0.16 from old MEPs. For details, see the Pig 0.16.0-1912 Release Notes.
Changes in Security with Default Configuration
- Added properties to default
hive-site.xmlconfiguration on secured cluster. See the following table:Table 1. Properties added to default hive-site.xml No. Property Value 1 hive.server2.webui.jetty.response.headers.file/opt/mapr/hive/hive-2.3/conf/headers.xml - Added properties to default
webhcat-site.xmlconfiguration on secured cluster. See the following table.
Note that the property above is added both in a Hive update (no version change) and a Hive upgrade (with version change).Table 2. Properties added to default webhcat-site.xml No. Property Value 1 templeton.jetty.response.headers.file/opt/mapr/hive/hive-2.3/conf/headers.xml
Fixes
This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.
| Commit | Date (YYYY-MM-DD) | Comment |
|---|---|---|
| 17fe3ee | 2019-11-20 | MAPR-HIVE-642: Joda Time Library update for Brazil to v2.10.3.jar |
| af42452 | 2019-11-19 | MAPR-HIVE-641: CLONE - old jetty version at hive lib |
| 3250b11 | 2019-07-01 | MAPR-HIVE-376 : Jetty web server version for HS2 web UI should be updated |
| b923f16 | 2019-10-31 | MAPR-HIVE-633: Update HBase dependency to 1.1.13 for MEP-6.3.0 |
| 3d3da90 | 2019-10-22 | MAPR-HIVE-618 : Add security headers for WebHCat (Part II) (#845) |
| 431b114 | 2019-10-18 | MAPR-HIVE-617 : Add Junit tests for configure.sh script. Verification of hive-site.xml and webhcat-site.xml |
| ba53b45 | 2019-10-18 | MAPR-HIVE-628 : Add utility method to represent xml properties as key-value map |
| 74a7408 | 2019-10-18 | MAPR-HIVE-627 : Remove unused option from configure.sh |
| 4fcf1ea | 2019-10-02 | MAPR-HIVE-618 : Add security headers for WebHCat |
| df40ce4 | 2019-09-30 | MAPR-HIVE-619 : Unnecessary excessive WebHCat security configuration on secure clusters after security refactoring |
| 1a5bde7 | 2019-09-23 | MAPR-HIVE-616 : Implement SBD behaviour in case of configuring Hive on KRB clusters after security refactoring |
| 6ba42a4 | 2019-09-20 | MAPR-HIVE-566: UDAF throws IllegalArgumentException for a complex input when column stats is not provided |
| 5d464bb | 2019-09-19 | MAPR-HIVE-614 : Hive on MapR DB JSON table does not honour the column's comment |
| 1e6abbf | 2019-09-17 | MAPR-HIVE-611: Hive - MapR-DB-json integration. NullPointerException in WHERE clause when no row found |
| ccdeb4a | 2019-09-02 | MAPR-HIVE-605 : Support of Avro 1.8 |
| f255efd | 2019-08-23 | MAPR-HIVE-604 : Process kerberos authentication method in HiveConf object |
| ea4d923 | 2019-09-05 | MAPR-HIVE-580: Hive on Mapr-db JSON table with nested schema showing wrong results |
| e3a107d | 2019-08-02 | MAPR-HIVE-593 : Spark thriftserver fails when work with hive-maprdb json table |
| 0ef33d7 | 2019-07-31 | MAPR-HIVE-586: hive.scratch.dir.permission doesn't work |
| d5455ce | 2019-07-27 | MAPR-HIVE-587 : Unable to retrieve Data through hcat/Pig to Hive |
| 7417cb6 | 2019-07-12 | MAPR-HIVE-573 : IndexOutOfBoundsException on parsing query |
| 27463ef | 2019-07-20 | MAPR-HIVE-583 : Configure hive.metastore.authentication for KRB and MAPRSASL security cluster |
| fec9184 | 2019-07-17 | MAPR-HIVE-559 : Hive configure.sh changes hive-site.xml permissions despite having .customSecure file presence |
| a48088c | 2019-06-14 | MAPR-HIVE-544 : Refactor Mapr SASL security implementation |
| 5b5b8dd | 2019-06-11 | MAPR-HIVE-541 : Hive creates Kerberos-based thrift connection when there is no delegation token on MapR SASL cluster |
| 65bc50a | 2019-06-08 | MAPR-HIVE-570 : ClassNotFoundException: HiveMapRDBJsonInputFormat in MaprDB-json integration |
| 1e6a377 | 2019-06-24 | MAPR-HIVE-556 : Error while starting Hive CLI on Tez: ERROR exec.Task: Failed to execute tez graph |
| 1d86356 | 2019-06-17 | MAPR-HIVE-520 : Use separate log file for each Hive service |
| 2286789 | 2019-06-10 | MAPR-HIVE-528 : UNION query with regular expressions works in Hive-2.1 and does not work in Hive-2.3 |
| 3d110189 | 2019-06-10 | MAPR-HIVE-534 : Change hive-maprdb-json-common version to 2.3.5 |
| dc6e806 | 2019-05-20 | MAPR-HIVE-519 : Hive ORC read INT, BIGINT as NULL for Data created by spark |
| 63b3829 | 2019-04-22 | MAPR-HIVE-509 : MapRDBJsonSplit class should be available for Spark |
This release by MapR also includes the following backported issues. For complete details, refer to the commit log for this project in GitHub.
| Commit | Date (YYYY-MM-DD) | Comment |
|---|---|---|
| f6f1ebb | 2019-11-12 | HIVE-19085: FastHiveDecimal abs(0) sets sign to +ve (Gopal V, reviewed by Matt McCline) |
| d5dd4e4 | 2019-09-06 | HIVE-17088: HS2 WebUI throws a NullPointerException when opened (Sergio Pena, reviewed by Aihua Xu) |
| 739b41c | 2019-06-27 | HIVE-16049: upgrade to jetty 9 (Sean Busbey & Aihua Xu, reviewed by Sergio Peña) |
| fced649 | 2019-10-25 | HIVE-22107: Correlated subquery producing wrong schema |
| 8bdc8158 | 2019-10-22 | HIVE-8472: Add ALTER DATABASE SET LOCATION (Mithun Radhakrishnan, reviewed by Alan Gates) (#851) |
| 3abf19c | 2019-10-22 | HIVE-20409: Hive ACID: Update/delete/merge does not clean hdfs staging directory (Rajkumar Singh, reviewed by Vineet Garg) (#847) |
| 626b50d | 2019-10-14 | HIVE-18553 : Support schema evolution in Parquet Vectorization reader (Ferdinand Xu, reviewed by Vihang Karajgaonkar) |
| 94190e2 | 2019-10-14 | HIVE-18323 : Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet (Vihang Karajgaonkar, reviewed by Aihua Xu and Ferdinand Xu) |
| cb2eb4d | 2019-10-14 | HIVE-17931: Implement Parquet vectorization reader for Array type |
| 6c78cdf | 2019-10-14 | HIVE-17381: When we enable Parquet Writer Version V2, hive throws an exception: Unsupported encoding: DELTA_BYTE_ARRAY. (Colin Ma, reviewed by Ferdinand Xu) |
| 622aab2 | 2019-10-11 | HIVE-17085: ORC file merge/concatenation should do full schema check (Prasanth Jaychandran reviewed by Zoltan Haindrich) |
| fece5c9 | 2019-07-16 | HIVE-21980: Parsing time can be high in case of deeply nested subqueries (Zoltan Haindrich reviewed by Vineet Garg) |
| d6bd1b6 | 2019-08-13 | HIVE-22096 Backport HIVE-21584 to branch-2.3 (Yuming Wang via Alan Gates) |
| fd97a4e | 2019-08-30 | HIVE-21859 Backport HIVE-17466 (get_partition_values) to branch-2.3 (Piotr Findeisen via Alan Gates) |
| 6752b7b | 2019-08-30 | HIVE-21786 : Update repo URLs in poms branch 2.3 version |
| a3860a7 | 2019-07-31 | HIVE-4605 : Hive job fails while closing reducer output - Unable to rename |
| 6a73e80 | 2019-07-26 | HIVE-14557: Nullpointer When both SkewJoin and Mapjoin Enabled |
| ef3a1f5 | 2019-07-25 | HIVE-21540: Query with join condition having date literal throws SemanticException (Sankar Hariappan, reviewed by Zoltan Haindrich) |
| 3d11872 | 2019-07-20 | HIVE-19990: Query with interval literal in join condition fails(Vineet Garg, reviewed by Zoltan Haindrich) |
| f26f063 | 2019-07-23 | HIVE-18786 : NPE in Hive windowing functions (Dongwook Kwon via Ashutosh Chauhan) |
| 45b037e | 2019-06-24 | HIVE-21104: PTF with nested structure throws ClassCastException (Rajesh Balamohan reviewed by Gopal |
| dee97c4 | 2019-06-22 | HIVE-20318 : NullPointerException when union with lateral view |
| 7e53093 | 2019-06-22 | HIVE-16334: Query lock contains the query string, which can cause OOM on ZooKeeper (Peter Vary via Chaoyu Tang) |
| 1616d4d | 2019-06-19 | HIVE-16907: "INSERT INTO" overwrite old data when destination table encapsulated by backquote |
| 9c26a52 | 2019-06-19 | HIVE-6590 : Hive does not work properly with boolean partition columns (wrong results and inserts to incorrect HDFS path) |
| 1e4cb68 | 2019-06-19 | HIVE-18907: Create utility to fix acid key index issue from HIVE-18817 (Jason Dere, reviewed by Prasanth Jayachandran) |
| 0ae141a | 2019-06-19 | HIVE-18817 - ArrayIndexOutOfBounds exception during read of ACID table. (Eugene Koifman, Jason Dere, Prasanth Jayachandran, reviewed by Jason Dere) |
| c7f7fd6 | 2019-05-03 | HIVE-21680 Backport HIVE-17644 to branch-2.3 (Yuming Wang via Alan Gates) |
| e822e4f | 2019-04-25 | Backport HIVE-17764 to branch-2.3 (Yuming Wang via Alan Gates) |
| 20e7e54 | 2019-04-24 | HIVE-21639 Spark test failed since HIVE-10632 (Yuming Wang via Alan Gates) |
| 8396169 | 2019-06-19 | HIVE-18624: Parsing time is extremely high (~10 min) for queries with complex select expressions (Zoltan Haindrich reviewed by Ashutosh Chauhan) |
Resolved Issues
- None.