Hive 2.3.8 - 2201 (EEP 7.1.2) Release Notes
The notes below relate specifically to the HPE Ezmeral Data
Fabric Distribution for Apache Hive. You may also be interested in the Apache Hive-2.3.8 Release Notes and the Apache Hive
homepage.
| Hive Version | 2.3.8 |
| Release Date | March 2022 |
| HPE Version Interoperability | See Hive and HCatalog Support Matrix andEcosystem Support Matrix and MEP Components and OS Support. |
| Source on GitHub | https://github.com/mapr/hive |
| GitHub Release Tag | 2.3.8-mapr-2201 |
| Maven Artifacts | https://repository.mapr.com/maven/ |
| Package Names | Navigate tohttp://package.ezmeral.hpe.com/releases/MEP/, and select your EEP and OS to view the list of package names. |
| ODBC/JDBC Drivers |
Hive 2.3.8 works with the following HPE Hive drivers:
For additional driver information, see Connecting to HiveServer2. |
Feature support
The following list describes support of various components and functionality with Hive 2.3.8 - 2201:
-
Supports Hive-2.3.8 on Tez-0.9.2 For more information, see Tez 0.9.2 - 2201 (EEP 7.1.2) Release Notes.
-
Does not support Hive on Spark. You cannot use Spark as an query engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
-
Does not support HDFS encryption in Hive tables.
-
Does not support LLAP with Hive-2.3.8 because Apache Slider is not a HPE supported ecosystem component.
-
Starting from Hive 2.1, Hive needs to run the
schematoolcommand as an initialization step.
Changes in default security configuration
The following list describes changes in default security for Hive 2.3.8 - 2201:
- None.
New in This Release
Hive 2.3.8 - 2201 introduces the following enhancements or HPE platform-specific behavior changes:
- Added configuration to view audit logs for connected, disconnected, and total connected users in HiveServer2.
- Added Service verifier.
Fixes
This HPE release includes the following fixes on the base release:
| Commit | Date (YYYY-MM-DD) | Comment |
|---|---|---|
| 3a48a4b629 | 2022-01-12 | MAPR-HIVE-1071 : Fix SslDefaultTest |
| 53ac871854 | 2022-01-12 | EEP-HIVE-1132 : Update Hadoop version to 2.7.6.0-mapr-712-SNAPSHOT in MEP-712 |
| f0d52b51ec | 2022-01-06 | EEP-HIVE-1065: CVE-2021-37136, CVE-2021-37137, WS-2020-0408, CVE-2021-21290: netty-*-4.1.55.Final.jar |
| 7071a6a977 | 2022-01-05 | EEP-HIVE-1062: CVE-2016-5007, CVE-2016-9878 ,CVE-2018-1271, CVE-2018-1272, CVE-2020-5421: spring-*-3.2.16.RELEASE.jar |
| 6c800f885f | 2022-01-05 | EEP-HIVE-1068: CVE-2020-9480: spark-network-common_2.11-2.3.0.jar, CVE-2018-17190: spark-core_2.11-2.3.0.jar |
| 70364dcbcb | 2022-01-04 | EEP-HIVE-1117: Update log4j v2 to the latest available (to 2.17+) |
| 2a3f1318a5 | 2022-01-04 | EEP-HIVE-1116: Hive returns an incorrect number of columns |
| 563da5a5fa | 2022-01-04 | Revert "MAPR-HIVE-930: Cannot run join with Order by and Limit clause specified at the same time" |
| d51bbd22c5 | 2021-12-30 | EEP-HIVE-1119: com.fasterxml.jackson.annotation.JsonFormat.empty()Lcom/fasterxml/jackson/annotation/JsonFormat |
| 737df2650d | 2021-12-24 | EEP-HIVE-1064: CVE-2021-30639 ; CVE-2021-33037: tomcat-coyote-10.0.4.jar |
| 2d0a5f247f | 2021-12-24 | EEP-HIVE-1059: CVE-2019-10172, CVE-2019-10202: jackson-mapper-asl-1.9.13.jar, jackson-mapper-asl-1.9.2.jar |
| a90c3290ca | 2021-12-24 | EEP-HIVE-1056: CVE-2021-35515, CVE-2021-35516, CVE-2021-35517, CVE-2021-36090: commons-compress-1.20.jar |
| 5fd89c2f7f | 2021-12-24 | EEP-HIVE-1055: CVE fixes of bcprov-jdk15on-1.52.jar |
| d1fec1c964 | 2021-12-24 | EEP-HIVE-1054: WS-2021-0419: gson-2.2.4.jar |
| 3b092554d6 | 2021-12-16 | MAPR-HIVE-1002 : Hive-2.3 does not remove old compressed logs |
| 9ca658cde2 | 2021-12-14 | EEP-HIVE-1097: CVE-2021-44228 - Log4j vulnerability |
| 1f6b1fc3cf | 2021-12-03 | EEP-HIVE-1087: CAST gives NULL values during insert when vectorization enabled. |
| 81dacfe359 | 2021-11-25 | MAPR-HIVE-1086 : Upgrade Jetty to 9.4.44.v20210927 |
| f858bbd8a3 | 2021-11-22 | MAPR-HIVE-1007: Permission denied to hbase temp files while running hcat jobs from other user |
| b9aed0dfb9 | 2021-11-22 | MAPR-HIVE-977 : Downgrade jackson to v2.11.1 or to 2.11.3 to be consistent with core version |
| 57e796b851 | 2021-11-22 | MAPR-HIVE-960 : CVE-2012-5783 vulnerability in commons-httpclient |
| aecc898b7a | 2021-11-22 | MAPR-HIVE-965 : Throws exception at INSERT statement with Avro table on MapReduce engine |
| d71410e462 | 2021-11-22 | MAPR-HIVE-959 : Update derbyclient and derbynet to most feasible version |
| e1d0cdc7ca | 2021-11-22 | MAPR-HIVE-963 : CVE-2020-13956,WS-2017-3734 vulnerabilities in httpclient |
| 993b83adb9 | 2021-11-22 | MAPR-HIVE-962 : WS-2019-0379: commons-codec vulnerability |
| 642c2b0ad2 | 2021-11-22 | MAPR-HIVE-896 : CVE-2020-17521 vulnerability in Groovy |
| 8dd7dc8ff3 | 2021-11-22 | MAPR-HIVE-927 : NPE thrown from XmlUtil by Hive Client |
| 9f152f2318 | 2021-11-22 | MAPR-HIVE-950 : HiveVersionInfo.getShortVersion returns wrong version |
| f986d83c36 | 2021-11-11 | MAPR-HIVE-1031 : logError: command not found if any error happens during configuring Hive |
| 9f85d37d4b | 2021-11-11 | MAPR-HIVE-1024 : Replace deprecated AuthMethod.DIGEST with AuthMethod.TOKEN in HadoopThriftAuthBridge25Sasl |
| 5d7040f540 | 2021-11-11 | MAPR-HIVE-975: Customer request to investigate temporary hive session files cleanup improvements |
| a6e9bde12c | 2021-09-30 | MAPR-HIVE-994 : Non mapr user unable to read SSL configuration from XML files on Core 7.0 |
| f0f1d78c10 | 2021-09-16 | DFDEVOPS-1979:Move WS scan to the shared library. |
| 9b4c3ad944 | 2021-06-08 | MAPR-HIVE-949 : Make SchemaEvolution class behavior the same as in Orc-1.5.12 |
| 9b75385d22 | 2021-06-06 | MAPR-HIVE-894 : CVE-2020-13955 vulnerability in Calcite |
| 2a7526fdea | 2021-05-25 | MAPR-HIVE-947 : org.apache.hadoop.hive.ql.exec.tez.TezTask at creating session |
| a64ce8195c | 2021-05-23 | MAPR-HIVE-945 : FAILURE! - in org.apache.hadoop.hive.maprdb.json.MapRDbJsonFetchByIdOptimizerPositiveTest |
| 4215a9ab82 | 2021-05-23 | MAPR-HIVE-944 : FAILURE! - in org.apache.hadoop.hive.hbase.TestHBaseSerDe |
| 30655745a5 | 2021-05-23 | MAPR-HIVE-943 : Fix org.apache.hadoop.hive.cli.TestCliDriverMethods |
| bfda9cc9a8 | 2021-05-23 | MAPR-HIVE-942 : FAILURE! - in org.apache.hadoop.hive.accumulo.predicate.TestAccumuloPredicateHandler |
This release from HPE also includes the following back-ported issues. For complete details,
refer to the commit log for this project in GitHub.
| Commit | Date (YYYY-MM-DD) | Comment |
|---|---|---|
| 0e6ddbd638 | 2021-12-16 | HIVE-16820 : TezTask may not shut down correctly before submit (Sergey Shelukhin, reviewed by Siddharth Seth) |
| ba928dded2 | 2021-11-22 | HIVE-24965: Describe table partition stats fetch should be configurable |
| a1d6b137d1 | 2021-11-22 | HIVE-22453: Describe table unnecessarily fetches partitions |
| 03bb3ad2ef | 2021-11-22 | HIVE-23756 : Added more constraints to the package.jdo file |
| b5670273db | 2021-11-22 | HIVE-24177 : hive mapjoin throws udf class not found |
| 37f70e1f09 | 2021-11-22 | HIVE-25054: Upgrade `jodd-core` dependency to get rid of CVE-2018-21234 (Abhay Chennagiri, reviewed by Jesus Camacho Rodriguez) |
| f870fdc54b | 2021-11-22 | HIVE-17659 : get_token thrift call fails for DBTokenStore in remote HMS mode (Vihang Karajgaonkar, reviewed by Aihua Xu) |
| 9dbd2ceb60 | 2021-11-22 | HIVE-17824 : msck repair table should drop the missing partitions from metastore (Janaki Lahorani, reviewed by Peter Vary, Alexander Kolbasov and Vihang Karajgaonkar) |
| cbc7f88a61 | 2021-11-22 | HIVE-16143: Improve msck repair batching (Vihang Karajgaonkar, reviewed by Sahil Takiar & Aihua Xu) |
| f2067dedea | 2021-12-08 | HIVE-17774: compaction may start with 0 splits and fail |
| 1ccb218119 | 2021-05-22 | HIVE-21085: Materialized views registry starts non-external tez session (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
| dea7190511 | 2021-05-22 | HIVE-19691: Start SessionState in materialized views registry (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
| 09b4ca437f | 2021-05-22 | HIVE-17853: RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout (Chris Drome, reviewed by Mithun Radhakrishnan) |
| b8902a7bb8 | 2021-05-22 | HIVE-23534: NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message (Stamatis Zampetakis, reviewed by Jesus Camacho Rodriguez) |
| 11db00d681 | 2021-05-22 | HIVE-18494: Regression: from HIVE-18069, the metastore directsql is getting disabled (Jesus Camacho Rodriguez, reviewed by Gopal V |
| 1920988b66 | 2021-05-22 | HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu) (addendum) |
| e5ed2cb9ed | 2021-05-22 | HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu) |
| 9b506546a4 | 2021-05-22 | HIVE-15436: Enhancing metastore APIs to retrieve only materialized views (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
| 051002d23a | 2021-05-22 | HIVE-6990 : Direct SQL fails when the explicit schema setting is different from the default on (Bing Li, Sergey Shelukhin via Ashutosh Chauhan) |
| 9120da5c4f | 2021-05-31 | HIVE-21075 : Metastore: Drop partition performance downgrade with Postgres DB |
| 39d42ddf12 | 2021-05-31 | HIVE-9447: Metastore: inefficient Oracle query for removing unused column descriptors when add/drop table/partition (Selina Zhang reviewed by Ashutosh Chauhan, Adam Szita) |
| 0912498e97 | 2021-11-11 | HIVE-20072 : Write access being requested when performing select on a table |
| 11e8219ace | 2021-11-22 | HIVE-19228: Remove commons-httpclient 3.x usage (Janaki Lahorani reviewed by Aihua Xu) |
| 1fc7585a2e | 2021-06-08 | ORC-437: Make acid schema checks case insensitive |
Known Issues and Limitations
-
HIVE-19502 Unable to insert values into table stored by JdbcStorageHandler
-
HIVE-19286 NPE in MERGE operator on MR mode
-
HIVE-789 [Hive-Hbase integration] Unable to run queries against hive-hbase tables. ClassNotFoundException: HiveHBaseTableInputFormat [MEP-7.0.0]
If you run an HBase + Hive + Tez integration in MEP 7.1.0, you may encounter the
following
exception:
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.client.mapr.BaseTableMappingRules
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 39 more
This exception can occur due to the new Tez classloader implemented in the Tez project. To
resolve this issue, put the following additional JAR files in the
/apps/tez/tez-0.9 folder.
Issue the following commands before you run HBase + Hive + Tez integration in MEP 7.1.0:
hadoop fs -mkdir /apps/tez/tez-0.9/hbase
hadoop fs -put /opt/mapr/hbase/hbase-1.4.12/lib/* /apps/tez/tez-0.9/hbase/
Add the following property to
/opt/mapr/tez/tez-0.9/conf/tez-site.xml:<property>
<name>tez.lib.uris</name>
<value>${fs.defaultFS}/apps/tez/tez-0.9,${fs.defaultFS}/apps/tez/tez-0.9/lib,${fs.defaultFS}/apps/tez/tez-0.9/hbase/</value>
</property>
It is assumed that the Hive version is 2.3, Hbase version is 1.4.12, Tez version is 0.9, Hadoop version is 2.7.4, Zookeeper version is 3.5.6.0, and ecosystem release is 2009.
HIVE-760 [Hive-2.3] Could not start hive-metastore on Centos 8 MetaException(message:Version information not found in metastore)
Starting in MEP 7.0.0, use the MySQL driver with MariaDB.
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
Some SELECT queries can be converted to a single FETCH task minimizing latency. Currently,
the query should be single sourced and should not have a subquery or any aggregations or
distincts (which incurs RS), lateral views and
joins:
none : disable hive.fetch.task.conversion
minimal : SELECT star, filter on partition columns, LIMIT only
more : SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
Resolved Issues
- None.