Hive 2.3.6 - 2110 (EEP 6.3.5) Release Notes

The notes below relate specifically to the MapR Data Platform Distribution for Apache Hive.
Hive Version 2.3.6
Release Date October 2021
HPE Version Interoperability See Hive and HCatalog Support Matrix and Ecosystem Support Matrix and MEP Components and OS Support.
Source on GitHub https://github.com/mapr/hive
GitHub Release Tag 2.3.6-mapr-2110
Maven Artifacts https://repository.mapr.com/maven/
Package Names Navigate to http://package.ezmeral.hpe.com/releases/MEP/, and select your MEP and OS to view the list of package names.
ODBC/JDBC Drivers
Hive 2.3.6 works with the following MapR Hive drivers:

For additional driver information, see Connecting to HiveServer2.

Feature support

The following list describes support of various components and functionality with Hive 2.3.6 - 2110:

  • Supports Hive-2.3.6 on Tez-0.9.1 For more information, see Tez 0.9.1-2104(MEP 6.3.4) Release Notes.

  • Does not support Hive on Spark, so you cannot use Spark as an execution engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.

  • Does not support HDFS encryption in Hive tables.

  • Does not support LLAP with Hive-2.3.6 because Apache Slider is not a supported HPE ecosystem component.

  • Starting from Hive 2.1, Hive must run the schematool command as an initialization step.

Changes in default security configuration

The following list describes changes in default security for Hive 2.3.6 - 2110:
  • None.

New in This Release

Hive 2.3.6 - 2110 introduces the following enhancements or HPE platform-specific behavior changes:

  • Improved Describe table operator in terms of fetching statistics of partitions. Starting from EEP 6.3.5, you can fetch the partition information using the describe command with formatted or extended statements.
    Configure the hive.describe.partitionedtable.ignore.stats property to change the behaviour of fetching statistics of partitions. It is set to the default value of false.
    <property>
    <name>hive.describe.partitionedtable.ignore.stats</name>
    <value>false</value>
    <description>Enables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description>
</property>
    <property>
    <name>hive.describe.partitionedtable.ignore.stats</name>
    <value>true</value>
    <description>Disables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description>
 </property

Fixes

This HPE release includes the following fixes on the base release:
GitHub Commit Number Data (YYYY-MM-DD) HPE Fix Number and Description
5cf874c69e 2021-09-14 MAPR-HIVE-1007: Permission denied to hbase temp files while running hcat jobs from other user
d40b435ebc 2021-08-17 MAPR-HIVE-977 : Downgrade jackson to v2.11.1 or to 2.11.3 to be consistent with core version

55471aefc0

2021-07-29 MAPR-HIVE-960 : CVE-2012-5783 vulnerability in commons-httpclient
96739404d6 2021-07-23 MAPR-HIVE-965 : Throws exception at INSERT statement with Avro table on MapReduce engine
864aa58888 2021-07-21 MAPR-HIVE-959 : Update derbyclient and derbynet to most feasible version
dbc1444cce 2021-07-21 MAPR-HIVE-963 : CVE-2020-13956,WS-2017-3734 vulnerabilities in httpclient
f1b65a14f4 2021-07-21 MAPR-HIVE-962 : WS-2019-0379: commons-codec vulnerability
de8eff00d4 2021-06-28 MAPR-HIVE-896 : CVE-2020-17521 vulnerability in Groovy
d6cc638433 2021-06-22 MAPR-HIVE-927 : NPE thrown from XmlUtil by Hive Client
cf0ce2e550 2021-06-13 MAPR-HIVE-950 : HiveVersionInfo.getShortVersion returns wrong version
4ba9150835 2021-06-08 MAPR-HIVE-949 : Make SchemaEvolution class behavior the same as in Orc-1.5.12
d6a0cd99ae 2021-06-05 MAPR-HIVE-894 : CVE-2020-13955 vulnerability in Calcite
This release also includes the following back-ported issues. For complete details, refer to the commit log for this project in GitHub.
GitHub Commit Number Data (YYYY-MM-DD) HPE Fix Number and Description
2e00b3ab7e 2021-09-15 HIVE-24965: Describe table partition stats fetch should be configurable
33c59e5bc1 2021-09-15 HIVE-22453: Describe table unnecessarily fetches partitions
59d9e532b1 2021-09-14 HIVE-23756 : Added more constraints to the package.jdo file
8f93305198 2021-09-12 HIVE-24177 : hive mapjoin throws udf class not found
d086e9f44e 2021-09-06 HIVE-25054: Upgrade `jodd-core` dependency to get rid of CVE-2018-21234 (Abhay Chennagiri, reviewed by Jesus Camacho Rodriguez)
f5c8373511 2021-08-18 HIVE-17659 : get_token thrift call fails for DBTokenStore in remote HMS mode (Vihang Karajgaonkar, reviewed by Aihua Xu)
2f963af3df 2021-08-18 HIVE-17824 : msck repair table should drop the missing partitions from metastore (Janaki Lahorani, reviewed by Peter Vary, Alexander Kolbasov and Vihang Karajgaonkar)
f219115a01 2021-08-18 HIVE-16143: Improve msck repair batching (Vihang Karajgaonkar, reviewed by Sahil Takiar & Aihua Xu)
b3197062e6 2021-07-28 HIVE-19228: Remove commons-httpclient 3.x usage (Janaki Lahorani reviewed by Aihua Xu)
9078072775 2021-07-23 HIVE-24324: Remove deprecated API usage from Avro (#1621)
b8d972a30a 2021-06-08 ORC-437: Make acid schema checks case insensitive
4a139e039f 2021-05-31 HIVE-21075 : Metastore: Drop partition performance downgrade with Postgres DB
6d952ad320 2021-05-31 HIVE-9447: Metastore: inefficient Oracle query for removing unused column descriptors when add/drop table/partition (Selina Zhang reviewed by Ashutosh Chauhan, Adam Szita)
79986d8ae1 2021-05-21 HIVE-21085: Materialized views registry starts non-external tez session (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
22a4f78a98 2021-05-21 HIVE-19691: Start SessionState in materialized views registry (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
46a18aa72b 2021-05-21 HIVE-17853: RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout (Chris Drome, reviewed by Mithun Radhakrishnan)
3b72d6b9c3 2021-05-21 HIVE-23534: NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message (Stamatis Zampetakis, reviewed by Jesus Camacho Rodriguez)
85b0e4b2dd 2021-05-21 HIVE-18494: Regression: from HIVE-18069, the metastore directsql is getting disabled (Jesus Camacho Rodriguez, reviewed by Gopal V)
d91f984cf6 2021-05-21 HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu) (addendum)
5e06c5859e 2021-05-21 HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu)
efa1df1148 2021-05-21 HIVE-15436: Enhancing metastore APIs to retrieve only materialized views (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
e642981bf1 2021-05-21 HIVE-6990 : Direct SQL fails when the explicit schema setting is different from the default on (Bing Li, Sergey Shelukhin via Ashutosh Chauhan)

Known Issues and Limitations

  • HIVE-19502 Unable to insert values into table stored by JdbcStorageHandler
  • HIVE-19286 NPE in MERGE operator on MR mode
Some SELECT queries can be converted to a single FETCH task minimizing latency. Currently, the query should be single sourced, not having any subquery, and should not have any aggregations or distincts (which incurs RS), lateral views or joins:
none : disable hive.fetch.task.conversion
minimal : SELECT star, filter on partition columns, LIMIT only
more :  SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)

Resolved Issues

  • None.