Hive 2.3.7-2009 (EEP 7.0.0) Release Notes

This section provides reference information, including new features, fixes, known issues, and limitations for Hive 2.3.7-2009 in EEP 7.0.0.

Below are release notes for the Hive component included in the MapR Converged Data Platform. You may also be interested in the Apache Hive homepage and the following Apache Hive release notes:

These release notes contain MapR-specific information only and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Hive Version 2.3.7
Release Date September 2020
MapR Version Interoperability See EEP Components and OS Support.
Source on GitHub https://github.com/mapr/hive
GitHub Release Tag

2.3.7-mapr-2009

Maven Artifacts See Maven Artifacts for MapR.
Package Names Navigate to https://package.ezmeral.hpe.com/releases/MEP/, and select your EEP and OS to view the list of package names.
ODBC / JDBC Drivers Hive 2.3.7 works with the following MapR Hive ODBC drivers:For additional driver information, see Connecting to HiveServer2.

Feature Support

  • Hive-2.3.7 works with Tez-0.9.2. For more information, see Tez 0.9.2-2009 (MEP 7.0.0) Release Notes.
  • Hive on Spark is not supported; you cannot use Spark as an execution engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
  • HDFS encryption is not supported in Hive tables.
  • LLAP is not supported with Hive-2.3.7, as Apache Slider is not a supported ecosystem component.
  • In Hive 2.1 and later, Hive must run the schematool command as an initialization step.
  • In EEPs 6.3.1 and 7.0.0, the default protocol version for TLS (SSL) is TLSv1.2, but you can modify the protocol version. For more information, see Configure the TLS (SSL) Protocol Version in Hive.

Default Security Configuration Change

By default, the hive-site.xml file now contains the following property on secured clusters:
Property Default Value
hive.metastore.pre.event.listeners org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener

This property is added to hive-site.xml during Hive updates (no version change) and upgrades (with version change).

New Features

  • None.

Known Issues

  • Some SELECT queries can be converted to a single FETCH task to minimize latency. Currently, a query should be single sourced, without a subquery, and should not have any aggregations or distincts (which incurs RS), lateral views, or joins:
    • none: disable hive.fetch.task.conversion
    • minimal: SELECT *, filter on partition columns, LIMIT only
    • more: SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
  • Use MySQL driver with MariaDB. In hive-site.xml, set the following properties:
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>
    
  • HIVE-19502 - Unable to insert values into table stored by JdbcStorageHandler.
  • HIVE-19286 - NPE in MERGE operator in MR mode.
  • HIVE-760 - [Hive-2.3] Could not start hive-metastore on Centos 8 MetaException(message:Version information not found in metastore)
  • HIVE-789 - [Hive-Hbase integration] Unable to run queries against hive-hbase tables.
    When using HBase with Hive-on-Tez, some queries, such as SELECT COUNT(*), fail because the new Tez classloader cannot load the MapR-specific classes. When this occurs, the system returns the following exception:
    org.apache.hadoop.hbase.client.mapr.BaseTableMappingRules
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 39 more
    

    Workaround

    This workaround resolves the issue when running HBase 1.4.12 with Hive 2.3 on Tez 0.9 in an environment with Hadoop 2.7.4 and ZooKeeper 3.5.6.0 in ecosystem release 2009.

    To resolve the issue, complete the following steps before you run HBase with Hive-on-Tez:

    1. Create the /apps/tez/tez-0.9/hbase directory:
      hadoop fs -mkdir /apps/tez/tez-0.9/hbase
    2. Add the HBase JAR files to the /apps/tez/tez-0.9/hbase directory:
      hadoop fs -put /opt/mapr/hbase/hbase-1.4.12/lib/* /apps/tez/tez-0.9/hbase/
    3. Update /opt/mapr/tez/tez-0.9/conf/tez-site.xml with the following property:
      <property>
          <name>tez.lib.uris</name>    
          <value>${fs.defaultFS}/apps/tez/tez-0.9,${fs.defaultFS}/apps/tez/tez-0.9/lib,${fs.defaultFS}/apps/tez/tez-0.9/hbase/</value>
      </property>
      

Fixes

This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

Commit Date (YYYY-MM-DD) Comment
f50bb9f 2020-09-04 MAPR-HIVE-791 : Unnecessary WARNINGs during hive CLI start
8755a63 2020-09-03 MAPR-HIVE-816 : Update ZK version to 3.5.6.0-mapr-SNAPSHOT
8060cf9 2020-08-28 MAPR-HIVE-790 Added jmxagent jar into classpath
6cbb679 2020-08-26 MAPR-HIVE-741: Exculude org.mortbay.jetty from maven dependencies (Part 2)
822aab1 2020-08-27 MAPR-HIVE-814 : Replace mapr.release.version with mapr.core.version
06ce5fe 2020-08-24 MAPR-HIVE-790 Added support for jmxagent at MAPR_JMXREMOTEHOST
ab62da2 2020-08-17 MAPR-HIVE-808 : Upgrade com.lmax.disruptor to version 3.4.2
1804443 2020-08-09 MAPR-HIVE-805 : FAILURE! - in org.apache.hive.hcatalog.streaming.mutate.client.lock.TestLock
818e7f2 2020-08-09 MAPR-HIVE-804 : FAILURE! - in org.apache.hive.hcatalog.pig.TestAvroHCatLoader
288030d 2020-08-08 MAPR-HIVE-802 : FAILURE! - in org.apache.hadoop.hive.hbase.TestPutResultWritable
69bad5f 2020-08-08 MAPR-HIVE-801 : Fix java.lang.NoClassDefFoundError: org/eclipse/jetty/util/ClassVisibilityChecker
2612916 2020-08-08 MAPR-HIVE-800 : FAILURE! - in org.apache.hadoop.hive.ql.session.TestClearScratchDirUtil
4dfb9b2 2020-08-08 MAPR-HIVE-798 : FAILURE! - in org.apache.hadoop.hive.ql.exec.TestMapRDbJsonFetchByIdOperator
8ff1d1a 2020-08-07 MAPR-HIVE-755 : Illegal reflective access of ShimLoader when using Java 11
37fad15 2020-08-01 MAPR-HIVE-795 : Hive script /serviceCheckScript.sql has world writable permissions
285f1cd 2020-07-30 MAPR-HIVE-762 : Select from MapRDB JSON table with >= AND NOT LIKE returns empty result
06d1643 2020-07-30 MAPR-HIVE-782 : Failed to start Hive services due to Password file not found: jmxremote.password [non-secure cluster]
d87ce2b 2020-07-16 MAPR-HIVE-709 : Too verbose MapRDBSerDe
86872aa 2020-07-10 CORE-458 : ERROR [HiveServer2-Background-Pool: Thread-44] zookeeper.ZKDataRetrieval: Most likely SessionExpirationException. Need to reset ZK and call myself again
e1fa18a

2020-07-08

MAPR-HIVE-783 : Exclude slf4j-log4j12 from mapr-encryption tool
e65f666 2020-07-03 MAPR-HIVE-777 : no WebHcat process on port 50111 after service started. MEP-7, CentOS 8.1
b0c82be 2020-07-01 MAPR-HIVE-778 : Update pig version to 0.17.0.0-mapr-SNAPSHOT
cc098ac 2020-06-30 MAPR-HIVE-749 : Hive queries on tables with hdfs URI Scheme failing in compilation phase in 2.3
41d5dc8 2020-06-25 MAPR-HIVE-772 : WebHCat service failed to start on CentOS 8.1 MEP-7.0.0 with IllegalArgumentException
06f7625 2020-06-22 ZOO-36: Upgrade Hive log jars to match the Zookeeper client
0e92269 2020-06-19 MAPR-HIVE-771 : Update datanucleus to latest version
4cdb8f2 2020-06-19 MAPR-HIVE-770 : Add logging for datanucleus maven plugin
e4dab46 2020-06-20 MAPR-HIVE-769 : Fix compilation error on centos 8: MockUriInfo is not abstract and does not override abstract method relativize(java.net.URI)
3673257 2020-06-18 MAPR-HIVE-768 : Fix compilation error on centos 8: hiveserver2_jsp is not abstract and does not override abstract method getDependants()
0179594 2020-06-18 MAPR-HIVE-767 : Change mapr-core dependency to 4 digit 6.2.0.0-mapr-SNAPSHOT
1cb0bd0 2020-06-18 MAPR-HIVE-764 : Fix testSetFullFileStatusFailInheritGroup
7a8f79b 2020-06-18 MAPR-HIVE-761 : Remove jdo-api dependency and use datanucleus-jdo instead
90b46dd 2020-06-15 MAPR-HIVE-753 : Can't find com.sun.common.util.logging.LogStrings bundle
2991010 2020-06-11 MAPR-HIVE-756 : ERROR Failed to execute goal org.apache.maven.plugins:maven-shade-plugin
47596d0 2020-06-10 MAPR-HIVE-752 : Set Java 11 for maven-compiler-plugin
8cdf3af 2020-06-10 MAPR-HIVE-751 : Fix: cannot find symbol class Cleaner, location: package sun.misc
6d85bb3 2020-06-10 MAPR-HIVE-750 : Fix java.lang.ClassNotFoundException: javax.xml.bind.JAXBContext
e49a866 2020-06-03 MAPR-HIVE-747 : Remove PrintGCTimeStamps option since it is not supported in Java 11
4f1dd19 2020-05-22 MAPR-HIVE-687 : Hiveserver2 Java JMX Server Insecure Configuration Remote Code Execution Vulnerability
516117c 2020-05-18 MAPR-HIVE-737 : Failed Hive Druid Handler tests
fe5b7af 2020-05-13 MAPR-HIVE-734 : Fix org.apache.hadoop.hive.llap.registry.impl.TestSlotZnode
8d15ca8 2020-05-13 MAPR-HIVE-733 : Update curator version to 4.2.0
6f2ab30 2020-05-12 MAPR-HIVE-730 : Update MapR Maven repository URL to use https
291b8d2

2020-05-12

MAPR-HIVE-729 : Fix setMetaStorePreEventListenerCustomNotRemoveProperty(org.apache.hive.conftool.ConfToolTest)
3b56612 2020-05-11 MAPR-HIVE-726 : Update Guava version to 28.2-jre
154f6fe 2020-05-07 MAPR-HIVE-725 : Avatica's unshaded Jackson (2.6) conflicts with Jackson 2.7+ through hive-exec dependency
c7d9a1e 2020-05-05 MAPR-HIVE-714 : ZK updates to v3.5.6 at MEP7.0.0
e6e0676 2020-05-04 MAPR-HIVE-724 : Change Tez version in Hive-2.3.7 MEP-7.0.0 to 0.9.2-mapr-SNAPSHOT
ec32584 2020-04-30 MAPR-HIVE-723 : [Hive-Hbase integration] failed to select from Hive-Hbase table. HBase-1.4.12.0 [MEP-7.0.0]
5cdf6b9 2020-04-29 MAPR-HIVE-708 : Update thrift to ver 0.13.0 or most feasible newer version
741ca67 2020-04-28 MAPR-HIVE-705 : Update jasper-compiler to most feasible version with fix
0b51959 2020-04-27 MAPR-HIVE-704 : Update scala-compiler to ver 2.11.12/ 2.12.4 or most feasible newer version
e62a0c9 2020-04-27 MAPR-HIVE-701 : CVE-2017-18640: snakeyaml vulnerability at Hive
ae366a8 2020-04-27 MAPR-HIVE-700 : CVE-2014-0114: Vulnerability with commons-beanutils
1f73d9d

2020-04-27

MAPR-HIVE-703 : Update plexus-utils to ver 3.0.16 or most feasible upper version
8c538fc 2020-04-27 MAPR-HIVE-706 : Update derby to ver 10.12.1.1 or most feasible newer/older version
e3f7723 2020-04-26 MAPR-HIVE-699 : Fix io.netty vulnerability
9479d49 2020-05-18 MAPR-HIVE-698 : Fix org.codehaus.jackson vulnerability
7312b11 2020-04-14 MAPR-HIVE-589 Hive jobs fail with "Unable to rename output"
bd9b9b1 2020-04-18 MAPR-HIVE-712 : Move Hive-2.3.6 MEP-7.x to two digit versioning in package name
8491cb9 2020-03-23 MAPR-HIVE-690 : Enable SSL protocol version TLSv1.2 as default
e8ee9eb 2020-03-16 MAPR-HIVE-561: Add hive configure.sh logging
8b0a614 2020-03-16 MAPR-HIVE-685: Enable AuthorizationPreEventListener in hive-site.xml for Security-By-Default
0176bad 2020-02-24 MAPR-HIVE-683 : Add JUnit test for StorageBasedAuthorizationProvider and update of hive.metastore.warehouse.dir
9d814cf 2020-02-20 MAPR-HIVE-681 : StorageBasedAuthorizationProvider does not allow create databases after changing hive.metastore.warehouse.dir
6724858 2020-02-19 MAPR-HIVE-674: Failed to launch Hive WebHCat job zookeeper-3.4.11-mapr-1808.jar does not exist.
04b6072 2020-01-28 MAPR-HIVE-660 : Create root scratch dir with 733 instead of 777 perms [HIVE-8143] (Part II)
a0bfa31 2020-01-14 MAPR-HIVE-615: Add Junit tests for MapR Db Json optimization when projection push down is used
d53a0ed 2020-01-13 MAPR-HIVE-660 : Create root scratch dir with 733 instead of 777 perms [HIVE-8143]
62e55cf 2020-01-15 MAPR-HIVE-572 : Evaluate impact of Protobuf upgrade for 6.2 to Protobuf 3.5 or higher on Hive
94736a6 2020-01-08 MAPR-HIVE-656: Hive on Tez : Job of merging small files will be submitted into another queue (default queue)
741e989 2019-12-06 MAPR-HIVE-652: SQL Standard Based Authorization does not allow to create a function over HiveServer2 even to Admin user

This release also includes the following back-ported issues. For complete details, refer to the commit log for this project in GitHub.

Commit Date (YYYY-MM-DD) Comment
cc603a0 2020-08-29 HIVE-10296: Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before applying cast (Karthik Manamcheri, reviewed by Sergey Shelukhin
21d923c 2020-08-17 HIVE-22278: Upgrade log4j to 2.12.1 (David Lavati via Zoltan Haindrich)
437a8c5 2020-08-17 HIVE-18384: ConcurrentModificationException in log4j2.x library (Prasanth Jayachandran reviewed by Sergey Shelukhin)
5a3db2b 2020-08-08 HIVE-21777: Maven jar goal is producing warning due to missing dependency (Aron Hamvas via Peter Vary)
d830346 2020-08-08 HIVE-21836: Update apache directory server version to 1.5.7 (Zoltan Haindrich reviewed by Laszlo Bodor)
f605ca8 2020-07-13 HIVE-18699: Check for duplicate partitions in HiveMetastore.exchange_partitions (Marta Kuczora, reviewed by Adam Szita, Peter Vary)
51de45d 2020-06-04 HIVE-19700: Workaround for JLine issue with UnsupportedTerminal (Naveen Gangam, reviewed by Yongzhi Chen)
549c374 2020-06-11 HIVE-17879 : Upgrade Datanucleus Maven Plugin
f68fea9 2020-06-11 HIVE-23363 : Upgrade DataNucleus dependency to 5.2
1aab82c 2020-03-29 HIVE-21512: Upgrade jms-api to 2.0.2 (Zoltan Haindrich reviewed by Peter Vary)
f6606ce 2020-08-08 HIVE-22066: Upgrade Apache parent POM to version 21 (David Mollitor, reviewed by Ashutosh Chauhan)
f535560 2020-08-08 HIVE-16281: Upgrade master branch to JDK8 (Aihua Xu, reviewed by Thejas M Nair, Sergio Peña, Sean Busbey)
1ceac6f 2020-08-08 HIVE-22097: Incompatible java.util.ArrayList for java 11 (Attila Magyar via Laszlo Bodor, Prasanth Jayachandran)
f5e8e4b 2020-08-08 HIVE-19383 : Add ArrayList$SubList kryo serializer (Ashutosh Chauhan via Prasanth J)
4ffaf82 2020-08-08 HIVE-21584: Java 11 preparation: system class loader is not URLClassLoader (Zoltan Matyus, reviewed by Adam Szita, Zoltan Haindrich)
84774e2 2020-08-08 HIVE-23209: ptest2 compilation failure after HIVE-21603 - upgrade mockito-core in testutils/ptest2
2f25d15 2020-08-08 HIVE-21603 : Java 11 preparation: update powermock version (Panos G via Ashutosh Chauhan)
a1eebe8 2020-06-15 HIVE-21670: Replacing mockito-all with mockito-core dependency (Ivan Suller via Zoltan Haindrich)
14c4d1a 2020-06-11 HIVE-19433: HiveJoinPushTransitivePredicatesRule hangs (Vineet Garg,reviewed by Jesus Camacho Rodriguez)
6e53218 2020-06-09 HIVE-19799: remove jasper dependency (Prasanth Jayachandran reviewed by Thejas M Nair)
741e268 2020-06-03 HIVE-16911: Upgrade groovy version to 2.4.11 (Aihua Xu, reviewed by Yongzhi Chen)
e266fea 2020-06-03 HIVE-16371: Add bitmap selection strategy for druid storage handler (Slim Bouguerra, reviewed by Jesus Camacho Rodriguez)
468ba91 2020-06-03 HIVE-16474: Upgrade Druid version to 0.10 (Nishant Bangarwa, reviewed by Jesus Camacho Rodriguez)
cacff9a 2020-06-03 HIVE-14069 : update curator version to 2.12.0 (Jason Dere via Ashutosh Chauhan)
c3ecf30 2020-06-03 HIVE-15708 : Upgrade calcite version to 1.12 (Remus Rusanu via Ashutosh Chauhan)
3719911 2020-05-18 HIVE-15393 : Update Guava version (Slim Bouguerra via Ashutosh Chauhan)
46ce644 2020-05-18 HIVE-18586: Upgrade Derby to 10.14.1.0 (Janaki Lahorani, reviewed by Aihua Xu)
53d3977 2020-05-18 HIVE-23073 : Shade netty and upgrade to netty 4.1.48.Final (Laszlo Bodor via Ashutosh Chauhan)
414a5d5 2020-05-18 HIVE-18436: Upgrade to Spark 2.3.0 (Sahil Takiar, reviewed by Rui Li)
a1fa8ab 2020-05-11 HIVE-18433: Upgrade version of com.fasterxml.jackson (Janaki Lahorani, reviewed by Aihua Xu)
76e2357 2020-04-23 HIVE-23086 Two tests fail on branch-2.3 (gates, reviewed by jcamacho)
302cd39 2020-04-23 HIVE-22249: Support Parquet through HCatalog (Jay Green-Stevens via Peter Vary)
839a94a 2020-04-23 HIVE-21508: ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer (Ana Jalba, via Peter Vary)
71ea54a 2020-03-03 HIVE-16839 : Unbalanced calls to openTransaction/commitTransaction when alter the same partition concurrently (Guang Yang, reviewed by Karthik Manamcheri and Vihang Karajgaonkar)
9bc66b7 2020-04-14 HIVE-19199: ACID: DbTxnManager heartbeat-service needs static sync init (Gopal V, reviewed by Eugene Koifman)
a7a0efa 2020-02-05 HIVE-7089 : StorageBasedAuthorizationProvider fails to allow non-admin users to create databases in writable directories
ce28628 2020-01-30 HIVE-11741: Add a new hook to run before query parse/compile
8c8322c 2019-12-17 HIVE-22151 : Turn off hybrid grace hash join by default (Ashutosh Chauhan via Vineet Garg)