Hive 2.3.7-2009 (EEP 7.0.0) Release Notes
This section provides reference information, including new features, fixes, known issues, and limitations for Hive 2.3.7-2009 in EEP 7.0.0.
These release notes contain MapR-specific information only and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.
Hive Version | 2.3.7 |
Release Date | September 2020 |
MapR Version Interoperability | See EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/hive |
GitHub Release Tag |
2.3.7-mapr-2009 |
Maven Artifacts | See Maven Artifacts for MapR. |
Package Names | Navigate to https://package.ezmeral.hpe.com/releases/MEP/, and select your EEP and OS to view the list of package names. |
ODBC / JDBC Drivers | Hive 2.3.7 works with the following MapR Hive ODBC drivers:For additional driver information, see Connecting to HiveServer2. |
Feature Support
- Hive-2.3.7 works with Tez-0.9.2. For more information, see Tez 0.9.2-2009 (MEP 7.0.0) Release Notes.
- Hive on Spark is not supported; you cannot use Spark as an execution engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
- HDFS encryption is not supported in Hive tables.
- LLAP is not supported with Hive-2.3.7, as Apache Slider is not a supported ecosystem component.
- In Hive 2.1 and later, Hive must run the
schematool
command as an initialization step. - In EEPs 6.3.1 and 7.0.0, the default protocol
version for TLS (SSL) is
TLSv1.2
, but you can modify the protocol version. For more information, see Configure the TLS (SSL) Protocol Version in Hive.
Default Security Configuration Change
hive-site.xml
file now contains the following
property on secured clusters: Property | Default Value |
---|---|
hive.metastore.pre.event.listeners | org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener |
This property is added to hive-site.xml
during Hive updates (no
version change) and upgrades (with version change).
New Features
- None.
Known Issues
- Some SELECT queries can be converted to a single FETCH task to minimize latency.
Currently, a query should be single sourced, without a subquery, and should not have any
aggregations or distincts (which incurs RS), lateral views, or joins:
- none: disable
hive.fetch.task.conversion
- minimal: SELECT *, filter on partition columns, LIMIT only
- more: SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
- none: disable
- Use MySQL driver with MariaDB. In
hive-site.xml
, set the following properties:<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property>
- HIVE-19502 - Unable to insert values into table stored by JdbcStorageHandler.
- HIVE-19286 - NPE in MERGE operator in MR mode.
- HIVE-760 - [Hive-2.3] Could not start hive-metastore on Centos 8 MetaException(message:Version information not found in metastore)
- HIVE-789 - [Hive-Hbase integration] Unable to run queries
against hive-hbase tables. When using HBase with Hive-on-Tez, some queries, such as SELECT COUNT(*), fail because the new Tez classloader cannot load the MapR-specific classes. When this occurs, the system returns the following exception:
org.apache.hadoop.hbase.client.mapr.BaseTableMappingRules at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 39 more
Workaround
This workaround resolves the issue when running HBase 1.4.12 with Hive 2.3 on Tez 0.9 in an environment with Hadoop 2.7.4 and ZooKeeper 3.5.6.0 in ecosystem release 2009.
To resolve the issue, complete the following steps before you run HBase with Hive-on-Tez:
- Create the
/apps/tez/tez-0.9/hbase
directory:hadoop fs -mkdir /apps/tez/tez-0.9/hbase
- Add the HBase JAR files to the
/apps/tez/tez-0.9/hbase
directory:hadoop fs -put /opt/mapr/hbase/hbase-1.4.12/lib/* /apps/tez/tez-0.9/hbase/
- Update
/opt/mapr/tez/tez-0.9/conf/tez-site.xml
with the following property:<property> <name>tez.lib.uris</name> <value>${fs.defaultFS}/apps/tez/tez-0.9,${fs.defaultFS}/apps/tez/tez-0.9/lib,${fs.defaultFS}/apps/tez/tez-0.9/hbase/</value> </property>
- Create the
Fixes
This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.
Commit | Date (YYYY-MM-DD) | Comment |
---|---|---|
f50bb9f | 2020-09-04 | MAPR-HIVE-791 : Unnecessary WARNINGs during hive CLI start |
8755a63 | 2020-09-03 | MAPR-HIVE-816 : Update ZK version to 3.5.6.0-mapr-SNAPSHOT |
8060cf9 | 2020-08-28 | MAPR-HIVE-790 Added jmxagent jar into classpath |
6cbb679 | 2020-08-26 | MAPR-HIVE-741: Exculude org.mortbay.jetty from maven dependencies (Part 2) |
822aab1 | 2020-08-27 | MAPR-HIVE-814 : Replace mapr.release.version with mapr.core.version |
06ce5fe | 2020-08-24 | MAPR-HIVE-790 Added support for jmxagent at MAPR_JMXREMOTEHOST |
ab62da2 | 2020-08-17 | MAPR-HIVE-808 : Upgrade com.lmax.disruptor to version 3.4.2 |
1804443 | 2020-08-09 | MAPR-HIVE-805 : FAILURE! - in org.apache.hive.hcatalog.streaming.mutate.client.lock.TestLock |
818e7f2 | 2020-08-09 | MAPR-HIVE-804 : FAILURE! - in org.apache.hive.hcatalog.pig.TestAvroHCatLoader |
288030d | 2020-08-08 | MAPR-HIVE-802 : FAILURE! - in org.apache.hadoop.hive.hbase.TestPutResultWritable |
69bad5f | 2020-08-08 | MAPR-HIVE-801 : Fix java.lang.NoClassDefFoundError: org/eclipse/jetty/util/ClassVisibilityChecker |
2612916 | 2020-08-08 | MAPR-HIVE-800 : FAILURE! - in org.apache.hadoop.hive.ql.session.TestClearScratchDirUtil |
4dfb9b2 | 2020-08-08 | MAPR-HIVE-798 : FAILURE! - in org.apache.hadoop.hive.ql.exec.TestMapRDbJsonFetchByIdOperator |
8ff1d1a | 2020-08-07 | MAPR-HIVE-755 : Illegal reflective access of ShimLoader when using Java 11 |
37fad15 | 2020-08-01 | MAPR-HIVE-795 : Hive script /serviceCheckScript.sql has world writable permissions |
285f1cd | 2020-07-30 | MAPR-HIVE-762 : Select from MapRDB JSON table with >= AND NOT LIKE returns empty result |
06d1643 | 2020-07-30 | MAPR-HIVE-782 : Failed to start Hive services due to Password file not found: jmxremote.password [non-secure cluster] |
d87ce2b | 2020-07-16 | MAPR-HIVE-709 : Too verbose MapRDBSerDe |
86872aa | 2020-07-10 | CORE-458 : ERROR [HiveServer2-Background-Pool: Thread-44] zookeeper.ZKDataRetrieval: Most likely SessionExpirationException. Need to reset ZK and call myself again |
e1fa18a |
2020-07-08 |
MAPR-HIVE-783 : Exclude slf4j-log4j12 from mapr-encryption tool |
e65f666 | 2020-07-03 | MAPR-HIVE-777 : no WebHcat process on port 50111 after service started. MEP-7, CentOS 8.1 |
b0c82be | 2020-07-01 | MAPR-HIVE-778 : Update pig version to 0.17.0.0-mapr-SNAPSHOT |
cc098ac | 2020-06-30 | MAPR-HIVE-749 : Hive queries on tables with hdfs URI Scheme failing in compilation phase in 2.3 |
41d5dc8 | 2020-06-25 | MAPR-HIVE-772 : WebHCat service failed to start on CentOS 8.1 MEP-7.0.0 with IllegalArgumentException |
06f7625 | 2020-06-22 | ZOO-36: Upgrade Hive log jars to match the Zookeeper client |
0e92269 | 2020-06-19 | MAPR-HIVE-771 : Update datanucleus to latest version |
4cdb8f2 | 2020-06-19 | MAPR-HIVE-770 : Add logging for datanucleus maven plugin |
e4dab46 | 2020-06-20 | MAPR-HIVE-769 : Fix compilation error on centos 8: MockUriInfo is not abstract and does not override abstract method relativize(java.net.URI) |
3673257 | 2020-06-18 | MAPR-HIVE-768 : Fix compilation error on centos 8: hiveserver2_jsp is not abstract and does not override abstract method getDependants() |
0179594 | 2020-06-18 | MAPR-HIVE-767 : Change mapr-core dependency to 4 digit 6.2.0.0-mapr-SNAPSHOT |
1cb0bd0 | 2020-06-18 | MAPR-HIVE-764 : Fix testSetFullFileStatusFailInheritGroup |
7a8f79b | 2020-06-18 | MAPR-HIVE-761 : Remove jdo-api dependency and use datanucleus-jdo instead |
90b46dd | 2020-06-15 | MAPR-HIVE-753 : Can't find com.sun.common.util.logging.LogStrings bundle |
2991010 | 2020-06-11 | MAPR-HIVE-756 : ERROR Failed to execute goal org.apache.maven.plugins:maven-shade-plugin |
47596d0 | 2020-06-10 | MAPR-HIVE-752 : Set Java 11 for maven-compiler-plugin |
8cdf3af | 2020-06-10 | MAPR-HIVE-751 : Fix: cannot find symbol class Cleaner, location: package sun.misc |
6d85bb3 | 2020-06-10 | MAPR-HIVE-750 : Fix java.lang.ClassNotFoundException: javax.xml.bind.JAXBContext |
e49a866 | 2020-06-03 | MAPR-HIVE-747 : Remove PrintGCTimeStamps option since it is not supported in Java 11 |
4f1dd19 | 2020-05-22 | MAPR-HIVE-687 : Hiveserver2 Java JMX Server Insecure Configuration Remote Code Execution Vulnerability |
516117c | 2020-05-18 | MAPR-HIVE-737 : Failed Hive Druid Handler tests |
fe5b7af | 2020-05-13 | MAPR-HIVE-734 : Fix org.apache.hadoop.hive.llap.registry.impl.TestSlotZnode |
8d15ca8 | 2020-05-13 | MAPR-HIVE-733 : Update curator version to 4.2.0 |
6f2ab30 | 2020-05-12 | MAPR-HIVE-730 : Update MapR Maven repository URL to use https |
291b8d2 |
2020-05-12 |
MAPR-HIVE-729 : Fix setMetaStorePreEventListenerCustomNotRemoveProperty(org.apache.hive.conftool.ConfToolTest) |
3b56612 | 2020-05-11 | MAPR-HIVE-726 : Update Guava version to 28.2-jre |
154f6fe | 2020-05-07 | MAPR-HIVE-725 : Avatica's unshaded Jackson (2.6) conflicts with Jackson 2.7+ through hive-exec dependency |
c7d9a1e | 2020-05-05 | MAPR-HIVE-714 : ZK updates to v3.5.6 at MEP7.0.0 |
e6e0676 | 2020-05-04 | MAPR-HIVE-724 : Change Tez version in Hive-2.3.7 MEP-7.0.0 to 0.9.2-mapr-SNAPSHOT |
ec32584 | 2020-04-30 | MAPR-HIVE-723 : [Hive-Hbase integration] failed to select from Hive-Hbase table. HBase-1.4.12.0 [MEP-7.0.0] |
5cdf6b9 | 2020-04-29 | MAPR-HIVE-708 : Update thrift to ver 0.13.0 or most feasible newer version |
741ca67 | 2020-04-28 | MAPR-HIVE-705 : Update jasper-compiler to most feasible version with fix |
0b51959 | 2020-04-27 | MAPR-HIVE-704 : Update scala-compiler to ver 2.11.12/ 2.12.4 or most feasible newer version |
e62a0c9 | 2020-04-27 | MAPR-HIVE-701 : CVE-2017-18640: snakeyaml vulnerability at Hive |
ae366a8 | 2020-04-27 | MAPR-HIVE-700 : CVE-2014-0114: Vulnerability with commons-beanutils |
1f73d9d |
2020-04-27 |
MAPR-HIVE-703 : Update plexus-utils to ver 3.0.16 or most feasible upper version |
8c538fc | 2020-04-27 | MAPR-HIVE-706 : Update derby to ver 10.12.1.1 or most feasible newer/older version |
e3f7723 | 2020-04-26 | MAPR-HIVE-699 : Fix io.netty vulnerability |
9479d49 | 2020-05-18 | MAPR-HIVE-698 : Fix org.codehaus.jackson vulnerability |
7312b11 | 2020-04-14 | MAPR-HIVE-589 Hive jobs fail with "Unable to rename output" |
bd9b9b1 | 2020-04-18 | MAPR-HIVE-712 : Move Hive-2.3.6 MEP-7.x to two digit versioning in package name |
8491cb9 | 2020-03-23 | MAPR-HIVE-690 : Enable SSL protocol version TLSv1.2 as default |
e8ee9eb | 2020-03-16 | MAPR-HIVE-561: Add hive configure.sh logging |
8b0a614 | 2020-03-16 | MAPR-HIVE-685: Enable AuthorizationPreEventListener in hive-site.xml for Security-By-Default |
0176bad | 2020-02-24 | MAPR-HIVE-683 : Add JUnit test for StorageBasedAuthorizationProvider and update of hive.metastore.warehouse.dir |
9d814cf | 2020-02-20 | MAPR-HIVE-681 : StorageBasedAuthorizationProvider does not allow create databases after changing hive.metastore.warehouse.dir |
6724858 | 2020-02-19 | MAPR-HIVE-674: Failed to launch Hive WebHCat job zookeeper-3.4.11-mapr-1808.jar does not exist. |
04b6072 | 2020-01-28 | MAPR-HIVE-660 : Create root scratch dir with 733 instead of 777 perms [HIVE-8143] (Part II) |
a0bfa31 | 2020-01-14 | MAPR-HIVE-615: Add Junit tests for MapR Db Json optimization when projection push down is used |
d53a0ed | 2020-01-13 | MAPR-HIVE-660 : Create root scratch dir with 733 instead of 777 perms [HIVE-8143] |
62e55cf | 2020-01-15 | MAPR-HIVE-572 : Evaluate impact of Protobuf upgrade for 6.2 to Protobuf 3.5 or higher on Hive |
94736a6 | 2020-01-08 | MAPR-HIVE-656: Hive on Tez : Job of merging small files will be submitted into another queue (default queue) |
741e989 | 2019-12-06 | MAPR-HIVE-652: SQL Standard Based Authorization does not allow to create a function over HiveServer2 even to Admin user |
This release also includes the following back-ported issues. For complete details, refer to the commit log for this project in GitHub.
Commit | Date (YYYY-MM-DD) | Comment |
---|---|---|
cc603a0 | 2020-08-29 | HIVE-10296: Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before applying cast (Karthik Manamcheri, reviewed by Sergey Shelukhin |
21d923c | 2020-08-17 | HIVE-22278: Upgrade log4j to 2.12.1 (David Lavati via Zoltan Haindrich) |
437a8c5 | 2020-08-17 | HIVE-18384: ConcurrentModificationException in log4j2.x library (Prasanth Jayachandran reviewed by Sergey Shelukhin) |
5a3db2b | 2020-08-08 | HIVE-21777: Maven jar goal is producing warning due to missing dependency (Aron Hamvas via Peter Vary) |
d830346 | 2020-08-08 | HIVE-21836: Update apache directory server version to 1.5.7 (Zoltan Haindrich reviewed by Laszlo Bodor) |
f605ca8 | 2020-07-13 | HIVE-18699: Check for duplicate partitions in HiveMetastore.exchange_partitions (Marta Kuczora, reviewed by Adam Szita, Peter Vary) |
51de45d | 2020-06-04 | HIVE-19700: Workaround for JLine issue with UnsupportedTerminal (Naveen Gangam, reviewed by Yongzhi Chen) |
549c374 | 2020-06-11 | HIVE-17879 : Upgrade Datanucleus Maven Plugin |
f68fea9 | 2020-06-11 | HIVE-23363 : Upgrade DataNucleus dependency to 5.2 |
1aab82c | 2020-03-29 | HIVE-21512: Upgrade jms-api to 2.0.2 (Zoltan Haindrich reviewed by Peter Vary) |
f6606ce | 2020-08-08 | HIVE-22066: Upgrade Apache parent POM to version 21 (David Mollitor, reviewed by Ashutosh Chauhan) |
f535560 | 2020-08-08 | HIVE-16281: Upgrade master branch to JDK8 (Aihua Xu, reviewed by Thejas M Nair, Sergio Peña, Sean Busbey) |
1ceac6f | 2020-08-08 | HIVE-22097: Incompatible java.util.ArrayList for java 11 (Attila Magyar via Laszlo Bodor, Prasanth Jayachandran) |
f5e8e4b | 2020-08-08 | HIVE-19383 : Add ArrayList$SubList kryo serializer (Ashutosh Chauhan via Prasanth J) |
4ffaf82 | 2020-08-08 | HIVE-21584: Java 11 preparation: system class loader is not URLClassLoader (Zoltan Matyus, reviewed by Adam Szita, Zoltan Haindrich) |
84774e2 | 2020-08-08 | HIVE-23209: ptest2 compilation failure after HIVE-21603 - upgrade mockito-core in testutils/ptest2 |
2f25d15 | 2020-08-08 | HIVE-21603 : Java 11 preparation: update powermock version (Panos G via Ashutosh Chauhan) |
a1eebe8 | 2020-06-15 | HIVE-21670: Replacing mockito-all with mockito-core dependency (Ivan Suller via Zoltan Haindrich) |
14c4d1a | 2020-06-11 | HIVE-19433: HiveJoinPushTransitivePredicatesRule hangs (Vineet Garg,reviewed by Jesus Camacho Rodriguez) |
6e53218 | 2020-06-09 | HIVE-19799: remove jasper dependency (Prasanth Jayachandran reviewed by Thejas M Nair) |
741e268 | 2020-06-03 | HIVE-16911: Upgrade groovy version to 2.4.11 (Aihua Xu, reviewed by Yongzhi Chen) |
e266fea | 2020-06-03 | HIVE-16371: Add bitmap selection strategy for druid storage handler (Slim Bouguerra, reviewed by Jesus Camacho Rodriguez) |
468ba91 | 2020-06-03 | HIVE-16474: Upgrade Druid version to 0.10 (Nishant Bangarwa, reviewed by Jesus Camacho Rodriguez) |
cacff9a | 2020-06-03 | HIVE-14069 : update curator version to 2.12.0 (Jason Dere via Ashutosh Chauhan) |
c3ecf30 | 2020-06-03 | HIVE-15708 : Upgrade calcite version to 1.12 (Remus Rusanu via Ashutosh Chauhan) |
3719911 | 2020-05-18 | HIVE-15393 : Update Guava version (Slim Bouguerra via Ashutosh Chauhan) |
46ce644 | 2020-05-18 | HIVE-18586: Upgrade Derby to 10.14.1.0 (Janaki Lahorani, reviewed by Aihua Xu) |
53d3977 | 2020-05-18 | HIVE-23073 : Shade netty and upgrade to netty 4.1.48.Final (Laszlo Bodor via Ashutosh Chauhan) |
414a5d5 | 2020-05-18 | HIVE-18436: Upgrade to Spark 2.3.0 (Sahil Takiar, reviewed by Rui Li) |
a1fa8ab | 2020-05-11 | HIVE-18433: Upgrade version of com.fasterxml.jackson (Janaki Lahorani, reviewed by Aihua Xu) |
76e2357 | 2020-04-23 | HIVE-23086 Two tests fail on branch-2.3 (gates, reviewed by jcamacho) |
302cd39 | 2020-04-23 | HIVE-22249: Support Parquet through HCatalog (Jay Green-Stevens via Peter Vary) |
839a94a | 2020-04-23 | HIVE-21508: ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer (Ana Jalba, via Peter Vary) |
71ea54a | 2020-03-03 | HIVE-16839 : Unbalanced calls to openTransaction/commitTransaction when alter the same partition concurrently (Guang Yang, reviewed by Karthik Manamcheri and Vihang Karajgaonkar) |
9bc66b7 | 2020-04-14 | HIVE-19199: ACID: DbTxnManager heartbeat-service needs static sync init (Gopal V, reviewed by Eugene Koifman) |
a7a0efa | 2020-02-05 | HIVE-7089 : StorageBasedAuthorizationProvider fails to allow non-admin users to create databases in writable directories |
ce28628 | 2020-01-30 | HIVE-11741: Add a new hook to run before query parse/compile |
8c8322c | 2019-12-17 | HIVE-22151 : Turn off hybrid grace hash join by default (Ashutosh Chauhan via Vineet Garg) |