Drill 1.11.0-1710 Release Notes
This section provides reference information, including new features, fixes, known issues, and limitations for Drill 1.11.0-1710.
The following release notes apply to the 1.11.0 version of the Apache Drill component included in the MapR Converged Data Platform.
| Version | 1.11.0 |
| Release Date | November 2017 |
| MapR Version Interoperability | See EEP Components and OS Support, Interoperability Matrix, and Drill Support Matrix. |
| Package Names | Package Names for Ezmeral Ecosystem Packs (EEPs) |
New in this Release
- Secondary index support on HPE Ezmeral Data Fabric Database for JSON tables.
- MapR default security configuration that secures the cluster and ecosystem components using MapR security (mapr tickets) to provide authentication, authorization, and encryption.
- Client to Drillbit Encryption using SSL/TLS.
- Spill to disk for the Hash Aggregate operator (in addition to the Sort operator).
- Throttle-based resource assignment (queuing feature).
- Drill integration with Zeppelin.
In addition to the features mentioned above, Drill 1.11 also includes the following
features and improvements:
| MapR Tracking Numbers | Features |
| MD-2490 | Addition of an internal options table. |
| MD-1813 | Multiplexing session over a connection. |
| MD-1655 | Provide an option to persist Query Profiles in a Drillbit server's memory |
| MD-1245 | MapR-SASL Plugin for ODBC/Drill (Authentication & Encryption) |
| MD-536 | SASL support between Drillclient and Drillbit |
| MapR Tracking Numbers | Improvements |
| MD-2962 | Add a UDF to decode encoded fieldpath |
| MD-2871 | Restricted scan (for non-covering) gets over-parallelized due to scan range size |
| MD-2799 | DRILL-5811 Large number of "Failure finding Drillbit" messages when using file system |
| MD-2789 | Fix correct batch size for merge back during non-covering query. |
| MD-2693 | Drill should use `com.mapr.db.index.IndexDesc.getNullOrdering()` to decide if an index can be used for order by |
| MD-2656 | Drill needs to return more specific error for DB |
| MD-2647 | Enable UTF-8 support in query string by default |
| MD-2506 | Standardize on JDBC driver that gets shipped with Drill and sqlline |
| MD-2422 | Drill performance - doing less than 40 queries/sec |
| MD-2416 | RestrictedJsonTableGroupScan should support projection pushdown |
| MD-2403 | Drillbit fails to start when only keystore path is provided without keystore password. |
| MD-2339 | Additional log events for debugging index selection |
| MD-2319 | Query Containing Wildcard in String Matching Filter Runs 4x Slower Than Impala |
| MD-2301 | Support Impersonation without authentication for REST API |
| MD-2278 | Include flags controlling secure Drill ZK ACLs in the secure installer |
| MD-2225 | Review the names of index planning config options |
| MD-2196 | DRILL-4264: Allow field names to include dots |
| MD-2192 | Provide log tracing of a query submitted by an OJAI client for user debugging |
| MD-2074 | Memory Fragmentation Fundamentals Completion |
| MD-2067 | Handle the case of order-by with no filter conditions |
| MD-2002 | Operator Specific Handle for Empty Batches |
| MD-1987 | Drill should read HPE Ezmeral Data Fabric Database tablets in parallel |
| MD-1962 | Create configuration file for distribution specific configuration |
| MD-1836 | Leverage hash index when doing secondary index planning |
| MD-1823 | Add logging information for listing candidate indexes for a given query |
| MD-1815 | Leverage HPE Ezmeral Data Fabric Database batch gets for better performance. |
| MD-1789 | Cast function support pull request and code review process |
| MD-1786 | Push down limit on HPE Ezmeral Data Fabric Database table scan |
| MD-1752 | Throttling-based per-query memory assignment |
| MD-1673 | Add Statistics support for HPE Ezmeral Data Fabric Database |
| MD-1665 | Provide query level option to pick specific indexes |
| MD-1643 | Disable table caching by default |
| MD-1498 | Plan queries against HPE Ezmeral Data Fabric Database JSON faster |
| MD-1318 | Improve Low Latency Performance of Drill |
| MD-1209 | Configuring ZooKeeper Access Controls for Drill on a Secure Cluster |
Fixes
None
Known Issues and Limitations
This release of Drill includes the following known issues:
| MapR Tracking Numbers | Known Issues |
| MD-3171 | Drillbit down due to fs client assertion in Java_com_mapr_fs_jni_MapRTableTools_GetScanner |
| MD-3162 | DRILL-5946: Document maprdb format plugin options |
| MD-3160 | Hit ExpressionParsingException: Expression has syntax error! line 1:0:no viable alternative at input 'TIME' |
| MD-3155 | Simba JDBC driver dependency issues with third party applications |
| MD-3131 | Memory leak by HashPartitionSender |
| MD-3108 | planner.memory.max_query_memory_per_node should be default to 4GB |
| MD-3089 | Some queries with join on two or more tables only pick non-covering index plans |
| MD-3071 | Queries using hash index plans (intermittently) fail with SYSTEM ERROR: RpcException: Data not accepted downstream. |
| MD-3047 | Apache Drill JDBC driver from mapr-drill package not works |
| MD-3041 | Identify closed jdbc connection |
| MD-3040 | need a better error message - RpcException: Drillbit (demo-179.lab) does not require auth, but auth is enabled. |
| MD-3037 | Performance issues with queries against HPE Ezmeral Data Fabric Database JSON tables (full table scans) |
| MD-3031 | Duplicate columns projected from HPE Ezmeral Data Fabric Database Scan for large IN condition |
| MD-2979 | Memory leak by ParquetRowGroupScan |
| MD-2950 | Full outer join on two empty hive tables throws IllegalArgumentException: Full outer join not currently supported |
| MD-2948 | SI: Non-covering queries hang after drillbit restart |
| MD-2848 | IteratorValidatorBatchIterator is throwing exception when assertions are enabled. |
| MD-2838 | After upgrading from MapR Drill 1.10.0 to 1.11.0, drillbit hangs on startup |
| MD-2837 | Queries on HPE Ezmeral Data Fabric Database 5.x tables hang |
| MD-2809 | Scalar replacement failures |
| MD-2791 | Cancelled query using index plan logs several errors |
| MD-2790 | Queries fail with various CONNECTION ERRORs |
| MD-2695 | Test Framework - Test fails with DATA_READ_ERROR : Failed to load table into HBase |
| MD-2589 | Drillbit startup has errors: "Can't load log handler java.util.logging.FileHandler" |
| MD-2544 | drillbit jvm dumps core when querying a table with SI |
| MD-2528 | Parquet int_64 causes UnsupportedOperationException in Drill |
| MD-2515 | Evaluate Codegen Compilers for performance overhead |
| MD-2513 | Querying last column in CSV causes IndexOutOfBoundsException |
| MD-2455 | Query with INTERVAL in predicate does not return any rows |
| MD-2420 | External sort with higher memory takes longer to finish vs. smaller memory |
| MD-2419 | Query fails with OOM with queuing enabled and exchanges disabled |
| MD-2415 | Resource leak during Hash Agg spill to disk longevity run |
| MD-2400 | Random Jenkins unit test failure due to ".crc" files in spill directory |
| MD-2397 | Zeppelin cannot connect to Drill through the JDBC driver on a secure MapR cluster when Zeppelin has Kerberos authentication enabled. |
| MD-2285 | Filter with interval was not fully pushed down |
| MD-2218 | Direct memory fragmentation cause OOM while running OJAI queries |
| MD-2141 | Predicates on indexed field, joined by OR, do not return the correct results for time and timestamp data types |
| MD-2120 | Order by queries on short and byte datatypes fail |
| MD-2112 | Query with LIKE Predicate on Non-Varchar Fields Returns No Results |
| MD-2060 | Hash Partitioner Sender taking unusual amount of time |
| MD-2041 | Potential wrong result from spillable hash agg |
| MD-1964 | OOM when Hash Aggr outputs large aggregate data types |
| MD-1923 | Simple query failed when planner.use_simple_optimizer=true |
| MD-1867 |
SYSTEM ERROR: CannotPlanException when query MDB Json table |
Resolved Issues
This release of Drill includes the following resolved issues:
| MapR Tracking Numbers | Resolved Issues |
| MD-2825 | initcap function returning incorrect result |
| MD-2769 | Default value of security.admin.user_groups and security.admin.users is "true" |
| MD-2738 | Memory Limit Exception in External Sort |
| MD-2621 | Fix XSS vulnerabilities in Drill |
| MD-2617 | NullPointerException: Querying maprdb json tables |
| MD-2531 | Error connecting to Drill from Data Stage through Simba ODBC driver |
| MD-2436 | Issue with drill-config.sh checking java version |
| MD-2216 | DRILL-5660: Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867 |
| MD-2197 | DRILL-4511: refresh over empty folder results in error, we need a better error message |
| MD-2195 | DRILL-4755: StringIndexOutOfBoundsException seen with CONVERT_FROM function |
| MD-2146 | DRILL-4970: Wrong results when casting double to bigint or int |
| MD-2144 | DRILL-4720: MINDIR() and IMINDIR() functions return no results with metadata cache |
| MD-2130 | Drill wrong old date-time values displaying |
| MD-2125 | Queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous |
| MD-1988 | Spillable Hash Agg : OOM while constructing the hash table |
| MD-1980 | DRILL-5130: UNION ALL difference in results |
| MD-1970 | ArrayIndexOutOfBoundException occurs when querying HPE Ezmeral Data Fabric Database JSON with Drill |
| MD-1946 | DRILL-5140: Fix CompileException in run-time generated code when record batch has large number of fields |
| MD-1945 | DRILL-4139: Fix parquet partition pruning for BIT, INTERVAL and DECIMAL types |
| MD-1922 | Use indexDesc instead of indexFid in getIndexTable API to support HashPartitioned Indexes |
| MD-1890 | Drill Explorer on the mac does not connect when started as an app |
| MD-1848 | Drill not able to read HPE Ezmeral Data Fabric Database when FQDN is longer than 64 characters |
| MD-1828 | DRILL-5507 Millions of "Failure finding Drillbit running on host" info messages in foreman logs |
| MD-1776 | Issue with column alias for nested table calculated column when using sum () |
| MD-1671 | Have to restart drillbits whenever hive metastore is restarted. |
| MD-1661 | A query with limit 0 on HPE Ezmeral Data Fabric Database binary table not allowing filter to push past project |
| MD-1657 | Random Error : Flatten does not support inputs of non-list values |
| MD-1640 | Drill fails to compare multi-byte characters from hive table(DRILL-3250) |
| MD-1633 | Query remains in CANCELLATION_REQUESTED state after "java.io.IOException: Broken pipe" |
| MD-1618 | Moving a directory which contains a cache file causes subsequent queries to fail |
| MD-1607 | Reverse a String: IndexOutOfBoundsException |
| MD-1597 | simba and mapr jdbc driver differ in implementation |
| MD-1586 | Pam authentication with Centrify doesn't work by JPam restriction |
| MD-1576 | JBOSS Drill Connectivity - NullPointerException |
| MD-1524 | linux ODBC is ALWAYS returning 64K as the length of a varchar or char |
| MD-1515 | Drill planner/optimizer pushes down the Filter columns into Scan even after Filter is eliminated |
| MD-1473 | Query failed with error Statement "break OrOP2" is not enclosed by a breakable statement with label "OrOP2" |
| MD-1455 | Hash aggregate does not support schema changes |
| MD-1382 | Use simple optimizer for low latency operational queries |
| MD-1361 | Out of heap running CTAS against text delimited |
| MD-1359 | Wrong results after set planner.enable_nljoin_for_scalar_only=false |
| MD-1339 | Query text show as empty string in profile when queries are executed using PreparedStatements |
| MD-1256 | unset MAPR_TICKETFILE_LOCATION for sqlline |
| MD-1238 | Wrong result with Drill displaying date fields with parquet files generated by spark |
| MD-1237 | Cancelled query stuck into CANCELLATION_REQUESTED forever |
| MD-1205 | Simple query with only one join condition failed with "Hash join does not support schema changes" |
| MD-1168 | Drill Web UI Page Source Has Links To External Sites |
| MD-1115 | DRILL-5464: HashAgg throws a SchemaChangeException when atleast 1 scan fragment reads 0 rows |
| MD-965 | UnsupportedOperationException: Unable to get holder type for minor type [LATE] and mode [OPTIONAL]. |
| MD-853 | Query on TPC-H SF100 dataset fails with "Hash join does not support schema changes" [HPE Ezmeral Data Fabric Database JSON Tables] |
| MD-752 |
Remove mapr jars from Drill packages and source them from $MAPR_HOME/lib |