Drill 1.11.0-1710 Release Notes
This section provides reference information, including new features, fixes, known issues, and limitations for Drill 1.11.0-1710.
The following release notes apply to the 1.11.0 version of the Apache Drill component included in the MapR Converged Data Platform.
Version | 1.11.0 |
Release Date | November 2017 |
MapR Version Interoperability | See EEP Components and OS Support, Interoperability Matrix, and Drill Support Matrix. |
Package Names | Package Names for Ecosystem Packs (EEPs) |
New in this Release
- Secondary index support on HPE Ezmeral Data Fabric Database for JSON tables.
- MapR default security configuration that secures the cluster and ecosystem components using MapR security (mapr tickets) to provide authentication, authorization, and encryption.
- Client to Drillbit Encryption using SSL/TLS.
- Spill to disk for the Hash Aggregate operator (in addition to the Sort operator).
- Throttle-based resource assignment (queuing feature).
- Drill integration with Zeppelin.
In addition to the features mentioned above, Drill 1.11 also includes the following
features and improvements:
MapR Tracking Numbers | Features |
MD-2490 | Addition of an internal options table. |
MD-1813 | Multiplexing session over a connection. |
MD-1655 | Provide an option to persist Query Profiles in a Drillbit server's memory |
MD-1245 | MapR-SASL Plugin for ODBC/Drill (Authentication & Encryption) |
MD-536 | SASL support between Drillclient and Drillbit |
MapR Tracking Numbers | Improvements |
MD-2962 | Add a UDF to decode encoded fieldpath |
MD-2871 | Restricted scan (for non-covering) gets over-parallelized due to scan range size |
MD-2799 | DRILL-5811 Large number of "Failure finding Drillbit" messages when using file system |
MD-2789 | Fix correct batch size for merge back during non-covering query. |
MD-2693 | Drill should use `com.mapr.db.index.IndexDesc.getNullOrdering()` to decide if an index can be used for order by |
MD-2656 | Drill needs to return more specific error for DB |
MD-2647 | Enable UTF-8 support in query string by default |
MD-2506 | Standardize on JDBC driver that gets shipped with Drill and sqlline |
MD-2422 | Drill performance - doing less than 40 queries/sec |
MD-2416 | RestrictedJsonTableGroupScan should support projection pushdown |
MD-2403 | Drillbit fails to start when only keystore path is provided without keystore password. |
MD-2339 | Additional log events for debugging index selection |
MD-2319 | Query Containing Wildcard in String Matching Filter Runs 4x Slower Than Impala |
MD-2301 | Support Impersonation without authentication for REST API |
MD-2278 | Include flags controlling secure Drill ZK ACLs in the secure installer |
MD-2225 | Review the names of index planning config options |
MD-2196 | DRILL-4264: Allow field names to include dots |
MD-2192 | Provide log tracing of a query submitted by an OJAI client for user debugging |
MD-2074 | Memory Fragmentation Fundamentals Completion |
MD-2067 | Handle the case of order-by with no filter conditions |
MD-2002 | Operator Specific Handle for Empty Batches |
MD-1987 | Drill should read HPE Ezmeral Data Fabric Database tablets in parallel |
MD-1962 | Create configuration file for distribution specific configuration |
MD-1836 | Leverage hash index when doing secondary index planning |
MD-1823 | Add logging information for listing candidate indexes for a given query |
MD-1815 | Leverage HPE Ezmeral Data Fabric Database batch gets for better performance. |
MD-1789 | Cast function support pull request and code review process |
MD-1786 | Push down limit on HPE Ezmeral Data Fabric Database table scan |
MD-1752 | Throttling-based per-query memory assignment |
MD-1673 | Add Statistics support for HPE Ezmeral Data Fabric Database |
MD-1665 | Provide query level option to pick specific indexes |
MD-1643 | Disable table caching by default |
MD-1498 | Plan queries against HPE Ezmeral Data Fabric Database JSON faster |
MD-1318 | Improve Low Latency Performance of Drill |
MD-1209 | Configuring ZooKeeper Access Controls for Drill on a Secure Cluster |
Fixes
None
Known Issues and Limitations
This release of Drill includes the following known issues:
MapR Tracking Numbers | Known Issues |
MD-3171 | Drillbit down due to fs client assertion in Java_com_mapr_fs_jni_MapRTableTools_GetScanner |
MD-3162 | DRILL-5946: Document maprdb format plugin options |
MD-3160 | Hit ExpressionParsingException: Expression has syntax error! line 1:0:no viable alternative at input 'TIME' |
MD-3155 | Simba JDBC driver dependency issues with third party applications |
MD-3131 | Memory leak by HashPartitionSender |
MD-3108 | planner.memory.max_query_memory_per_node should be default to 4GB |
MD-3089 | Some queries with join on two or more tables only pick non-covering index plans |
MD-3071 | Queries using hash index plans (intermittently) fail with SYSTEM ERROR: RpcException: Data not accepted downstream. |
MD-3047 | Apache Drill JDBC driver from mapr-drill package not works |
MD-3041 | Identify closed jdbc connection |
MD-3040 | need a better error message - RpcException: Drillbit (demo-179.lab) does not require auth, but auth is enabled. |
MD-3037 | Performance issues with queries against HPE Ezmeral Data Fabric Database JSON tables (full table scans) |
MD-3031 | Duplicate columns projected from HPE Ezmeral Data Fabric Database Scan for large IN condition |
MD-2979 | Memory leak by ParquetRowGroupScan |
MD-2950 | Full outer join on two empty hive tables throws IllegalArgumentException: Full outer join not currently supported |
MD-2948 | SI: Non-covering queries hang after drillbit restart |
MD-2848 | IteratorValidatorBatchIterator is throwing exception when assertions are enabled. |
MD-2838 | After upgrading from MapR Drill 1.10.0 to 1.11.0, drillbit hangs on startup |
MD-2837 | Queries on HPE Ezmeral Data Fabric Database 5.x tables hang |
MD-2809 | Scalar replacement failures |
MD-2791 | Cancelled query using index plan logs several errors |
MD-2790 | Queries fail with various CONNECTION ERRORs |
MD-2695 | Test Framework - Test fails with DATA_READ_ERROR : Failed to load table into HBase |
MD-2589 | Drillbit startup has errors: "Can't load log handler java.util.logging.FileHandler" |
MD-2544 | drillbit jvm dumps core when querying a table with SI |
MD-2528 | Parquet int_64 causes UnsupportedOperationException in Drill |
MD-2515 | Evaluate Codegen Compilers for performance overhead |
MD-2513 | Querying last column in CSV causes IndexOutOfBoundsException |
MD-2455 | Query with INTERVAL in predicate does not return any rows |
MD-2420 | External sort with higher memory takes longer to finish vs. smaller memory |
MD-2419 | Query fails with OOM with queuing enabled and exchanges disabled |
MD-2415 | Resource leak during Hash Agg spill to disk longevity run |
MD-2400 | Random Jenkins unit test failure due to ".crc" files in spill directory |
MD-2397 | Zeppelin cannot connect to Drill through the JDBC driver on a secure MapR cluster when Zeppelin has Kerberos authentication enabled. |
MD-2285 | Filter with interval was not fully pushed down |
MD-2218 | Direct memory fragmentation cause OOM while running OJAI queries |
MD-2141 | Predicates on indexed field, joined by OR, do not return the correct results for time and timestamp data types |
MD-2120 | Order by queries on short and byte datatypes fail |
MD-2112 | Query with LIKE Predicate on Non-Varchar Fields Returns No Results |
MD-2060 | Hash Partitioner Sender taking unusual amount of time |
MD-2041 | Potential wrong result from spillable hash agg |
MD-1964 | OOM when Hash Aggr outputs large aggregate data types |
MD-1923 | Simple query failed when planner.use_simple_optimizer=true |
MD-1867 |
SYSTEM ERROR: CannotPlanException when query MDB Json table |
Resolved Issues
This release of Drill includes the following resolved issues:
MapR Tracking Numbers | Resolved Issues |
MD-2825 | initcap function returning incorrect result |
MD-2769 | Default value of security.admin.user_groups and security.admin.users is "true" |
MD-2738 | Memory Limit Exception in External Sort |
MD-2621 | Fix XSS vulnerabilities in Drill |
MD-2617 | NullPointerException: Querying maprdb json tables |
MD-2531 | Error connecting to Drill from Data Stage through Simba ODBC driver |
MD-2436 | Issue with drill-config.sh checking java version |
MD-2216 | DRILL-5660: Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867 |
MD-2197 | DRILL-4511: refresh over empty folder results in error, we need a better error message |
MD-2195 | DRILL-4755: StringIndexOutOfBoundsException seen with CONVERT_FROM function |
MD-2146 | DRILL-4970: Wrong results when casting double to bigint or int |
MD-2144 | DRILL-4720: MINDIR() and IMINDIR() functions return no results with metadata cache |
MD-2130 | Drill wrong old date-time values displaying |
MD-2125 | Queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous |
MD-1988 | Spillable Hash Agg : OOM while constructing the hash table |
MD-1980 | DRILL-5130: UNION ALL difference in results |
MD-1970 | ArrayIndexOutOfBoundException occurs when querying HPE Ezmeral Data Fabric Database JSON with Drill |
MD-1946 | DRILL-5140: Fix CompileException in run-time generated code when record batch has large number of fields |
MD-1945 | DRILL-4139: Fix parquet partition pruning for BIT, INTERVAL and DECIMAL types |
MD-1922 | Use indexDesc instead of indexFid in getIndexTable API to support HashPartitioned Indexes |
MD-1890 | Drill Explorer on the mac does not connect when started as an app |
MD-1848 | Drill not able to read HPE Ezmeral Data Fabric Database when FQDN is longer than 64 characters |
MD-1828 | DRILL-5507 Millions of "Failure finding Drillbit running on host" info messages in foreman logs |
MD-1776 | Issue with column alias for nested table calculated column when using sum () |
MD-1671 | Have to restart drillbits whenever hive metastore is restarted. |
MD-1661 | A query with limit 0 on HPE Ezmeral Data Fabric Database binary table not allowing filter to push past project |
MD-1657 | Random Error : Flatten does not support inputs of non-list values |
MD-1640 | Drill fails to compare multi-byte characters from hive table(DRILL-3250) |
MD-1633 | Query remains in CANCELLATION_REQUESTED state after "java.io.IOException: Broken pipe" |
MD-1618 | Moving a directory which contains a cache file causes subsequent queries to fail |
MD-1607 | Reverse a String: IndexOutOfBoundsException |
MD-1597 | simba and mapr jdbc driver differ in implementation |
MD-1586 | Pam authentication with Centrify doesn't work by JPam restriction |
MD-1576 | JBOSS Drill Connectivity - NullPointerException |
MD-1524 | linux ODBC is ALWAYS returning 64K as the length of a varchar or char |
MD-1515 | Drill planner/optimizer pushes down the Filter columns into Scan even after Filter is eliminated |
MD-1473 | Query failed with error Statement "break OrOP2" is not enclosed by a breakable statement with label "OrOP2" |
MD-1455 | Hash aggregate does not support schema changes |
MD-1382 | Use simple optimizer for low latency operational queries |
MD-1361 | Out of heap running CTAS against text delimited |
MD-1359 | Wrong results after set planner.enable_nljoin_for_scalar_only=false |
MD-1339 | Query text show as empty string in profile when queries are executed using PreparedStatements |
MD-1256 | unset MAPR_TICKETFILE_LOCATION for sqlline |
MD-1238 | Wrong result with Drill displaying date fields with parquet files generated by spark |
MD-1237 | Cancelled query stuck into CANCELLATION_REQUESTED forever |
MD-1205 | Simple query with only one join condition failed with "Hash join does not support schema changes" |
MD-1168 | Drill Web UI Page Source Has Links To External Sites |
MD-1115 | DRILL-5464: HashAgg throws a SchemaChangeException when atleast 1 scan fragment reads 0 rows |
MD-965 | UnsupportedOperationException: Unable to get holder type for minor type [LATE] and mode [OPTIONAL]. |
MD-853 | Query on TPC-H SF100 dataset fails with "Hash join does not support schema changes" [HPE Ezmeral Data Fabric Database JSON Tables] |
MD-752 |
Remove mapr jars from Drill packages and source them from $MAPR_HOME/lib |