What's New in Version 6.1.0

The 6.1.0 MapR release supplies substantial new features for the components of the data platform.

To see new features delivered as part of the Ezmeral Ecosystem Pack, see What's New in EEP 6.0.0 There are multiple new features for MapR-Drill. See the Drill release notesfor details.

New Repository for MapR Software

On August 1, 2023, Hewlett Packard Enterprise introduced a new download repository for MapR Data Platform core and ecosystem software packages. https://package.ezmeral.hpe.com/ is the new repository for MapR downloads. For all MapR releases, https://package.ezmeral.hpe.com/ replaces two older repositories:
Description URL Authentication Required?
New repository https://package.ezmeral.hpe.com/ Yes
Old repositories https://package.mapr.com/

https://package.mapr.hpe.com/

Yes1
1Beginning October 2023, the old repositories are redirected to the new repository URL, which requires authentication.

The new repository requires you to provide the email and token for your HPE Passport account. Software that points to the old repositories must be updated to include your HPE Passport email and token. For more information about using the new repository, see Using the HPE Ezmeral Token-Authenticated Internet Repository.

If you plan to use the MapR Installer, you must update the Installer to version 1.18.0.3 or later. Earlier versions of the Installer will not work with the new repository. See Updating the MapR Installer.

MapR 6.1.0 Is "Secure by Default"

Because MapR 6.1.0 in its default configuration is more secure than MapR 6.0.1, documentation references to "built-in security" have been changed to "secure by default." See Security for Ecosystem Components.

Streaming Security and Critical Data Asset Protection

ZooKeeper Upgraded

MapR 6.1.0 includes ZooKeeper 3.4.11 (upgraded from Zookeeper 3.4.5 in MapR 6.0.1) to support ZooKeeper server-to-server authentication.

Zookeeper Supports Server-to-Server Authentication
As of MapR 6.1.0, ZooKeeper is automatically configured for server-to-server authentication with new installations of MapR Core. The following ZooKeeper security parameters are set to "true" whenever you use configure.sh to perform a new installation (including when configure.sh is invoked by the MapR Installer or MapR Installer Stanzas).
  • quorum.auth.enableSasl
  • quorum.auth.learnerRequireSasl
  • quorum.auth.serverRequireSasl

These parameters enable secure communication between peer servers in the ZooKeeper quorum using SASL.

Simplified Development and Deployment of AI and Analytics Applications

Lightweight Client Architecture

EEP 6.0.0 introduces Node.js and Python OJAI clients that enable you to write MapR Database JSON applications using a language other than Java. These clients use the MapR Data Access Gateway to access your MapR cluster. The Data Access Gateway performs some data processing that otherwise runs in the client. This keeps the clients lightweight and simplifies their installation and use.

See Understanding the MapR Data Access Gateway for more details about the Data Access Gateway. See What's New in EEP 6.0.0 for more details about the new clients.

New Features in MapR Filesystem

Storage Tiers Support for Files
MapR Filesystem v6.1.0 includes rule-based automated tiering functionality for businesses looking to leverage low-cost storage solutions either on low-cost hardware or on the cloud to gain limitless storage capacity. MapR's tiering functionality can seamlessly integrate with the following:
  • Low-cost hardware to store data that is less frequently accessed, which is referred to as "warm" data
  • Cloud resources to store data that is rarely accessed or archived, which is referred to as "cold" data

Enabling warm or cold tiering allows you to use valuable on-premise storage resources for more active or "hot" file data and applications. You can use "warm" or "cold" tiering for file data retained for compliance, historical, or other business reasons.

See Data Tiering for more information.

NFSv4 Protocol Support
As of MapR v6.1, the MapR Filesystem includes support for NFSv4 protocol. NFSv4 provides end-to-end secure filesystem access and a much higher throughput compared to NFSv3, improving performance. MapR uses NFS Ganesha, which is an Open Source userspace implementation of the NFS server, for supporting NFSv4 features. See Accessing Data with NFS v4 for more information.

Also supported are mixed-mode NFS configurations in which some nodes of a cluster use NFSv3 and other nodes use NFSv4. See Installing MapR NFS.

When installed through the MapR Installer, there is no security between the client and the NFS gateway whether or not the cluster is secure. On a secure cluster, the connection between NFSv4 server and MapR File System is secure. The following options are supported for configuring Kerberos security for NFSv4:
  • Kerberos only (users are in the local node database)
  • LDAP and Kerberos (users are in LDAP)
  • Active Directory and Kerberos (users are in the Active Directory)

    For more information, see Configuring NFSv4 Server for Kerberos.

When configuing NFSv4 for security, make a note of the following:
  • If NFSv4 is installed on (existing or new) edge nodes, the edge nodes must be part of the Kerberos setup.
  • If NFSv4 is installed on cluster nodes, then all the cluster nodes must be part of the Kerberos setup.

For more information, see Accessing Data with NFS v4.

Secure by Default
The MapR Data Platform v6.1 and EEP v6.0 components are secure out-of-the-box on all new installations, ensuring all network connections require authentication and all data in motion is protected with wire-level encryption. Without requiring an external security manager server or a particular security plug-in for each ecosystem component, MapR provides the ability to apply security protection directly for data as it comes into and moves out of the platform. The security semantics are applied automatically on data being retrieved or stored by any ecosystem component, application, or users. See Security for more information.

Because MapR 6.1.0 in its default configuration is more secure than MapR 6.0.1, documentation references to "built-in security" have been changed to "secure by default." See Security for Ecosystem Components.

Encryption of Data at Rest
MapR v6.1 includes support for encryption of data at rest. Data on disk (or data-at-rest) in a secure MapR cluster can be encrypted, enabling you to protect the data if a disk is compromised. Encryption of data-at-rest not only prevents unauthorized users from accessing sensitive data, but it also protects against data theft via sector-level disk access.

See Security for more information.

Client-side Port Binding

MapR now requires only a single source port on the client side. MapR will bind multiple sockets to the same port for establishing connections to all the nodes on the MapR cluster.

Core Data Services Innovations to Speed AI and Analytics and Lower TCO

Complex Types Support in MapR Database JSON

In MapR Database 6.1.0, you can write more expressive queries on complex types, which includes arrays of scalar types and arrays of nested documents. MapR Database JSON introduces the notion of a container field path. Using a container field path, you can access a field that is either a single value or an arbitrary array element. This is useful if you want to perform one of the following operations:

  • Perform comparisons on a field path that is either a single value or an arbitrary array element
  • Access subfields in a nested document, where the nested document is either an arbitrary array element or a single nested document
  • Access arbitrary elements in an array

For example, suppose your JSON document contains an addresses field that is an array of nested documents with address details like street, city, and state. You can use a container field path to write a query that filters on any element in the addresses array that matches a specific city and state.

You can also create secondary indexes using container field paths, improving the performance of these more expressive queries.

See the following topics for more details about the feature:

In MapR Database 6.1.0, you also are no longer restricted to creating secondary indexes on scalar data fields. You can now create indexes on fields with arrays and nested documents. See Data Types and Secondary Index Fields for more information.

To understand how indexes behave, depending on the version you are using, see Secondary Indexes and Upgrades.

Complex Type Support in Drill
The query planner in Drill can leverage indexes created on MapR Database JSON document fields with complex data types. You must write queries using specific SQL syntax for the query planner in Drill to leverage indexes on complex fields.

New Features in MapR Event Store For Apache Kafka

Idempotent (Exactly-Once) Producer

An "exactly-once" message delivery semantic produces messages without duplication. Each message is delivered once and only once. Exactly-once is insured by uniquely identifying a group of messages that are atomically persisted. Exactly-once message delivery is set with the producer idempotence option. See Enabling an Idempotent Producer for more information.

Log Compaction

Log compaction purges previous, older messages that were published to a topic-partition and retains the latest version of the record. See Log Compaction for more information.

Apache Kafka 1.1 Support
MapR 6.1.0 introduces support for the Apache Kafka 1.1 API.

Other Enhancements

Security Certificate Expiry Alarm
A new alarm has been added to alert users to security certificates that are expiring within a configurable time period. See Security Certificate Expiry Alarm.
MapR Database Enhancements
MapR Database Table Metrics

MapR Database 6.1.0 supports table metrics and enhanced node metrics. Table metrics provide more granular metrics. They enable you to detect and diagnose bottlenecks and performance issues that are specific to individual tables.

MapR enables table metrics by default. You cannot disable metrics on individual tables. During installation, you can disable table metrics across your entire cluster by selecting a minimal configuration for metrics collection.

See MapR Database Metrics for more details. For details about table metrics, including how to disable them, see MapR Database Table Metrics. To learn about how to view these metrics in the MapR Control System (MCS), see Visualizing Table Metrics in the Control System.

OJAI 3.0 Support
MapR 6.1.0 introduces support for the OJAI 3.0 API. This version includes extensions for new complex types support in MapR Database JSON. See Complex Types Support in MapR Database JSON for an overview of this feature.
Documentation Enhancements
New Component Versions Matrix

A new matrix has been added to the list of Interoperability Matrices. The Component Versions for Released EEPs matrix lets you compare the versions of ecosystem and MapR Monitoring components across different EEPs.

Revised Product Naming

Some components of the MapR Data Platform have been changed. MapR-Streams is now called MapR Event Store For Apache Kafka. MapR-DB is now MapR Database. MapR-FS is now MapR File System.

Removal of Third-Party Solutions Content
Third-Party Solutions information has been removed from the MapR 6.1 documentation. Installation and usage information for the following products is no longer current and will not be updated:
  • Datameer
  • HParser
  • Pentaho
  • Vertica Analytics Platform