What's New in Release 7.0.0

Describes the new features in release 7.0.0 and provides links to more information.

Release 7.0.0 of the HPE Ezmeral Data Fabric provides substantial new features for the components of the data platform.

For new features delivered as part of the Ecosystem Pack, see What's New in EEP 8.0.0 and What's New in EEP 8.1.0.

HPE Ezmeral Data Fabric Object Store

HPE Ezmeral Data Fabric version 7.0.0 introduces Object Store, a native object storage solution that efficiently stores objects and metadata for optimized access. HPE Ezmeral Data Fabric Object Store directly leverages the HPE Ezmeral Data Fabric infrastructure for scalability and performance.

HPE Ezmeral Data Fabric Object Store is deployable on-premises, on edge, or in the cloud. You can access HPE Ezmeral Data Fabric Object Store through multiple protocols, including the native S3 API and S3 Select.

In previous releases, HPE Ezmeral Data Fabric supported the S3 Gateway ecosystem component. Going forward, HPE Ezmeral Data Fabric only supports HPE Ezmeral Data Fabric Object Store because it has several advantages over S3 Gateway. For more information, see Object Store vs S3 Gateway.

HPE Ezmeral Data Fabric Object Store utilizes proprietary HPE software to securely and efficiently store objects in the HPE Ezmeral Data Fabric platform. The efficient storage of objects optimizes access to data and analytic workloads.

HPE Ezmeral Data Fabric Object Store leverages the following patented HPE Ezmeral Data Fabric File Store capabilities that protect data and make object storage extremely reliable and scalable:

  • Snapshots
  • Mirroring
  • Replication
  • Global namespace
  • Data tiering/erasure coding

In addition to the HPE Ezmeral Data Fabric File Store features, HPE Ezmeral Data Fabric Object Store provides:

Object Store UI
Administrators can perform administrative tasks, such as managing and monitoring accounts, IAM users, access policies, buckets, and objects through a simple, intuitive interface or the CLI.
Multi-part uploads
Ability to upload a large object as a set of contiguous parts; each part is uploaded independently, in any order. Multi-part uploads are useful for objects greater than 100 MB.
WORM (Write Once Read Many)
When the Object Lock feature is enabled (via CLI or UI), write operations that would normally overwrite an existing object result in the creation of a new version of that object in the same bucket.
Load balancing
Object Store supports the following types of load balancing:
  • DNS and VIP-based load balancing
  • IP-based load balancing
Multiple S3 gateways can serve one bucket, and any S3 gateway can serve any bucket.
Accounts
Like AWS accounts, accounts in HPE Ezmeral Data Fabric Object Store are the administrative units that own buckets, policies, and users.

For more information, see HPE Ezmeral Data Fabric Object Store.

Security and Governance Enhancements

FIPS 140-2 Level 1 Compliance
Release 7.0.0 adds support for FIPS for new installations on Red Hat Enterprise Linux (RHEL). New features include:
  • Use of strong, FIPS-approved cryptographic algorithms:
    • Use of the OpenSSL 1.1.1 FIPS-approved cryptographic algorithms for encryption of over-the-wire data as well as data at rest.
    • Use of the Bouncy Castle BCFKS key and trust stores for encrypting sensitive data in key and trust stores.
  • Encryption of all sensitive data using strong cryptographic algorithms and the elimination of clear-text passwords.
  • Good security practices, such as:
    • Allowing separate passwords for keystores containing sensitive private keys, and trust stores that contain CA and server certificates that must be made readable to all users.
    • Enforcing the use of the PKIX trust manager over the vendor-specific SunX509 trust manager for performing certificate validation when using TLS 1.2 over HTTPS.
For more information about FIPS support and FIPS restrictions for the current release, see FIPS Compliance for HPE Ezmeral Data Fabric.
Improved Security
Non-FIPS compliant data-fabric installations continue to be supported. Release 7.0.0 offers improved security for all users, regardless of whether FIPS mode is enabled. Release 7.0.0:
  • Eliminates all clear-text passwords. Only non-secure installations continue to use clear-text passwords in the Hadoop configuration files:
    • ${MAPR_HOME}/hadoop/hadoop-${HADOOP_VERSION}/etc/hadoop/ssl-client.xml
    • ${MAPR_HOME}/hadoop/hadoop-${HADOOP_VERSION}/etc/hadoop/ssl-server.xml
    All secure (FIPS and non-FIPS) installations encrypt these passwords using the Hadoop Credential Provider API. Clear-text passwords no longer appear in these Hadoop configuration files.
  • Uses only FIPS-approved OpenSSL 1.1.1 algorithms for cryptographic operations. Unvalidated cryptographic libraries such as CryptoPP or custom-developed cryptographic algorithms are no longer supported. Weak cipers such as MD5 and DES are no longer used.
  • Supports separate passwords for key and trust stores. Previous data-fabric releases supported only a single password for both key and trust stores. This is undesirable because key stores contain sensitive information, so the passwords should be kept separate from trust stores, which need to be known to all clients.
  • Enhances the mrhsm utility. mrhsm was introduced in release 6.2.0 to configure KMIP support. Release 7.0 enhances mrhsm to include support for file-based key stores.
  • Encrypts the CLDB and DARE master keys using PKCS#11 and stores them in the ${MAPR_HOME}/conf/tokens folder. The CLDB and DARE master keys no longer exist in the plain-text files ${MAPR_HOME}/conf/cldb.key and ${MAPR_HOME}/conf/dare.master.key. Support for the plain-text, file-based cldb.key and dare.master.key is now optional for clusters upgraded from previous versions.
  • Includes a new property (isFips) in the output of the maprcli node list command to indicate whether a particular node is FIPS-enabled.
Ecosystem Component Support for FIPS
When used with release 7.0.0, most EEP 8.1.0 components support FIPS. For more information, see What's New in EEP 8.1.0.
Client Changes for FIPS
Enabling release 7.0.0 clients to communicate with earlier clusters requires copying an additional file (and not just the ssl_truststore and ssl-client.xml files) from the /opt/mapr/conf directory to the /opt/mapr/conf directory on the client. For more information, see Installing the Data Fabric Client (FIPS).
ZooKeeper Security Enhancements
Release 7.0.0 enhances ZooKeeper security by adding the following features:
  • Zookeeper now supports the BCFKS store type. When FIPS mode is enabled, the SSL key and trust stores used for ZooKeeper-quorum communication use the BCFKS store type. The following is an extract of the Zookeeper configuration file (${MAPR_HOME}/zookeeper/zookeeper-<version>/conf/zoo.cfg) when FIPS mode is enabled:
    ssl.quorum.keyStore.location=/opt/mapr/conf/ssl_keystore.bcfks
    ssl.quorum.trustStore.location=/opt/mapr/conf/ssl_truststore.bcfks
    On regular, non-FIPS secure installations, the key and trust stores continue to use the PKCS#12 store type:
    ssl.quorum.keyStore.location=/opt/mapr/conf/ssl_keystore.p12 
    ssl.quorum.trustStore.location=/opt/mapr/conf/ssl_truststore.p12 
  • When FIPS mode is enabled, the key and trust store passwords used for quorum communication are stored in the encrypted key and trust store credentials store in ${MAPR_HOME}/conf/maprkeycreds.bcfks and ${MAPR_HOME}/conf/maprtrustcreds.bcfks respectively, with a __##CREDENTIALS_STORE##__ tag as a password placeholder in the ZooKeeper configuration file:
    ssl.quorum.keyStore.password=__##CREDENTIALS_STORE##__ 
    ssl.quorum.trustStore.password=__##CREDENTIALS_STORE##__ 
    On regular, non-FIPS secure installations, the key and trust store passwords used for quorum communication are protected in the same manner. For non-secure installations, the key and trust store passwords continue to be stored in clear text as in previous releases.
For more information, see FIPS Compliance for HPE Ezmeral Data Fabric.
Enhancements to the mrhsm Utility
Release 7.0.0 introduced a new storetype option for use with the mrhsm init and mrhsm set commands to support both file-based and KMIP object stores. In release 6.2.0, the objectstore.backend setting of the mrhsm configuration file (${MAPR_HOME}/conf/maprhsm.conf) was set to kmip by default since the file-based object store was not supported. For example:
# more maprhsm.conf  
directories.tokendir = /opt/mapr/conf/tokens 
objectstore.type = external 
objectstore.backend = kmip 
log.level = INFO 
token.kmip.retry.interval = 5 
token.kmip.retry.attempts = 30 
token.kmip.read.timeout = 10
In release 7.0.0 and later, the storetype has file and kmip options but continues to be set to kmip by default to preserve backward compatibility:
# /opt/mapr/server/mrhsm init 
init 
  -sopin <PIN>               The PIN for the Security Officer (SO) 
  -label <text>              Defines the label of the object or the token. 
  [ -ip <ip1,ip2,...> ]      Comma-separated list of KMIP server IP addresses 
  [ -port <kmip-port> ]      KMIP port number. Default is 5696 
  [ -cacert <ca-cert> ]      Path to KMIP server CA certificate in PEM format 
  [ -clientcert <cert> ]     Path to client certificate in PEM format 
  [ -clientkey <key> ]       Path to client private key in PEM format 
  [ -kmipversion <version> ] KMIP version: 1.0, 1.1, 1.2, 1.3 or 1.4. Default: 1.1 
  [ -storetype file|kmip ]   Store type. Default: kmip
Release 7.0.0 also enhanced the mrhsm info command to display the key-store contents. For usage considerations and more information about the new file option, see:
Log4j Updates
To address several critical vulnerabilities, release 7.0.0 includes a patched version of Log4j v1.2.17 released as 1.3.1-mapr. The following classes have been removed from the 1.3.1-mapr JAR for Log4j:
  • src/main/java/org/apache/log4j/jdbc/JDBCAppender.class
  • src/main/java/org/apache/log4j/net/JMSAppender.class
  • src/main/java/org/apache/log4j/net/JMSSink.class
  • src/main/java/org/apache/log4j/net/SimpleSocketServer.class
  • src/main/java/org/apache/log4j/net/SocketNode.class
  • src/main/java/org/apache/log4j/net/SocketServer.class
  • src/main/java/org/apache/log4j/chainsaw/*.class
Collectively, these changes address the following vulnerabilities:
  • CVE-2019-17571
  • CVE-2017-5645
  • CVE-2021-4104
  • CVE-2021-44228
  • CVE-2022-23302
  • CVE-2022-23305
  • CVE-2022-23307
Release 6.x verions of data-fabric core also have been patched with the Log4j update. For more information, see the following support advisories:
The updated JAR is available in the external Maven repository: http://repository.mapr.com/maven/. Any data-fabric core or EEP project that includes Log4j v1 in its distribution should update the build script to the following Maven equivalent:
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.3.1-mapr</version>
 </dependency>
Data-Fabric SASL Authentication Enhancements for Ticket Handling
Release 7.0.0 enhances data-fabric SASL to enable applications that are not cluster aware, such as data-fabric ecosystem components, to gain access to services in another cluster for which they have a ticket. Peviously, applications such as Hive and Drill, that are not cluster aware could only connect to a default cluster.
For more information, see Authentication Enhancements for Ticket Handling. These enhancements are included in release 7.0.0 and in EBFs that can be applied to releases 6.1.x and 6.2.0.
Cross-Cluster Support for FIPS
Releases 7.0.0 enhances the configure-crosscluster.sh script to support cross-cluster configuration of FIPS clusters. For more information, see "FIPS Support" in configure-crosscluster.sh.
New Ticket Type: servicewithimpersonationandticket
Release 7.0.0 introduces a new servicewithimpersonationandticket ticket type that allows some ticket holders to generate tickets subject to their impersonation authority. For more information, see Managing Tickets and Generating an Impersonation Ticket with Ticket Generation Privileges.

Dynamic Data Masking

Release 7.0.0 adds support for dynamic data masking (DDM). DDM is the ability to apply a variety of data masks in real-time, depending on who is accessing the data. DDM aims to mask data in transit, leaving the original data in the database unaltered.

Dynamic Data Masking has the following benefits:
  • Includes six predefined DDM formats for different types of data obscuration​.
  • You can add DDM formats to columns in new and existing JSON tables, and use the new permissions for ACEs and/or security policies to read unmasked data.
  • The format of the original data is preserved, so existing applications run unchanged.
  • Secures data and reduces compliance, scope, and costs​.

For more information, see Dynamic Data Masking.

Performance Improvements

Remote Direct Memory Access (RDMA)
Release 7.0.0 adds support for remote direct memory access (RDMA). New 7.0.0 installations use RDMA by default if network cards support it​. RDMA transfers data directly between user space process buffers on separate servers​ to bypass the Linux kernel and server CPU​ for increased performance and lower CPU utilization. RDMA uses a network card to manage data transfer and memory access. ​
​RDMA is used for data transfers between components on distinct servers:​
  • HPE Ezmeral Data Fabric file client (Java, C, FUSE-based, loopback NFS) and HPE Ezmeral Data Fabric fileserver (mfs, fileserver)​
  • Linux NFS clients and HPE Ezmeral Data Fabric NFS gateway​
  • HPE Ezmeral Data Fabric NFS gateway and HPE Ezmeral Data Fabric fileserver​
  • HPE Ezmeral Data Fabric fileservers with each other​

If RDMA is not available, TCP/IP is used instead.

MOSS Metrics Added to guts
Release 7.0.0 adds new metrics to the guts utility that allow you to measure and analyze the performance of the Multithreaded Object Store Server (MOSS).
For more information, see guts.

Operating System Support

Ubuntu 20.04 Support
Release 7.0.0 adds support for SLES 15 SP3 and Ubuntu 20.04 but removes support for CentOS, which has reached end of life (EOL) status. For a complete list of supported operating systems, see Operating System Support Matrix.

HPE Ezmeral Ecosystem Pack (EEP) Support

EEP 8.1.0 is new for this release and can be used with release 7.0.0 and release 6.2.0. For more information about EEP 8.1.0, see EEP 8.1.0 Reference Information. Note that EEP 7.1.2 can also be used with release 7.0.0 and release 6.2.0.

For more information about the EEPs that can be used with different versions of core releases, see EEP Support and Lifecycle Status.

Spark Support in Release 7.0.0

The EEP you install for use with release 7.0.0 determines the Spark version that is available:
EEP Works with Core And Includes Spark Version
8.1.0 6.2.0 and 7.0.0 3.2.0.0
7.1.2 6.2.0 and 7.0.0 2.4.7.200

If your environment requires Spark 2.x, be sure to install EEP 7.1.2. If your environment requires Spark 3.x, install EEP 8.1.0 or later.

For the versions of other components included in EEPs 7.1.2 and 8.1.0, see Component Versions for Released EEPs. For the Spark release notes, see Spark Release Notes. For EEP reference information, see Ecosystem Pack (EEP) Reference.

Updated Apache Kafka Java APIs

See Apache Kafka Java APIs and Apache Kafka 2.6.1 APIs used with HPE Ezmeral Data Fabric Data Streams.

Documentation Enhancements

Documentation Content Removed for Release 7.0.0
The following products that were supported in release 6.2.0 are not supported in release 7.0.0. Therefore, documentation topics for these products have been removed from the release 7.0.0 documentation:
  • Persistent Application Container Client (PACC)
Revised Product Naming
Release 6.2 introduced a new name for the data platform:
In addition to the data platform, many former MapR products and features have new names. Product documentation and interfaces are being updated to reflect the new names.
Even though the product names are different:
  • The platform works the same.
  • Upgrades from MapR installations are supported unless noted otherwise.
  • HPE Ezmeral features are fully compatible with legacy MapR features unless noted otherwise.
  • Product versioning remains the same, with the exception of the four-digit versions used for core and patches. See Installation Notes (Release 7.0.0).
Some product interfaces continue to use the term MapR. These interfaces may or may not be updated. The following table shows key terms that are changing in the product documentation for release 6.2.0:
MapR Term HPE Ezmeral Data Fabric Term
MapR <release-number> release <release-number>
MapR Academy HPE Training
MapR admin data-fabric admin
MapR client data-fabric client
MapR cluster data-fabric cluster
MapR Container for Developers development environment for HPE Ezmeral Data Fabric
MapR Control System (MCS) control system (MCS)
MapR core core
MapR Data Access Gateway data-access gateway
MapR Data Platform HPE Ezmeral Data Fabric
MapR Data Platform for Kubernetes Kubernetes Interfaces for Data Fabric
MapR Database HPE Ezmeral Data Fabric Database
MapR Data Science Refinery Data Science Refinery*
MapR Distribution for Apache Hadoop HPE Ezmeral Data Fabric Distribution for Apache Hadoop
MapR Ecosystem Pack (MEP) Ezmeral Ecosystem Pack (EEP)
MapR Edge HPE Ezmeral Data Fabric Edge
MapR Event Store for Apache Kafka HPE Ezmeral Data Fabric Streams
MapR Filesystem file system
MapR gateway data-fabric gateway
MapR Hadoop Hadoop for the HPE Ezmeral Data Fabric
MapR Installer Installer
MapR Installer Stanza Installer Stanza
MapR license HPE Ezmeral Data Fabric license
MapR loopbacknfs POSIX client loopbacknfs POSIX client
MapR Monitoring monitoring
MapR NFS NFS or NFS for the HPE Ezmeral Data Fabric
MapR package data-fabric package
MapR Object Store with S3-Compatible API S3 Gateway
MapR patch patch
MapR Persistent Application Container Client (PACC) Persistent Application Container Client (PACC)
MapR POSIX Client POSIX client
MapR Professional Services HPE Pointnext
MapR Sandbox sandbox
MapR server ticket server ticket
MapR services services
MapR software data-fabric software or HPE Ezmeral Data Fabric software
MapR Support HPE Support
MapR Technologies Hewlett Packard Enterprise Company
MapR ticket ticket
MapR user data-fabric user
MapR volume volume
MapR XD Distributed File and Object Store HPE Ezmeral Data Fabric File Store
MapR-SASL data-fabric SASL
MEP (MapR Ecosystem Pack) MEP (ecosystem pack)
*Deprecated for release 6.2.0. For more information, see "Deprecated Products and Features" earlier on this page.