HPE Ezmeral Data Fabric 6.1.x is In Maintenance and transitions to "End of Maintenance" in June 2024. Please see the latest documentation.

About MapR 6.1
This site contains the main documentation for Version 6.1 of the MapR Converged Data Platform, including installation, configuration, administration, and reference information.
6.1 Installation
This section contains information about installing and upgrading MapR software. It also contains information about how to migrate data and applications from an Apache Hadoop cluster to a MapR cluster.
6.1 MapR Data Platform
MapR Data Platform is the industry-leading data platform for AI and analytics that solves enterprise business needs.
6.1 Administration
This section describes how to manage the nodes and services that make up a cluster.
- Administering Users and Clusters
  Lists topics that help manage a MapR cluster.
- Administering Nodes
  Provides a synopsis of managing nodes in a cluster.
- Administering Volumes
  This section provide information about how to organize and manage data using volumes, a unique feature of MapR clusters.
- Administering Files and Directories
- Administering Tables
  Administration of the MapR Database is done primarily via the command line (maprcli) or with the Managed Control System (MCS). Regardless of whether the MapR Database table is used for binary files or JSON documents, the same types of commands are used with slightly different parameter options. MapR Database administration is associated with tables, columns and column families, and table regions.
- Administering Streams
- Administering MapR Gateways
  A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. You can replicate MapR Database tables (binary and JSON) and MapR Event Store For Apache Kafka streams. MapR gateways also apply updates from JSON tables to their secondary indexes and propagate Change Data Capture (CDC) logs.
- Administering Services
- Monitoring the Cluster
  This section describes how to monitor the health and performance of a MapR cluster.
- Configuring Security
  Describes how to configure security and manage secure clusters.
- Managing Secure Clusters
  Provides procedures that will enable you to use MapR clusters securely.
- Administering the MapR Data Access Gateway
  The MapR Data Access Gateway is a service that acts as a proxy and gateway for translating requests between lightweight client applications and the MapR cluster. This section describes considerations when upgrading the service, how to modify configuration settings, and how to administer and manage the service.
- Planning for High Availability
- Administrator's Reference
  This section contains in-depth reference information for the administrator.
  - maprcli and REST API Syntax
    This section provides information about the MapR command API. Most commands can be run on the command-line interface (CLI), or by making REST requests programmatically or in a browser.
  - Utilities
    Contains information about various scripts and utilities, that help setup, maintain, and monitor clusters.
  - Configuration Files
    This section contains reference information about various configuration files.
  - Alarms Reference
    The pages in this section provide details about all of the types of alarms.
  - MapR Data Platform Environment
    This section provides information associated with the MapR Data Platform environment.
    - MapR Data Platform Parameters
      Describes MapR Data Platform parameters and their default values.
    - Default MapR Data Platform Configurations
      Lists sources from which default MapR Data Platform configuration parameters are derived.
    - Environment Variables
      Describes the environment variables specific to the MapR Data Platform.
    - Ports Used by MapR Data Platform Software
      Lists the ports used by Data Fabric services.
    - Log Files
      Lists the log files for each MapR Data Platform component.
      - Increasing Log Retention
      - Setting the Tracing Level
      - Configuring Profiling for Operations
      - Archiving CLDB Logs
      - Enabling Runtime Logging
      - Viewing Audit Logs
      - Viewing Application Logs
        Logging Options
        YARN Log Aggregation
        Enabling YARN Log Aggregation
        Enabling YARN Local-Node Log Aggregation
        Viewing Logs for Completed Applications
        Editing the Retention Settings of Aggregated Logs
        Centralized Logging
        Describes the centralized logging feature of MapR Data Platform.
      - Viewing the Service Log
        Esplains how to view service logs using Kibana.
    - Cluster Maintenance Schedule
      Lists a sample maintenance schedule for the cluster.
    - Language Support for MapR Database Tables
      This section lists the human languages that MapR Data Platform tables can store, retrieve, and process.
  - Sample JSON File for Metering
    A sample metering JSON file for an 8-node cluster with no workloads enabled.
  - Metering Data Descriptions
    This table lists the metrics collected by the metering feature.
- Troubleshooting Cluster Administration
  Lists the common errors and their solutions.
- Best Practices for Backing Up MapR Information
  Lists the best practices and performance considerations to follow when backing up MapR information.
6.1 Development
This section contains information related to application development for Ezmeral ecosystem components and MapR Data Platform products, including the file system, Database (Key-Value and JSON), and Event Streams.
Other Docs
This section contains release-independent information, including: MapR Installer documentation, Ecosystem release notes, interoperability matrices, security vulnerabilities, and links to other MapR version documentation.
Glossary
Definitions for commonly used terms in MapR Converged Data Platform environments.

Enabling YARN Local-Node Log Aggregation

The steps in this procedure configure log aggregation for NodeManager processes, enabling you to store the logs on nodes (node-local volumes) where the YARN containers are launched.

To enable YARN local-node log aggregation, add or edit the following properties in the yarn-site.xml file:

Set the yarn.node-local-log-aggregation.enable value to true.
NOTE The default setting for YARN Log Aggregation (yarn.log-aggregation-enable) should be removed or set to false in the yarn-site.xml file.
Optional: Set the yarn.node-local-log-aggregation.metadata-path value to a location in the system. By default the location is maprfs:///NM_REMOTE_APP_LOG_DIR/<user>/logsMeta. NM_REMOTE_APP_LOG_DIR should match the yarn.nodemanager.remote-app-log-dir property.
NOTE The location should not be an absolute path (the location begins from /). In the MapR File System (maprfs), the default setting for NM_REMOTE_APP_LOG_DIR is /tmp/logs.
Optional: Set the node-local-log-aggregation.metadata-filename value to the name of the metafile that should contain the information about containers for each node. By default, the file name is containers.seq. If you use default paths, the file is stored at /tmp/logs/${user}/logsMeta/<appId>/<nodeName>/containers.seq.
Restart NodeManager and HistoryServer services.

Aggregated logs are owned by the user who runs the job.

Different users cannot see each other's logs. For example, if the MapR Data Platform user admin runs a job, the logs are stored in maprfs:///var/mapr/local/<nodeNames>/mapred/nodeManager/logs/admin/<appId>. If a user analyst runs a job, the logs are stored in maprfs:///var/mapr/local/<nodeNames>/mapred/nodeManager/logs/analyst/<appId>.