Home
6.0 Installation
This section contains information about installing and upgrading MapR software. It also contains information about how to migrate data and applications from an Apache Hadoop cluster to a MapR cluster.
Installing MapR and/or MapR Ecosystem Components
Describes how to install MapR software and ecosystem components with the MapR Installer or without the MapR Installer.
Installing without the MapR Installer
Describes how to install MapR software and ecosystem components manually.
Step 10: Install Ecosystem Components Manually
You can install one or more ecosystem components from any MapR Ecosystem Pack (MEP) that is supported by the MapR cluster version. A MEP consists of a group of ecosystem components that work together.
Installing Spark on YARN
This topic includes instructions for using package managers to download and install Spark on YARN from the MEP repository.

MapR 6.0 Documentation

6.0 Installation
This section contains information about installing and upgrading MapR software. It also contains information about how to migrate data and applications from an Apache Hadoop cluster to a MapR cluster.
- Planning the Cluster
  Describes information and factors used in planning your cluster.
- Installing MapR and/or MapR Ecosystem Components
  Describes how to install MapR software and ecosystem components with the MapR Installer or without the MapR Installer.
  - MapR Repositories and Packages
    Describes the repositories for MapR software and Hadoop Ecosystem tools.
  - Preparing each Node
    Identifies major factors in preparing each node in your cluster.
  - Installing with the MapR Installer
    The MapR Installer automates the process of installing MapR software and offers you a variety of options to complete the installation.
  - Installing without the MapR Installer
    Describes how to install MapR software and ecosystem components manually.
    - Step 1: Install the MapR Package Key
      Before you install MapR packages, you must install the MapR package key.
    - Step 2: Prepare Packages and Repositories
      In order to correctly install services, each node must have access to the package files.
    - Step 3: Install MapR Service Packages
      The installation process will vary based on the location of your packages and the configuration of your cluster.
    - Step 4: Verify Installation Success
      To confirm success, check each node.
    - Step 5: Set Environment Variables
      Before starting ZooKeeper or Warden, you must complete this step.
    - Step 6: Configure Nodes
      Connect nodes to the cluster, configure security, and arrange node storage.
    - Step 7: Bring up the Cluster
      Before you can install metrics monitoring, log monitoring, or ecosystem components, you must enable the cluster by starting ZooKeeper and Warden and verifying the cluster installation status.
    - Step 8: Install Metrics Monitoring
      To provide metrics monitoring, the MapR Control System (MCS) requires the collectd and OpenTSDB components to be installed. Metrics monitoring is part of MapR Monitoring, which also includes log monitoring. MapR Monitoring components are available as part of the MapR Ecosystem Pack (MEP) that you selected for the cluster.
    - Step 9: Install Log Monitoring
      Installing the logging components of MapR Monitoring is optional. The logging components enable the collection, storage, and visualization of MapR Core logs, system logs, and ecosystem-component logs. MapR Monitoring components are available as part of the MapR Ecosystem Pack (MEP) that you selected for the cluster.
    - Step 10: Install Ecosystem Components Manually
      You can install one or more ecosystem components from any MapR Ecosystem Pack (MEP) that is supported by the MapR cluster version. A MEP consists of a group of ecosystem components that work together.
      - Installing AsyncHBase Libraries
        This topic includes instructions for using package managers to download and install AsyncHBase from the MEP repository.
      - Installing Drill
        This topic provides instructions for using package managers to download and install Drill.
      - Installing Flume
        This topic includes instructions for using package managers to download and install Flume from the MEP repository.
      - Installing HBase Client and Tools
        MapR 6.0.x and MapR 6.1 provide Apache HBase-compatible APIs and client interfaces but do not support HBase as an ecosystem component. This topic describes the HBase Client and other tools that are available for use with MapR-DB.
      - Installing Hive
        This topic includes instructions for using package managers to download and install Hive from the MEP repository.
      - Installing HttpFS
        This topic includes instructions for using package managers to download and install HttpFS from the MEP repository.
      - Installing Hue
        This topic includes instructions for using package managers to download and install Hue from the MEP repository.
      - Impala Installation Overview
      - Installing MapR-ES Clients
        This topic includes instructions for using package managers to download and install MapR-ES Clients from the MEP repository.
      - Installing MapR-ES Tools
        This topic includes instructions for using package managers to download and install MapR-ES Tools (such as Kafka REST Proxy and Kafka Connect for MapR-ES) from the MEP repository.
      - Installing MapR-FS Native Driver for Manila
      - Installing MapR Data Access Gateway
        This topic includes instructions for using package managers to download and install the MapR Data Access Gateway from the MEP repository.
      - Installing Myriad
        Describes Myriad requirements and installation.
      - Installing Oozie
        This topic includes instructions for using package managers to download and install Oozie from the MEP repository.
      - Installing Pig
        This topic includes instructions for using package managers to download and install Pig from the MEP repository.
      - Installing Sentry
        This topic includes instructions for using package managers to download and install Sentry from the MEP repository.
      - Installing Spark Standalone
        This topic includes instructions for using package managers to download and install Spark Standalone from the MEP repository.
      - Installing Spark on YARN
        This topic includes instructions for using package managers to download and install Spark on YARN from the MEP repository.
      - Installing Spark on Mesos
        This section includes instructions to download and install Apache Spark on Apache Mesos.
      - Installing Sqoop1
        This topic includes instructions for using package managers to download and install Sqoop1 from the MEP repository.
      - Installing Sqoop2
        This topic includes instructions for using package managers to download and install Sqoop2 from the MEP repository.
    - Step 11: Run configure.sh
      Run configure.sh with the -R option to complete the configuration of ecosystem components that were added manually.
  - Configuring the Cluster
    This section contains information on configuring MapR clusters.
- Installing the File Migration Service on the Edge Cluster
- Installing MapR-XD
  Describes how to install MapR-XD software with or without the MapR Installer.
- Installing MapR Data Fabric for Kubernetes
  This section describes how to plan for, install, and upgrade the MapR Container Storage Interface (CSI) Storage Plugin and MapR Data Fabric for Kubernetes FlexVolume Driver.
- Installing MapR in the Cloud
  Provides an overview of MapR cloud installer capabilities.
- Upgrading MapR Core or MEP Components
  Depending on your current configuration, you may choose to upgrade the MapR version (MapR Core), ecosystem components, clients, or MapR Monitoring components.
- Setting Up Clients
  Describes how to set up and use interfaces to a MapR cluster from a client computer.
- Setting Up the MapR Control System
  Describes how to configure and access the MapR Control System (MCS).
- Migrating to MapR
  Provides instructions for migrating business-critical data and applications from an Apache Hadoop cluster to a MapR cluster.
- Applying a Patch
  You can apply a patch by using the MapR Installer, by using the command line (a manual process), or by using a MapR Installer Stanza.

Installing Spark on YARN

This topic includes instructions for using package managers to download and install Spark on YARN from the MEP repository.

Prerequisites

For instructions on setting up the MEP repository, see Step 10: Install Ecosystem Components Manually.

About this task

Spark is distributed as three separate packages:

Package	Description
mapr-spark	Install this package on each node where you want to install Spark. This package is dependent on the mapr-client package.
mapr-spark-historyserver	Install this optional package on Spark History Server nodes. This package is dependent on the mapr-spark and mapr-core packages.
mapr-spark-thriftserver	Install this optional package on Spark Thrift Server nodes. This package is available starting in the MEP 4.0 release. It is dependent on the mapr‍-‍spark and mapr‍-‍core packages.

To install Spark on YARN (Hadoop 2), execute the following commands as root or using sudo:

Procedure

Verify that JDK 1.7 or later is installed on the node where you want to install Spark.
Create the /apps/spark directory on MapR file system, and set the correct permissions on the directory:
```
hadoop fs -mkdir /apps/spark
hadoop fs -chmod 777 /apps/spark
```

Install the packages:

On Ubuntu

apt-get install mapr-spark mapr-spark-historyserver mapr-spark-thriftserver

On RedHat / CentOS

yum install mapr-spark mapr-spark-historyserver mapr-spark-thriftserver

On SUSE

zypper install mapr-spark mapr-spark-historyserver mapr-spark-thriftserver

NOTE: The mapr-spark-historyserver and mapr-spark-thriftserver packages are optional.

If you want to integrate Spark with MapR-ES, install the Streams Client on each Spark node:
- On Ubuntu:
```
 apt-get install mapr-kafka
```
- On RedHat/CentOS:
```
yum install mapr-kafka
```
If you want to use a Streaming Producer, add the spark-streaming-kafka-producer_2.11.jar from the MapR Maven repository to the Spark classpath (/opt/mapr/spark/spark-<versions>/jars/).
For repository-specific information, see Maven Artifacts for MapR
After installing Spark on YARN and before running your Spark jobs, follow the steps outlined at Configuring Spark on YARN.

(Topic last modified: 2019-02-28)