MapR 5.2 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.2 Application Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
Ecosystem Components
The following sections provide information about each open source project that MapR supports.
Spark

MapR 5.2 Documentation

5.2 Application Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
- Connecting to the Cluster
  Application are often run on nodes that are not part of the MapR Cluster. There are many methods to connect to a MapR cluster; this section will briefly describe each option.
- MapR-FS and Applications
  The following sections provide information about accessing MapR-FS with C and Java applications.
- MapR-DB and Applications
  This section contains information on developing client applications for binary and JSON tables.
- MapR Streams and Applications
  MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
- MapReduce and Applications
  This section contains information associated with developing MapReduce programs or YARN applications.
- Ecosystem Components
  The following sections provide information about each open source project that MapR supports.
  - MapR Expansion Packs
    A MapR Expansion Pack (MEP) provides a set of ecosystem components that work together on one or more MapR cluster versions. Only one version of each ecosystem component is available in each MEP. For example, only one version of Hive and one version of Spark is supported in a MEP.
  - AsyncHBase
  - Cascading
  - Drill
  - Flume
  - HBase
  - HCatalog
  - Hive
  - HttpFS
  - Hue
  - Impala
  - Mahout
  - MapR Streams Clients and Tools
  - Myriad
  - Oozie
  - Pig
  - OpenStack Sahara (Ocata)
  - Sentry
  - Spark
    - Spark Feature Support
      MapR supports most Spark features. However, there a few exceptions.
    - Spark Standalone
    - Spark on YARN
    - Spark on Mesos
    - Spark SQL Thrift Server
    - MapR-DB Connectors for Apache Spark
    - Integrating Spark
    - Spark JDBC and ODBC Drivers
      MapR provides JDBC and ODBC drivers so you can write SQL queries that access the Apache Spark data processing engine. This section provides instructions on how to download the drivers, and install and configure them.
    - Spark API Changes
      This topic describes the public API changes that occurred for specific Spark versions.
    - Spark 2.0.0 Developer Preview
      Apache Spark is an open-source processing engine that you can use to process Hadoop data. If you would like to experiment with Spark 2.0.0, you can install and use Spark 2.0.0 on a non-secure MapR 5.2 cluster or on a secure MapR 5.2 cluster that uses MapR-SASL authentication.
  - Sqoop
  - Storm
  - Third Party Solutions
- Maven and MapR
  This section discusses topics associated with Maven and MapR.
- Developer's Reference
  This section contains in-depth information for the developer.
- API Documentation
  This topic provides links to the locations of MapR's public APIs. These APIs are available for application development purposes.

Spark

Apache Spark is an open-source processing engine. The following diagram shows the components involved in running Spark jobs. See Spark Cluster Mode Overview for further details on the different components.

MapR supports the following three types of cluster managers:

Spark's own standalone cluster manager
YARN
Apache Mesos

NOTE: Spark on Mesos is available starting in the MEP 3.0.1 Spark 2.1.0-1707 release.

The configuration and operational steps for Spark differ based on the Spark mode you choose to install. The steps to integrate Spark with other components are the same when using Standalone, YARN cluster mode, and Mesos cluster manager, except where otherwise noted.

This section provides documentation about configuring and using Spark with MapR, but it does not duplicate the Apache Spark documentation.

You can also refer to additional documentation available on the MapR Converge Community website.