Home
6.0 Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
Ecosystem Components
The following sections provide information about each open source project that MapR supports.
MapR-ES Clients and Tools
Kafka Connect 2.0.1 for MapR-ES
Kafka Connect for MapR-ES is a utility for streaming data between MapR-ES and Apache Kafka and other storage systems. This release of Kafka Connect is associated with MEP 2.x, 3.x, and 4.x.
Kafka Connect 2.0.1: HDFS Connector
These topics describe the Kafka Connect for MapR-ES HDFS connector, driver, and configuration parameters.

6.0 Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
- Application Development Process
  Before you start developing applications on MapR’s Converged Data Platform, consider how you will get the data onto the platform, the format it will be stored in, the type of processing or modeling that is required, and how the data will be accessed.
- MapR-FS and Apps
  The following sections provide information about accessing MapR-FS with C and Java applications.
- MapR-DB and Apps
  This section contains information about developing client applications for JSON and binary tables.
- MapR-ES and Apps
  MapR-ES brings integrated publish and subscribe messaging to the MapR Converged Data Platform.
- MapReduce and Apps
  This section contains information associated with developing YARN applications.
- MapR Data Science Refinery
  The MapR Data Science Refinery is an easy-to-deploy and scalable data science toolkit with native access to all platform assets and superior out-of-the-box security.
- MapR Data Fabric for Kubernetes FlexVolume Driver
  This section describes how to use and troubleshoot the MapR Data Fabric for Kubernetes FlexVolume Driver.
- Ecosystem Components
  The following sections provide information about each open source project that MapR supports.
  - MapR Ecosystem Packs
    A MapR Ecosystem Pack (MEP) provides a set of ecosystem components that work together on one or more MapR cluster versions. Only one version of each ecosystem component is available in each MEP. For example, only one version of Hive and one version of Spark is supported in a MEP.
  - AsyncHBase
  - Cascading
  - Drill
  - Flume
  - HBase Client and MapR-DB Binary Tables
  - HCatalog
  - Hive
  - HttpFS
  - Hue
  - Impala
  - MapR-ES Clients and Tools
    - Kafka REST Proxy 4.0.0
      The Kafka REST Proxy provides a RESTful interface to MapR-ES clusters to consume and produce messages and to perform administrative operations.
    - Kafka Connect 4.0.0
      Kafka Connect is a utility for streaming data between MapR-ES and other storage systems.
    - Kafka REST Proxy 2.0.1 for MapR-ES
      The Kafka REST Proxy 2.0.1 for MapR-ES provides a RESTful interface to MapR-ES clusters to consume and produce messages and to perform administrative operations. This release of Kafka REST Proxy 2.0.1 is associated with MEP 2.x, 3.x, and 4.x.
    - Kafka Connect 2.0.1 for MapR-ES
      Kafka Connect for MapR-ES is a utility for streaming data between MapR-ES and Apache Kafka and other storage systems. This release of Kafka Connect is associated with MEP 2.x, 3.x, and 4.x.
      - Kafka Connect 2.0.1: Architecture
        Kafka Connect for MapR-ES has the following major models in its design: connector, worker, and data.
      - Kafka Connect 2.0.1: Connectors, Tasks, and Workers
        This section describes how Kafka Connect for MapR-ES work and how connectors, tasks, offsets, and workers are associated wth each other.
      - Kafka Connect 2.0.1: Connector Configuration
        This section describes how and where connectors are configured.
      - Kafka Connect 2.0.1: Worker Configuration
        This section describes how and where to configure workers.
      - Kafka Connect 2.0.1: JDBC Connector
        The topics describes the JDBC connector, drivers, and configuration parameters.
      - Kafka Connect 2.0.1: Managing Kafka Connect Services
        Lists the commands you use to start, stop, or restart Kafka Connect Services
      - Kafka Connect 2.0.1: HDFS Connector
        These topics describe the Kafka Connect for MapR-ES HDFS connector, driver, and configuration parameters.
        Kafka Connect 2.0.1: HDFS Configuration Options
        Use the following parameters to configure the Kafka Connect for MapR-ES HDFS connector.
        Kafka Connect 2.0.1: HDFS Example: Publish to MapR-FS
        These examples provides sample code for streaming data to and from MapR-FS.
      - Kafka Connect 2.0.1: REST API
        The Kafka Connect REST API for MapR-ES manages connectors.
      - Kafka Connect 2.0.1: Hive Integration
        This topic describes how to integrate a Hive database with Kafka Connect for MapR-ES.
  - Myriad
  - OpenStack Manila
  - Oozie
  - Pig
  - Sentry
  - Spark
  - Sqoop
  - Third Party Solutions
- Maven and MapR
  This section discusses topics associated with Maven and MapR.
- Developer's Reference
  This section contains in-depth information for the developer.
- API Documentation
  MapR supports public APIs for MapR-FS, MapR-DB, and MapR-ES. These APIs are available for application development purposes.

Kafka Connect 2.0.1: HDFS Connector

These topics describe the Kafka Connect for MapR-ES HDFS connector, driver, and configuration parameters.

The HDFS connector allows you to export data from MapR-ES or Apache Kafka topics to MapR-FS or HDFS files in a variety of formats. In addition, Hive integration is available, which can be use to make data immediately available for querying with HiveQL.