6.0 Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
- Application Development Process
  Before you start developing applications on MapR’s Converged Data Platform, consider how you will get the data onto the platform, the format it will be stored in, the type of processing or modeling that is required, and how the data will be accessed.
- MapR-FS and Apps
  The following sections provide information about accessing MapR-FS with C and Java applications.
- MapR-DB and Apps
  This section contains information about developing client applications for JSON and binary tables.
- MapR-ES and Apps
  MapR-ES brings integrated publish and subscribe messaging to the MapR Converged Data Platform.
- MapReduce and Apps
  This section contains information associated with developing YARN applications.
- MapR Data Science Refinery
  The MapR Data Science Refinery is an easy-to-deploy and scalable data science toolkit with native access to all platform assets and superior out-of-the-box security.
- MapR Data Fabric for Kubernetes FlexVolume Driver
  This section describes how to use and troubleshoot the MapR Data Fabric for Kubernetes FlexVolume Driver.
- Ecosystem Components
  The following sections provide information about each open source project that MapR supports.
- Maven and MapR
  This section discusses topics associated with Maven and MapR.
- Developer's Reference
  This section contains in-depth information for the developer.
  - MapR-DB Shell (JSON Tables)
    The mapr dbshell is a tool that enables you to create and perform basic manipulation of JSON tables and documents. You run dbshell by typing mapr dbshell on the command line after logging into a node in a MapR cluster.
  - Utilities for MapR-DB JSON Tables
    MapR-DB JSON provides utilities to copy, export, and import data, compare table content, and verify the consistency of secondary indexes.
  - MapR-DB HBase Shell (Binary Tables)
    You can manage MapR-DB tables using HBase shell commands and additional HBase shell commands included in the MapR distribution of Hadoop.
  - Utilities for MapR-DB Binary Tables
    MapR-DB provides utilities to copy and compare data in MapR-DB binary tables.
  - MapR-ES Utilities
    - mapr copystream
      This utility copies data from one MapR stream to another MapR stream. You can use it, for example, if you want to set up replication manually from one stream to another.
    - mapr diffstreams
      This utility compares the message IDs, metadata, and data in two MapR streams. Then, generates two directories that contain sequence files that you can use to merge the rows from the two MapR streams.
    - mapr diffstreamswithcrc
      This utility uses a cyclic redundancy check to detect differences between sets of messages in the specified MapR streams. Then, for each set of non-identical messages, it performs a detailed comparison. Finally, it generates one or more directories of sequence files. You can use these files either to make a MapR stream identical to its master or merge the messages from two MapR streams.
    - mapr exportstream and mapr importstream
      Use these utilities together to export data from MapR streams into binary sequence files, and then import the data from the binary sequence files into other MapR streams. You can also use the mapr importstream utility to import changes that are specified in sequence files output by the mapr diffstreams utility.
    - mapr perfconsumer
      This utility runs a consumer reading messages from topics in a MapR stream. Use this utility to run consumers when you want to estimate the performance of consumers for your MapR-ES applications, given your network configuration.
    - mapr perfproducer
      This utility runs a producer, generating messages and publishing them to a MapR stream. Use this utility to run producers when you want to estimate the performance of producers for your MapR-ES applications, given your network configuration.
    - mapr streamanalyzer
      This light-weight utility, which is a sample application for the Streams Java class for analytics on MapR streams, lets you count the messages in a stream or a subset of the topics in a stream. The utility also lets you print either whole retrieved messages or a subset of the fields in each message.
  - YARN Commands
    This section describes the YARN commands.
  - Source Code for MapR Software
    MapR releases source code to the open-source community for enhancements that MapR has made to the Apache Hadoop project and other ecosystem components.
  - Hadoop Commands
    This section describes the Hadoop commands.
- API Documentation
  MapR supports public APIs for MapR-FS, MapR-DB, and MapR-ES. These APIs are available for application development purposes.

MapR-ES Utilities

You can use the following utilities to with MapR-ES streams: