Home
6.0 Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
Ecosystem Components
The following sections provide information about each open source project that MapR supports.
MapR-ES Clients and Tools
Kafka REST Proxy 2.0.1 for MapR-ES
The Kafka REST Proxy 2.0.1 for MapR-ES provides a RESTful interface to MapR-ES clusters to consume and produce messages and to perform administrative operations. This release of Kafka REST Proxy 2.0.1 is associated with MEP 2.x, 3.x, and 4.x.
Kafka REST Proxy 2.0.1
This section provides a HTTP method and URI summary of the Kafka REST Proxy for MapR-ES.
Kafka REST 2.0.1: GET /topics/{topic: string}/partitions/{partition_id: string}/messages?offset={int}[&count={int}]
Consumes messages from a partition of a topic.

MapR 6.0 Documentation

6.0 Development
This section contains information related to application development for ecosystem components and MapR products including MapR-DB (binary and JSON), MapR-FS, and MapR Streams.
- Application Development Process
  Before you start developing applications on MapR’s Converged Data Platform, consider how you will get the data onto the platform, the format it will be stored in, the type of processing or modeling that is required, and how the data will be accessed.
- MapR-FS and Apps
  The following sections provide information about accessing MapR-FS with C and Java applications.
- MapR-DB and Apps
  This section contains information about developing client applications for JSON and binary tables.
- MapR-ES and Apps
  MapR-ES brings integrated publish and subscribe messaging to the MapR Converged Data Platform.
- MapReduce and Apps
  This section contains information associated with developing YARN applications.
- MapR Data Science Refinery
  The MapR Data Science Refinery is an easy-to-deploy and scalable data science toolkit with native access to all platform assets and superior out-of-the-box security.
- MapR Data Fabric for Kubernetes FlexVolume Driver
  This section describes how to use and troubleshoot the MapR Data Fabric for Kubernetes FlexVolume Driver.
- Ecosystem Components
  The following sections provide information about each open source project that MapR supports.
  - MapR Ecosystem Packs
    A MapR Ecosystem Pack (MEP) provides a set of ecosystem components that work together on one or more MapR cluster versions. Only one version of each ecosystem component is available in each MEP. For example, only one version of Hive and one version of Spark is supported in a MEP.
  - AsyncHBase
  - Cascading
  - Drill
  - Flume
  - HBase Client and MapR-DB Binary Tables
  - HCatalog
  - Hive
  - HttpFS
  - Hue
  - Impala
  - MapR-ES Clients and Tools
    - Kafka REST Proxy 4.0.0
      The Kafka REST Proxy provides a RESTful interface to MapR-ES clusters to consume and produce messages and to perform administrative operations.
    - Kafka Connect 4.0.0
      Kafka Connect is a utility for streaming data between MapR-ES and other storage systems.
    - Kafka REST Proxy 2.0.1 for MapR-ES
      The Kafka REST Proxy 2.0.1 for MapR-ES provides a RESTful interface to MapR-ES clusters to consume and produce messages and to perform administrative operations. This release of Kafka REST Proxy 2.0.1 is associated with MEP 2.x, 3.x, and 4.x.
      - Kafka REST Proxy 2.0.1 Configuration
        This section provides information for configuring the Kafka REST Proxy for MapR-ES.
      - Kafka REST 2.0.1: Manage Services
        The Kafka REST Proxy for MapR-ES service can be started, restarted, and stopped via the maprcli nodes services command or using the REST API equivalent.
      - Kafka REST Proxy 2.0.1
        This section provides a HTTP method and URI summary of the Kafka REST Proxy for MapR-ES.
        Kafka REST 2.0.1: GET /topics
        Retrieves a list of topic names.
        Kafka REST 2.0.1: GET /topics/{topic:string}
        Retrieves metadata about a specific topic.
        Kafka REST 2.0.1: POST /topics/{topic:string}
        Produces messages into a topic with the following behavior.
        Kafka REST 2.0.1: GET /topics/{topic:string}/partitions
        Retrieves a list of partitions for the topic.
        Kafka REST 2.0.1: GET /topics/{topic: string}/partitions/{partition_id: string}
        Retrieves metadata about a specific partition within a topic.
        Kafka REST 2.0.1: POST /topics/{topic: string}/partitions/{partition_id: string}
        Produces messages into a partition of a topic.
        Kafka REST 2.0.1: GET /topics/{topic: string}/partitions/{partition_id: string}/messages?offset={int}[&count={int}]
        Consumes messages from a partition of a topic.
        Kafka REST 2.0.1: POST /consumers/{group: string}
        Creates a new consumer instance in the consumer group.
        Kafka REST 2.0.1: POST /consumers/{group: string}/instances/{instance: string}/offsets
        Commit offsets for the consumer. Returns a list of the partitions with the committed offsets.
        Kafka REST 2.0.1: DELETE /consumers/{group: string}/instances/{instance: string}
        Destroys the consumer instance.
        Kafka REST 2.0.1: GET /consumers/{group: string}/instances/{instance: string}/topics/{topic: string}
        Consumes messages by a specific consumer from a topic .
        Kafka REST 2.0.1: GET /streams/{stream: string}/topics
        Retrieves a list of topics in a given stream.
    - Kafka Connect 2.0.1 for MapR-ES
      Kafka Connect for MapR-ES is a utility for streaming data between MapR-ES and Apache Kafka and other storage systems. This release of Kafka Connect is associated with MEP 2.x, 3.x, and 4.x.
  - Myriad
  - OpenStack Manila
  - Oozie
  - Pig
  - Sentry
  - Spark
  - Sqoop
  - Third Party Solutions
- Maven and MapR
  This section discusses topics associated with Maven and MapR.
- Developer's Reference
  This section contains in-depth information for the developer.
- API Documentation
  MapR supports public APIs for MapR-FS, MapR-DB, and MapR-ES. These APIs are available for application development purposes.

Kafka REST 2.0.1: GET /topics/{topic: string}/partitions/{partition_id: string}/messages?offset={int}[&count={int}]

Consumes messages from a partition of a topic.

Description

Depending on the configuration, the type of information retrieved has different behavior. See the streams.default.stream parameter in Kafka REST 2.0.1: Configuration Parameters.

Table 1. Response Behavior
Parameters Defined	Response
streams.default.stream is defined	Consumes messages from a partition of a MapR-ES topic. The user could pass fully qualified topic name or not. If a fully qualified topic name is not used, messages are consumed from a partition of topic in the default stream path.
streams.default.stream is not defined	Consumes messages from partition of MapR-ES topic. The user could only pass a fully qualified topic name that contains a stream path.

Table 2. Parameters
Parameters	Description
topic_name (string)	Topic to consume the messages from.
partition_id (int)	Partition to consume the messages from.
offset (int)	Offset to start from.
count (int)	Number of messages to consume (Optional). Default: 1

Syntax

http://<host>:8082/topics/<topic_name>/partitions/<partition_id>/messages?offset=<integer>[&count=<int>]

Request Example

curl -X GET "http://localhost:8082/topics/testtopic1/partitions/0/messages?offset=2&count=1"

Response Example

[{
  "key":"a2v5",
  "value":
  "Y29uZmx1ZW50",
  "topic":"/streaming_data/stream:testtopic1",
  "partition":0,
  "offset":2
}]