MapR Event Store For Apache Kafka Java API Library

Use the MapR Event Store For Apache Kafka Admin Java API library as an alternative to maprcli commands and the REST APIs for performing administrative tasks on streams and topics. This library can also be used for analysis of the contents of streams.

Javadoc

The following Apache Kafka Java API versions are supported:
Table 1. Supported Apache Kafka APIs
Core version Apache Kafka API
As of 6.1 1.1
As of 6.0.1 1.0
6.0.0 and earlier 0.90

Java Interfaces, Classes, and Enums

The MapR Event Store For Apache Kafka Java API library consists of the following interfaces and class:
Admin
Provides methods for performing administrative tasks on streams and topics, as well as for obtaining StreamDescriptor and TopicDescriptor objects.
StreamDescriptor
Provides methods for setting and retrieving values for stream attributes. StreamDescriptor is passed into methods when performing operations on streams, for example, creating a new stream and editing an existing stream. StreamDescriptor is also used to find attribute values for an existing stream.
TopicDescriptor
NOTE TopicDescriptor is new as of 6.0.1.
Provides methods for setting and retrieving topic values. TopicDescriptor is passed into methods when performing operations on topics, for example, setting and retrieving topic partitions and timestamp type. The available timestamp type can be either createtime or logappendtime.
When a producer writes a message to a MapR Event Store For Apache Kafka topic, a timestamp is included that is part of the message record. This timestamp can be used to implement time-based indexing. Time-based indexing enables consumers to find the offsets for messages based on timestamps. The returned message offset corresponds to the earliest topic-partition message whose timestamp is equal to or greater than the provided timestamp.
NOTE APIs that support timeout semantics will adhere to the specified timeouts only if the client is configured for soft mount.
Streams Class
Streams class is also required to create an instance of Admin, which is used for all admin operations on streams. It can also be used to create a StreamDescriptor or TopicDescriptor objects.

Provides the entry point to accessing MapR Event Store For Apache Kafka streams for analytics purposes.

TimestampType Enum
Provides the timestamp type of the records.
TopicRefreshListListener / TopicRefreshRegexListener
NOTE Internal; not for public usage.

MapR Event Store For Apache Kafka Java APIs (as of 6.1)

The following MapR Event Store For Apache Kafka Java APIs are available as of MapR Data Platform 6.1:

Table 2. MapR Event Store For Apache Kafka Java APIs applicable for MapR Data Platform 6.1:
Interface Method Description
StreamDescriptor void setCompact(boolean compact) Sets log compaction on a stream.
StreamDescriptor boolean getCompact() Gets the log compaction on a stream. Returns true if the stream has log compaction on the stream.
StreamDescriptor void setMinCompactionLagMS(long ts) Sets the time in (milliseconds) that a message should remain uncompacted in the topic-partition. Applies only if log compaction is enabled on the stream.
StreamDescriptor long getMinCompactionLagMS() Returns the minimum time (in milliseconds) a message will remain uncompacted in the topic-partition. Applies only if log compaction is enabled on the stream.
StreamDescriptor void setDeleteRetentionMS(long ts) Sets the time (in milliseconds) for which deleted records are retained. Applies only if log compaction is enabled on the stream.
StreamDescriptor long getDeleteRetentionMS() Returns the time (in milliseconds) for which deleted records are retained. Applies only if log compaction is enabled on the stream.
Producer ProducerConfig class The idempotence producer option is set by setting the enable.idempotence value of true passed through the ProducerConfig class.

MapR Event Store For Apache Kafka Java APIs (as of 6.0.1)

The following table lists the new Interfaces and APIs for MapR Data Platform 6.0.1. They are the delta between MapR Data Platform 6.0.1 and 6.0.0, meaning, they are applicable to MapR Data Platform6.0.1 but not MapR Data Platform 6.0.0.

Table 3. MapR Event Store For Apache Kafka APIs (as of 6.0.1)
Interface and Methods Description
Admin.close Long duration for TimeUnit.
Admin.createTopic TopicDescriptor array for topic attributes.
Admin.editTopic TopicDescriptor array for topic attributes.
Admin.getTopicDescriptor Method for retrieving topic attributes.
Admin.listTopic Method for listing all the topics in a stream.
Admin.streamExists Method for determining whether a stream exists.
StreamDescriptor.getDefaultTimestampType Method for retrieving the timestamp type.
StreamDescriptor.setDefaultTimestampType Method for setting the timestamp type.
TopicDescriptor New MapR Data Platform interface.
TopicDescriptor.getPartitions Method associated with the new interface.
TopicDescriptor.setPartitions Method associated with the new interface.
TopicDescriptor.getTimestampType Method associated with the new interface.
TopicDescriptor.setTimestampType Method associated with the new interface.
Enum TimestampType New Enum class and associated methods.

Backward Compatibility

As of MapR Data Platform 6.0.1, Apache Kafka 1.0 is supported. The following pause, resume, seekToBeginning, and seekToEnd APIs support the Collection Interface. The deprecated APIs will continue to run unchanged, however, they may be removed in a future release.
Table 4. Deprecated MapR Data Platform APIs (as of 6.0.1)
Replacement Collection APIs Deprecated APIs
void pause(Collection<TopicPartition> partitions); void pause(TopicPartition... partitions);
void resume(Collection<TopicPartition>partitions); void resume(TopicPartition... partitions);
void seekToBeginning(Collection<TopicPartition>); void seekToBeginning(TopicPartition... partitions);
void seekToEnd(Collection<TopicPartition>); void seekToEnd(TopicPartition... partitions);
The following subscribe and assign APIs support the Collection Interface (which is more generalized) as well as the List Interface. Support for the List Interface has been retained for backward binary compatibility.
Table 5. Retained MapR Data Platform APIs (as of 6.0.1)
Replacement Collection APIs Retained APIs
void subscribe(Collection<String> topics); void subscribe(java.util.List<java.lang.String> topics);
void subscribe(Collection<String> topics, ConsumerRebalanceListener); void subscribe(java.util.List<java.lang.String> topics, ConsumerRebalanceListener listener);
void assign(Collection<TopicPartition> partitions); void assign(java.util.List<TopicPartition> partitions);