Read MapR Event Store For Apache Kafka Using a Kafka Source
IMPORTANT This component is deprecated. Hewlett Packard
Enterprise recommends using an alternate product. For more information, see Discontinued Ecosystem Components.
The following parameters are required when configuring Kafka source to read data from MapR
Streams topics:
Property Name | Description |
---|---|
channels | If the source writes events to a channel, this is the name of the channel that the source writes events to. |
type | This property must be set
to
|
kafka.consumer.group.id | A unique identifier of the consumer group. This defaults to
flume . Setting the same id in multiple sources or agents indicates that they
are part of the same consumer group; when these sources or agents are running at the
same time, they will each read a unique set of partitions for the topics in the group.
|
kafka.topics | A comma separated list of MapR Event Store For Apache Kafka topics where each topic is specified with the
volume path and stream name. For example:
NOTE It
is critical that the path to the topic starts with a slash(/), as the slash is what
distinguishes the topic as a MapR Event Store For Apache Kafka topic. |
batchDurationMillis | Maximum number of milliseconds to wait before closing a batch. The default value is 1000. |
For additional properties that you may want to configure, see the Flume
documentation. However, note that kafka.bootstrap.servers
is not
required for reading MapR Event Store For Apache Kafka.
Tip: To increase throughput, set the batchSize
to a higher value. The
batchSize
is the maximum number of messages written to Channel in one
batch. By default, it is set to 1000.