Write to MapR Event Store For Apache Kafka using a Kafka Sink
IMPORTANT This component is deprecated. Hewlett Packard
Enterprise recommends using an alternate product. For more information, see Discontinued Ecosystem Components.
The following parameters are required when configuring Kafka sink to write to a MapR
Stream:
Property Name | Description |
---|---|
type | This property must be set to
|
kafka.topic | A topic specified with the volume path and stream name. For
example:
NOTE It is critical
that the path to the topic starts with a slash(/), as the slash is what distinguishes
the topic as a MapR Event Store For Apache Kafka topic. The default is "default-flume-topic". NOTE If the
header for a given message contains a "topic" field, the message will be published to
that topic instead of the topic that you configure here. |
For additional properties that you may want to configure for the Kafka sink, see the Flume
documentation.
NOTE
kafka.flumeBatchSize
sets the number of messages to process in a batch and
the default is 100. To ensure that the events are written in the original order, set
kafka.flumeBatchSize=1
and specify a topic with a single partition. If
maintaining the order is not required, you can increase the value. However, note that larger
batches improve throughput while also adding latency.