Package org.apache.kafka.streams.processor.internals

Class StreamsRebalanceListener

java.lang.Object

org.apache.kafka.streams.processor.internals.StreamsRebalanceListener

All Implemented Interfaces:

ConsumerRebalanceListener

public class StreamsRebalanceListener
extends Object
implements ConsumerRebalanceListener

Method Summary

Modifier and Type	Method	Description
void	onPartitionsAssigned(Collection<TopicPartition> partitions)	A callback method the user can implement to provide handling of customized offsets on completion of a successful partition re-assignment.
void	onPartitionsLost(Collection<TopicPartition> partitions)	A callback method you can implement to provide handling of cleaning up resources for partitions that have already been reassigned to other consumers.
void	onPartitionsRevoked(Collection<TopicPartition> partitions)	A callback method the user can implement to provide handling of offset commits to a customized store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

onPartitionsAssigned

public void onPartitionsAssigned(Collection<TopicPartition> partitions)

Description copied from interface: ConsumerRebalanceListener

A callback method the user can implement to provide handling of customized offsets on completion of a successful partition re-assignment. This method will be called after the partition re-assignment completes and before the consumer starts fetching data, and only as the result of a poll(long) call.

It is guaranteed that under normal conditions all the processes in a consumer group will execute their ConsumerRebalanceListener.onPartitionsRevoked(Collection) callback before any instance executes its ConsumerRebalanceListener.onPartitionsAssigned(Collection) callback. During exceptional scenarios, partitions may be migrated without the old owner being notified (i.e. their ConsumerRebalanceListener.onPartitionsRevoked(Collection) callback not triggered), and later when the old owner consumer realized this event, the ConsumerRebalanceListener.onPartitionsLost(Collection) (Collection)} callback will be triggered by the consumer then.

It is common for the assignment callback to use the consumer instance in order to query offsets. It is possible for a WakeupException or InterruptException to be raised from one of these nested invocations. In this case, the exception will be propagated to the current invocation of KafkaConsumer.poll(java.time.Duration) in which this callback is being executed. This means it is not necessary to catch these exceptions and re-attempt to wakeup or interrupt the consumer thread.

Specified by:

onPartitionsAssigned in interface ConsumerRebalanceListener

Parameters:

partitions - The list of partitions that are now assigned to the consumer (previously owned partitions will NOT be included, i.e. this list will only include newly added partitions)

onPartitionsRevoked

public void onPartitionsRevoked(Collection<TopicPartition> partitions)

Description copied from interface: ConsumerRebalanceListener

A callback method the user can implement to provide handling of offset commits to a customized store. This method will be called during a rebalance operation when the consumer has to give up some partitions. It can also be called when consumer is being closed (KafkaConsumer.close(Duration)) or is unsubscribing (KafkaConsumer.unsubscribe()). It is recommended that offsets should be committed in this callback to either Kafka or a custom offset store to prevent duplicate data.

In eager rebalancing, it will always be called at the start of a rebalance and after the consumer stops fetching data. In cooperative rebalancing, it will be called at the end of a rebalance on the set of partitions being revoked iff the set is non-empty. For examples on usage of this API, see Usage Examples section of KafkaConsumer.

It is common for the revocation callback to use the consumer instance in order to commit offsets. It is possible for a WakeupException or InterruptException to be raised from one of these nested invocations. In this case, the exception will be propagated to the current invocation of KafkaConsumer.poll(java.time.Duration) in which this callback is being executed. This means it is not necessary to catch these exceptions and re-attempt to wakeup or interrupt the consumer thread.

Specified by:

onPartitionsRevoked in interface ConsumerRebalanceListener

Parameters:

partitions - The list of partitions that were assigned to the consumer and now need to be revoked (may not include all currently assigned partitions, i.e. there may still be some partitions left)

onPartitionsLost

public void onPartitionsLost(Collection<TopicPartition> partitions)

Description copied from interface: ConsumerRebalanceListener

A callback method you can implement to provide handling of cleaning up resources for partitions that have already been reassigned to other consumers. This method will not be called during normal execution as the owned partitions would first be revoked by calling the ConsumerRebalanceListener.onPartitionsRevoked(java.util.Collection<org.apache.kafka.common.TopicPartition>), before being reassigned to other consumers during a rebalance event. However, during exceptional scenarios when the consumer realized that it does not own this partition any longer, i.e. not revoked via a normal rebalance event, then this method would be invoked.

For example, this function is called if a consumer's session timeout has expired, or if a fatal error has been received indicating the consumer is no longer part of the group.

By default it will just trigger ConsumerRebalanceListener.onPartitionsRevoked(java.util.Collection<org.apache.kafka.common.TopicPartition>); for users who want to distinguish the handling logic of revoked partitions v.s. lost partitions, they can override the default implementation.

It is possible for a WakeupException or InterruptException to be raised from one of these nested invocations. In this case, the exception will be propagated to the current invocation of KafkaConsumer.poll(java.time.Duration) in which this callback is being executed. This means it is not necessary to catch these exceptions and re-attempt to wakeup or interrupt the consumer thread.

Specified by:

onPartitionsLost in interface ConsumerRebalanceListener

Parameters:

partitions - The list of partitions that were assigned to the consumer and now have been reassigned to other consumers. With the current protocol this will always include all of the consumer's previously assigned partitions, but this may change in future protocols (ie there would still be some partitions left)