[FIXED] How to remove a Kafka consumer group from a specific topic?

Issue

I changed the consumer-group-id of my web service listening to a Kafka topic. Now, the old group id is still registered to the topic, but there is no consumer with that group id. Therefore, it is lagging. How can I remove a specific consumer group from a specific topic?

I tried this:

kafka-consumer-groups --bootstrap-server kafka01.myserver.com:9092 --topic notification-topic --delete --group old-consumer-group --execute

But it returns: "The consumer does not support topic-specific offset deletion from a consumer group."

Should I remove the consumer group totally? I use the same group id listening to other topics, are they going to be affected?

Solution

Starting with Kafka 2.4.0 it is possible to delete individual consumer group id offsets from a topic.

The call is very close to what you have already tried, but requires --delete-offsets instead of --delete:

./kafka-consumer-groups.sh \
  --bootstrap-server <bootstrap-server-url> \
  --delete-offsets \
  --group <my-group> \
  --topic <topic-name>

To date this is not reflected in the official documentation (Kafka 2.7.0 and below). However, it is described in a confluence document of improvement proposal KIP-496 that has been implemented and released.

Answered By - summon

Answer Checked By - Marilyn (PHPFixing Volunteer)