Roger Blanchard (Osprey Retail Systems) asked a question.
I am wondering what this is used for; OpenEdge.Messaging.Kafka.KafkaConsumerBuilder SetAutoDeleteConsumerGroup (logical)
From reading the documentation you would think when the consumer that is running quits that the consumer group is deleted and the group offset would be reset. Maybe not, but that is how I read it but not how it works. I was just looking for some clarification.
/*Specify whether the consumer group should be automatically deleted when the consumer object isgarbage collected.
Its an OpenEdge specific property that does exactly what the name implies.
With the OpenEdge Kafka consumer, all consumers are added to a consumer group as it is needed for recovery, rollback, coordination, etc. We require you set the consumer group for consumers. For this consumer group, If you specify a group that does not yet exist, the group is automatically created by Kafka (if allowed by policy). Its a group of one consumer. Simply by specifying a consumer group name for the consumer, a consumer group is created by the broker.
When the consumer leaves the group, the group remains, along with its consumer offset information. If the consumer rejoins the group, the offset information is restored to the client, and the client can consume from those topics where it left off. No special steps are needed.
The broker will automatically clean these up eventually depending on policy if there is no consumer that rejoins that specific group within the alloted time period. Default I think is 1 week.
In a testing scenario, or a scenario where you don't have consumer groups pre setup, and you dynamically created the group identifier, this can leave a lot of "dead" groups on the broker. Its not hard to clean these up, and the broker will eventually clean them on its own, but its messy.
You can enable this, and OpenEdge will request the broker delete the group that was created when the Consumer was created. Its up to the broker to respect that request. There's a timing bug due to a changed API that was introduced in a Kafka client version after the feature was implemented so sometimes the OpenEdge consumer side is cleaned up before the delete group request is sent, when the Consumer is deleted by the ABL. So sometimes it doesn't get cleaned up, leaving an administrative task...which you would have had to do anyway.
In short, its a convenience feature, but maybe best left to managing more holistically.
Hey Matt,
Thanks for the quick response.
>> When the consumer leaves the group, the group remains, along with its consumer offset information. If the consumer rejoins the group, the offset information is restored to the client, and the client can consume from those topics where it left off. No special steps are needed.
That is what we see and want.
>> The broker will automatically clean these up eventually depending on policy if there is no consumer that rejoins that specific group within the alloted time period. Default I think is 1 week.
Yes, it is one week or at least in Confluent Cloud.
If we set this to true it sounds like the Consumer Group should get deleted before the 1 week. That is not what we see or want so it sounds like this should be set to false. We have had this set to true since ABL kafka support was added and have not had any issues. I guess we have been lucky. (SetAutoDeleteConsumerGroup (logical))