Cooperative Sticky Rebalance Error: application *assign() call failed: Topic [3] is already part of the current assignment

[sjhr007]

Hi,

We use multiple static consumer instances in a consumer group to distribute tasks among available consumers using assigned partitions.

We moved to cooperative Sticky rebalance with the static consumer to minimize rebalance overhead.

In the rebalancing callback method, incrementalAssign() and incrementalUnassign() are called to assign or revoke partitions by application code.

After testing cooperative sticky rebalance by scaling down and then scaling up consumer instances, found the following error which occurs only for a few consumer instances and then consumers timeout after MAXPOLL.

1677722116.619|ASSIGN|App#consumer-2| [thrd:main]: Group "app-group": application *assign() call failed: Topic [3] is already part of the current assignment

1677722413.710|MAXPOLL|App#consumer-2| [thrd:main]: Application maximum poll interval (300000ms) exceeded by 97ms (adjust max.poll.interval.ms for long-running message processing): leaving group

Thanks,
Sachin