You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be nice if the embedded broker started by @EmbeddedKafka (or the similar JUnit Rule) defaulted the replication factor for the transaction state topic to the number of brokers, or a min(<number-of-brokers>, 3).
Current Behavior
The embedded broker starts with the default configuration of transaction.state.log.replication.factor = 3 which doesn't make sense for a single-broker cluster, which is quite common in tests:
org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
Context
It's not a big deal, as it can be resolved by explicitly setting broker properties like follows. But a meaningful default would save some time for thousands of developers googling the problem again and again.
@dlyash The tests we have in the framework for transactions that use EmbeddedKafka do exactly that - i.e., setting that property transaction.state.log.replication.factor to 1. Should we document that and not make any code changes primarily because retrieving the number of brokers consistently might be challenging? What do you think?
Note that, by default, transactions require a cluster of at least three brokers
which is the recommended setting for production; for development you can change this,
by adjusting broker setting transaction.state.log.replication.factor.
But I bet 99% of developers will only look for them once running into the issue and then will end up on the StackOverflow anyway.
As a developer, I would appreciate the testing framework taking care of it for me. That's why I was suggesting the programmatic solution.
.. retrieving the number of brokers consistently might be challenging
Oh, is it? I didn't dive too deep into the code, but I was imagining something like this in the EmbeddedKafkaContextCustomizer, but I may be simplifying things of course:
Expected Behavior
It would be nice if the embedded broker started by
@EmbeddedKafka
(or the similar JUnit Rule) defaulted the replication factor for the transaction state topic to the number of brokers, or amin(<number-of-brokers>, 3)
.Current Behavior
The embedded broker starts with the default configuration of
transaction.state.log.replication.factor = 3
which doesn't make sense for a single-broker cluster, which is quite common in tests:Context
It's not a big deal, as it can be resolved by explicitly setting broker properties like follows. But a meaningful default would save some time for thousands of developers googling the problem again and again.
The text was updated successfully, but these errors were encountered: