Skip to content

Commit

Permalink
karapace: avoid starting until topic its successfully created
Browse files Browse the repository at this point in the history
Since the underlying `create_topics` function unless the `operation_timeout` its async (the effects of this call aren't immediately visible after the call its performed), if karapace its started on a cluster that its slow enough in creating the topic, we get an unhandled exception in the startup of the service of type:
```
aiokafka.errors.UnknownTopicOrPartitionError: [Error 3] UnknownTopicOrPartitionError
```

The fix its simple, try to query from the cluster (immediately after
having commanded the creation of the topic) the offsets of the topic.
If the returned status its `OFFSET_UNINITIALIZED` we simply need to try
again shortly later.
  • Loading branch information
eliax1996 committed Jun 20, 2024
1 parent 3f899ac commit e3c51fc
Showing 1 changed file with 6 additions and 5 deletions.
11 changes: 6 additions & 5 deletions karapace/schema_reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -240,11 +240,12 @@ def run(self) -> None:
# Handles also a unusual case of purged schemas topic where starting offset can be > 0
# and no records to process.
self.offset = self._get_beginning_offset()
try:
self.handle_messages()
except Exception as e: # pylint: disable=broad-except
self.stats.unexpected_exception(ex=e, where="schema_reader_loop")
LOG.exception("Unexpected exception in schema reader loop")
else:
try:
self.handle_messages()
except Exception as e: # pylint: disable=broad-except
self.stats.unexpected_exception(ex=e, where="schema_reader_loop")
LOG.exception("Unexpected exception in schema reader loop")

def _get_beginning_offset(self) -> int:
assert self.consumer is not None, "Thread must be started"
Expand Down

0 comments on commit e3c51fc

Please sign in to comment.