Skip to content

Commit

Permalink
doc: Mention mapAsyncPartitioned in context of at-least-once (#1630)
Browse files Browse the repository at this point in the history
This is arguably the canonical use-case for mapAsyncPartitioned.
  • Loading branch information
leviramsey authored Jun 9, 2023
1 parent 29dbb22 commit 8670cf9
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/src/main/paradox/atleastonce.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ However it can only lead to reordering between messages sent to different substr

If a particular substream expects to see all messages regarding some entity, it then requires that writers to the source topic become responsible for placing messages about various entities in the appropriate partitions. If your application already has a requirement to preserve the order of messages about a particular entity within a Kafka topic, you will already need to ensure those messages go to the same partition since Kafka only preserves order information within a partition.

Consider instead using `mapAsyncPartitioned` in place of a `groupBy` followed by `mergeSubstreams`. Both allow for demultiplexing an input stream, but `mapAsyncPartitioned` will not reorder output messages and also allows its partitions to be finer-grained than a Kafka partition. Note that partitioning in `mapAsyncPartitioned` only happens within that stage: complex processing of a partition may require techniques outside of streams, such as using the ask pattern to an actor.

## Conditional Message Processing

Most flows will require some messages to be handled differently from others. Unfortunately this is difficult to do while preserving the at-least-once guarantee because the order of messages must be maintained.
Expand Down

0 comments on commit 8670cf9

Please sign in to comment.