Volatile offset store #128
Conversation
This commit permits a projection consumer to optionally receive an offset store. Where it is not supplied, the source is requested to provided all of the events that it has.
akka-projection-rs/src/consumer.rs
Outdated
| envelope | ||
| } else { | ||
| envelope | ||
| } | ||
| } else { | ||
| envelope |
There was a problem hiding this comment.
Using no offset store together with remote gRPC projections will result in weird results, but I guess we don't have an easy way to fail fast for that "unsupported" combination? Maybe we should anyway fail here when it is a Offset::Timestamp with an error message saying that an offset store is required when using gRPC projections?
There was a problem hiding this comment.
Depends what we mean by fail... I want avoid panicking as the source can be from anywhere, including something remote. We don't want remote providers of events to be able to halt our process.
There was a problem hiding this comment.
Sorry, looking further... we can fail as we do for other conditions such as when there's a problem with the offset store. The existing failures were considered in the light of being transient, but I think I've already broken that rule.
There was a problem hiding this comment.
UPDATE: offset stores are always provided as before. I've introduced a new volatile_offset_store::task function to produce an in-memory store designed to always start from the beginning of a journal. The type of offset is immaterial.
This reverts commit ff93e31.
...and joins the offset store with the consumer for convenience. Also saves creating another task for the projection as these two things will always go hand-in-hand.
This commit permits a projection consumer to receive a volatile offset store for the purposes of always sourcing events from the start of a journal. This is used, for example, when sourcing events for an http connection where no start offset is provided.
The store can be used with all both sequence and timestamp offsets.
As an extra convenience, the tuple returned from the offset store task functions is now passed directly into the the projection consumer. The projection consumer will take care of running the offset store task and saves the caller from having to spawn it. Under the covers, the offset store task is joined with the projection consumer task, which are subsequently spawned as one.
Related: https://github.com/lightbend/akka-projection-temp/pull/23