Inversion of control for serializers

[damooo]

Currently serializer apis ask us to provide impl Write, instead of returning serialized data as some stream. This makes it very difficult to adapt serializing api to unconventional environments (for a worker in tokio spawn_blocking, etc). If it gives some output, we can juggle it. But it instead expects a blocking writer.

[pchampin]

That's a good point, thanks for raising it. I confess I am not completely up to speed with the async part of the Rust ecosystem.

When you write "Stream", do you mean this?

If serializers returned a Stream, would there be an easy way to dump the content of that stream into a blocking Write? More specifically, would it be possible to keep the current API in addition to the new async API?

[damooo]

We can use few bridges like AllowStdIo, to wrap astd::io::write as an AsyncWrite. But it comes with caveats as it may block event loop.

If we use async, it is expected to use AsyncWrite instead.

Even if not async stream, if current serializer returns a fallible blocking iterator, we can adapt that into async code by spawning blocking steps in worker threads.

[pchampin]

There are actually two orthogonal questions in your question : 1) inversion of control, and 2) support for async.

1. Inversion of control would mean that instead of being constructed around a "sink", the serializer would act as a "source" of strings (or byte sequences). That "source" could be an Iterator (sync) or a Stream (async).
   While this would be perhaps more flexible, this would also force all serializers implementations to allocate a new buffer for each piece of information it produces (because it must yield ownership of that buffer to whoever is consuming it).
   The current design allows serializers to reuse buffer and therefore to be more memory efficient.

2. Support for async would be nice, as discussed above. As a partial answer, please notice that nothing in the TripleSerializer trait binds it to std::io::Write. It is technically possible to write an implementation of TripleSerializer that works with an AsyncWrite... but of course, the methods serialize_triples and serialize_graph should then use some form of blocking because they are nor async...

[damooo]

Yes. Main point was first. And async streams is the case, which benifits much from that, as adaptation will be simple enough.

[pchampin]

Therefore, I believe that item 1 in this reply is the answer to your question: the current flow of control is a design decision to favor memory efficiency.