-
I have read through the documentation but still do not fully understand the differences between stream and stateless decoding. Could you provide a reference document or an explanation? As I understand that when decoding in a stream mode, only one stream can be decoded at a time. When requesting a zstd-encoded HTTP page, what are the differences between decoding it using stream mode and receiving the stream completely and decoding buffer in stateless mode? thank you in advance |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
The main difference is the memory requirements, but also some speed considerations. The []byte -> []byte requires you to have both the input and the output in memory at the same time. If your payload is a 1GB JSON file, this is probably not a great idea. Secondly []byte -> []byte means that every step must complete before the next can start. This means your flow will be Request -> io.ReadAll -> DecodeAll -> Process serial flow. With a stream, as soon as the first block is returned by the remote and has been compressed it is ready for your processor. However streams require more setup, so typically for smaller payloads it is not worth setting it up, for effectively a single block (up to 128KB) of data will provide none of these benefits. |
Beta Was this translation helpful? Give feedback.
The main difference is the memory requirements, but also some speed considerations.
The []byte -> []byte requires you to have both the input and the output in memory at the same time. If your payload is a 1GB JSON file, this is probably not a great idea.
Secondly []byte -> []byte means that every step must complete before the next can start. This means your flow will be Request -> io.ReadAll -> DecodeAll -> Process serial flow. With a stream, as soon as the first block is returned by the remote and has been compressed it is ready for your processor.
However streams require more setup, so typically for smaller payloads it is not worth setting it up, for effectively a single block (up to 12…