Add some developer documentation

doc/dev-recv-chunk-group.rst

@@ -0,0 +1,49 @@
+Synchronisation in chunk stream groups
+======================================
+.. cpp:namespace-push:: spead2::recv
+
+For chunk stream groups to achieve the goal of allowing multi-core scaling, it
+is necessary to minimise locking. The implementation achieves this by avoiding
+any packet- or heap-granularity locking, and performing locking only at chunk
+granularity. Chunks are assumed to be large enough that this minimises total
+overhead, although it should be noted that these locks are expected to be
+highly contended and there may be further work possible to reduce the
+overheads.
+
+To avoid the need for heap-level locking, each member stream has its own
+sliding window with pointers to the chunks, so that heaps which fall inside an
+existing chunk can be serviced without locking. However, this causes a problem
+when flushing chunks from the group's window: a stream might still be writing
+to the chunk at the time. Additionally, it might not be possible to allocate a
+new chunk until an old chunk is flushed e.g., if there is a fixed pool of
+chunks rather than dynamic allocation.
+
+Each chunk has a reference count, indicating the number of streams that still
+have the chunk in their window. This reference count is non-atomic since it is
+protected by the group's mutex. When the group wishes to evict a chunk, it
+first needs to wait for the reference count of the head chunk to drop to zero.
+It needs a way to be notified that it should try again, which is provided by a
+condition variable. Using a condition variable (rather than, say, replacing
+the simple reference count with a semaphore) allows the group mutex to be
+dropped while waiting, which prevents the deadlocks that might otherwise occur
+if the mutex was held while waiting and another stream was attemping to lock
+the group mutex to make forward progress.
+
+In lossless eviction mode, this is all that is needed, although it is
+non-trivial to see that this won't deadlock with all the streams sitting in
+the wait loop waiting for other streams to make forward progress. That this
+cannot happen is due to the requirement that the stream's window cannot be
+larger than the group's. Consider the active call to
+:cpp:func:`chunk_stream_group::get_chunk` with the smallest chunk ID. That
+stream is guaranteed to have already readied any chunk due to be evicted from
+the group, and the same is true of any other stream that is waiting in
+:cpp:func:`~chunk_stream_group::get_chunk`, and so forward progress depends
+only on streams that are not blocked in
+:cpp:func:`~chunk_stream_group::get_chunk`.
+
+In lossy eviction mode, we need to make sure that such streams make forward
+progress even if no new packets arrive on them. This is achieved by posting an
+asynchronous callback to all streams requesting them to flush out chunks that
+are now too old.
+
+.. cpp:namespace-pop::

doc/dev-recv-destruction.rst

@@ -0,0 +1,54 @@
+Destruction of receive streams
+==============================
+The asynchronous and parallel nature of spead2 makes destroying a receive
+stream a tricky operation: there may be pending asio completion handlers that
+will try to push packets into the stream, leading to a race condition. While
+asio guarantees that closing a socket will cancel any pending asynchronous
+operations on that socket, this doesn't account for cases where the operation
+has already completed but the completion handler is either pending or is
+currently running.
+
+Up to version 3.11, this was handled by a shutdown protocol
+between :cpp:class:`spead2::recv::stream` and
+:cpp:class:`spead2::recv::reader`. The reader was required to notify the
+stream when it had completely shut down, and
+:cpp:func:`spead2::recv::stream::stop` would block until all readers had
+performed this notification (via a semaphore). This protocol was complicated,
+and it relied on the reader being able to make forward progress while the
+thread calling :cpp:func:`~spead2::recv::stream::stop` was blocked.
+
+Newer versions take a different approach based on shared pointers. The ideal
+case would be to have the whole stream always managed by a shared pointer, so
+that a completion handler that interfaces with the stream could keep a copy of
+the shared pointer and thus keep it alive as long as needed. However, that is
+not possible to do in a backwards-compatible way. Instead, a minimal set of
+fields is placed inside a shared pointer, namely:
+
+- The ``queue_mutex``
+- A flag indicating whether the stream has stopped.
+
+For convenience, the flag is encoded as a pointer, which holds either a
+pointer to the stream (if not stopped) or a null pointer (if stopped). Each
+completion handler holds a shared reference to this structure. When it wishes
+to access the stream, it should:
+
+1. Lock the mutex.
+2. Get the pointer back to the stream from the shared structure, aborting if
+   it gets a null pointer.
+3. Manipulate the stream.
+4. Drop the mutex.
+
+This prevents use-after-free errors because the stream cannot be destroyed
+without first stopping, and stopping locks the mutex. Hence, the stream cannot
+disappear asynchronously during step 3. Note that it can, however, stop
+during step 3 if the completion handler causes it to stop.
+
+Using shared pointers in this way can add overhead because atomically
+incrementing and decrementing reference counts can be expensive, particularly
+if it causes cache line migrations between processor cores. To minimise
+reference count manipulation, the :cpp:class:`~spead2::recv::reader` class
+encapsulates this workflow in its
+:cpp:class:`~spead2::recv::reader::bind_handler` member function, which
+provides the facilities to move the shared pointer along a linear chain of
+completion handlers so that the reference count does not need to be
+adjusted.

doc/developer.rst

@@ -0,0 +1,12 @@
+Developer documentation
+=======================
+
+This section documents internal design decisions that users will generally not
+need to be aware of, although some of it may be useful if you plan to subclass
+the C++ classes to extend functionality.
+
+.. toctree::
+   :maxdepth: 2
+
+   dev-recv-destruction
+   dev-recv-chunk-group

doc/index.rst

@@ -18,6 +18,7 @@ Contents:
    perf
    tools
    migrate-3
+   developer
    changelog
    license