Explore multi-packet I/O (aka. fast UDP I/O)

[mxinden]

Summary

Explore multi-packet at (a) the I/O layer (sendmmsg, recvmmsg, GSO, GRO) and (b) within the Neqo state machines.

Very early draft

Have a fixed sized send and fixed sized receive buffer.

write: [Box<[u8; MAX_MTU * MAX_GSO_SEGMENTS]>; MAX_SENDMMSG_BATCH_SIZE],
read: [Box<[u8; MAX_MTU * MAX_GRO_SEGMENTS]>; MAX_RECVMMSG_BATCH_SIZE],

• can be used across multiple connections, e.g. one pair per process
• would this be too much per Firefox tab?

Have fn process take a set of input Datagram's and a set of to-be-written-to output Datagrams.

fn process<'a>(&mut self, input: impl Iterator<Item = &'a Datagram>, output: impl Iterator<Item = &'a mut Datagram>, now: Instant) -> Option<Duration>

• process datagrams in batches
• input Datagrams are a view into the read buffer and output Datagrams are a view into the write buffer thus no allocation
• we can as well wrap the iterators in separate types, offering stricter access methods to prevent misuse

Have Socket::send and Socket::recv take write and read buffer respectively.

Have Datagram carry a segment_size (taken from quinn-udp).

  #[derive(Clone, PartialEq, Eq)]
  pub struct Datagram {
      src: SocketAddr,
      dst: SocketAddr,
      tos: IpTos,
      ttl: Option<u8>,
      d: Vec<u8>,
+     /// The segment size if this transmission contains multiple datagrams.
+     /// This is `None` if the [`Datagram`] only contains a single datagram
+     segment_size: Option<usize>,
  }

• a single process call can write one or more (same size) datagrams for a single destination into a single Datagram

Next steps

• Write benchmarks
  • perf(bin): add criterion benchmarks #1758
  • More is always good
• Expose quinn-udps multi-packet APIs through neqo_common::udp
  • feat(udp): use sendmmsg and recvmmsg #1741
• Pass multiple packets to and accept from neqo-http3 / neqo-transport process method.
  • changing the function signature and simply iterating over the datagrams like process_multiple_input is easy, taking actual advantage of accessing the whole batch at once is hard

Past discussions

• Enhance socket read to handle multiple packets #1530
• Would close Try to avoid copying QuicDatagrams data  #1206.