Unix datagram socket and FIONREAD

[andrewbaxter]

Right now UnixDatagram sockets are only really usable for packets with a reasonable known maximum size since the data size can't be known before read. It looks like the FIONREAD ioctl can be used to get the packet size, which with readable() would make this possible.

I googled around and haven't found many examples of this usage, so I was wondering if there was some reason it's not more common (aside from needing an ioctl). For IPC it'd be be an order of magnitude easier than managing connections and doing the framing manually with a stream socket.

I was thinking compatibility, but Unix sockets are already unix-only, or else some sort of performance issue...

[Darksonn]

Generally people just use a known maximum.

[Darksonn]

Yes, we want to match stdlib where it makes sense. If we add an API first, then stdlib may add it later with a different design. We wish to avoid that.

You can read about the ACP (API Change Proposal) process here.

[sfackler]

You could also look into adding it to the socket2 crate, where things will probably move faster than the standard library. That can freely interoperate with Tokio's types via the socket2::SockRef type.

[andrewbaxter]

Ah thanks for the pointers! I must have been really looking in the wrong place... I'll try ACP first and if that doesn't take I'll bother the socket2 people.

[andrewbaxter]

Ah... the readable() method is unique to Tokio I think, so the design would necessarily be a bit different for Tokio than for the stdlib.

In Tokio(/mio) (very roughly) you poll on the socket, then when notified it's likely FIONREAD would return useful data, you'd allocate the buffer, and then you'd read the message.

In stdlib, I think you just do a blocking read, so there's no real opportunity for FIONREAD. You could maybe peek with a minimum-sized buffer, then do FIONREAD, then read the whole thing, but overall I think it's more awkward and may be a harder sell.

I'll still do the ACP and see where it goes.

[andrewbaxter]

Okay sorry for liveblogging, but I think this probably won't work. I was really hoping to avoid inventing a new framing protocol for a unix stream for IPC, but I think that might be the only real solution.

Reading more, the unix datagram socket packet size is limited by available kernel buffers and won't use arbitrary amounts of memory, and messages that exceed it will be rejected, so it'd possibly be unreliable for large messages. It's really designed for small messages.

AFAICT linux net.core.{r,w}mem_max controls the kernel buffers... either setting that high would cause the same memory waste issue or it'd fail to allocate the buffer and maybe reject the packet.

Thanks for the direction here, it's good to know about ACP and socket2 unrelated to this!