Turning "normal" FD into a fixed/direct one

[cmazakas]

I tried looking through the docs and the test suite but I wasn't really able to find something that I can use.

I'm wondering, is it possible to take a normal FD and then register it with io_uring?

Right now, I have an application which registers a sparse file table with the ring and then I use the correct indices for io_uring_prep_socket_direct() and such. But now I'm looking at extending the code to work with user-provided FDs which aren't ring-private.

I feel like I should be using io_uring_register_files_update for this but I'm exactly sure how.

Is there an example or test I can look at?

[axboe]

Yes, you want to use io_uring_register_files_update() for that case. It basically just replaces an existing fixed file slot with whatever you provide. If the slot is currently empty (eg holds -1, sparse), then the sparse entry is replaced with the file you specify and put into the given offset.

There are really two ways to turn a normal file descriptor into a regular one - either register it like that in the first place, eg when you register the table, or do it via io_uring_register_files_update.

The test cases have some usage of it, here's a bare bones example of registering a sparse table and then later putting a pipe fd in there:

int main(int argc, char *argv[])
{
	struct io_uring_sqe *sqe;
	struct io_uring_cqe *cqe;
	struct io_uring ring;
	int fds[2], fd;
	char buf[32], buf2[32];
	int ret;

	io_uring_queue_init(32, &ring, IORING_SETUP_DEFER_TASKRUN | IORING_SETUP_SINGLE_ISSUER);
	fd = -1;
	ret = io_uring_register_files(&ring, &fd, 1);

	if (pipe(fds) < 0) {
		perror("pipe");
		return 1;
	}

	fd = fds[0];
	io_uring_register_files_update(&ring, 0, &fd, 1);

	sqe = io_uring_get_sqe(&ring);
	io_uring_prep_read(sqe, 0, buf, sizeof(buf), 0);
	sqe->flags |= IOSQE_FIXED_FILE;

	io_uring_submit(&ring);

	memset(buf2, 0xaa, sizeof(buf2));
	write(fds[1], buf2, sizeof(buf2));

	ret = io_uring_wait_cqe(&ring, &cqe);
	if (ret) {
		fprintf(stderr, "wait %d\n", ret);
		return 1;
	}
	if (cqe->res < 0)
		printf("res %s\n", strerror(-cqe->res));
	io_uring_cqe_seen(&ring, cqe);
	return 0;
}

[axboe]

Generally you can use the fixed files in two ways:

1. Application manages the space. This is what is done in the above example, application knows that slot 0 is free and hence just uses that.
2. Kernel manages it, eg when you instantiate a direct socket and you ask the kernel to find a free descriptor for you.

Some mix of the two also allowed, and you can even split the space into two if you prefer to use it that way and have one part managed by the kernel and one by the application.

Does that answer your question?

[cmazakas]

Oh, you know what I just realized? Does this example work with:

       int io_uring_register_files_sparse(struct io_uring *ring,
                                          unsigned nr_files);

?

I realize, the only reason this example works is because when you map in the files table, it's because it's shared with userspace:

	fd = fds[0];
	io_uring_register_files_update(&ring, 0, &fd, 1);

So in the case of the sparse call, there's no way to do the required assignment to the files table because it's not accessible.

Fwiw, I'm under the impression that we can only have one set of registered files at a time so correct me if I'm wrong on that.

[axboe]

It works with sparse files, you can either use the io_uring_register_files_sparse() helper or just register a big table where the values are -1 in each fd member. The end result is identical, a sparse set is just one where all the values are initially -1. The table is maintained in the kernel, whatever you pass in are just the initial values for the size of that set.

You can only have one registered file table, but you can just register a big one. When I mention you can split it what I mean is that you can limit the allocations of files of files with io_uring_register_file_alloc_range() and only have the kernel allocate descriptors (when asked to) within a given range. Then outside of that range the application still manages the space.

[cmazakas]

It works with sparse files, you can either use the io_uring_register_files_sparse() helper or just register a big table where the values are -1 in each fd member. The end result is identical, a sparse set is just one where all the values are initially -1. The table is maintained in the kernel, whatever you pass in are just the initial values for the size of that set.

Holy cow, I'm slow! For some reason, I thought &fd was special here and the liburing was doing some cool mapping thing and the memory was shared. But &fd isn't special.

I've updated your example to work with a sparse table w/ size >1 files:

#include <cstdio>
#include <cstring>
#include <liburing.h>

int main(int argc, char *argv[])
{
	struct io_uring_sqe *sqe;
	struct io_uring_cqe *cqe;
	struct io_uring ring;
	int fds[2];
	char buf[32], buf2[32];
	int ret;

	int nr_files = 8;

	io_uring_queue_init(32, &ring, IORING_SETUP_DEFER_TASKRUN | IORING_SETUP_SINGLE_ISSUER);
	ret = io_uring_register_files_sparse(&ring, nr_files);

	if (pipe(fds) < 0) {
		perror("pipe");
		return 1;
	}

	int fd = 7;
	io_uring_register_files_update(&ring, fd, &fds[0], 1);

	sqe = io_uring_get_sqe(&ring);
	io_uring_prep_read(sqe, fd, buf, sizeof(buf), 0);
	sqe->flags |= IOSQE_FIXED_FILE;

	io_uring_submit(&ring);

	memset(buf2, 0xaa, sizeof(buf2));
	write(fds[1], buf2, sizeof(buf2));

	ret = io_uring_wait_cqe(&ring, &cqe);
	if (ret) {
		fprintf(stderr, "wait %d\n", ret);
		return 1;
	}
	if (cqe->res < 0)
		printf("res %s\n", strerror(-cqe->res));
	io_uring_cqe_seen(&ring, cqe);
	return 0;
}

Ha, so this is all super trivial/easy in all actuality.

Hey, that's amazing! I was actually a little worried this might be difficult lol. But luckily, I already have the infrastructure setup to handle which slots are occupied or not so this is all gonna be smoothing sailing.

Hey, thanks for the help as always, Jens!

[axboe]

Good to hear! I had a suspicion there was some confusion related to how these arrays are mapped and shared (they are not) based on your reply. And yes, that example looks good.

Let me know if there are bit in the man pages that could be better.