Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new FileStream(fd,..., isAsync:true) doesn't appear to be supported on Unix #85560

Closed
wasabii opened this issue Apr 29, 2023 · 5 comments
Closed

Comments

@wasabii
Copy link

wasabii commented Apr 29, 2023

I don't appear to have any way to create a new FileStream from a raw int FD on Linux with async set to true. It's easy enough for me to call open() with O_ASYNC. The next step would be to create a SafeFileHandle, and then hand it to the ctor of FileStream. However, SafeFileHandle's Unix implementation seems to have no way to tell it the underlying FD is async. The Windows version calls into ntdll to actually detect whether the FD is async. The Unix version doesn't have any such call, and only sets IsAsync when it is responsible for calling open().

However, new FileStream(sfh,,,true) validates that IsAsync on the SafeFileHandle is true.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Apr 29, 2023
@ghost
Copy link

ghost commented Apr 29, 2023

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

I don't appear to have any supported way to create a new FileStream from a raw int FD on Linux with async set to true. It's easy enough for me to call open() with O_ASYNC. The next step would be to create a SafeFileHandle, and then hand it to the ctor of FileStream. However, SafeFileHandle's Unix implementation seems to have no way to tell it the underlying FD is async. The Windows version calls into ntdll to actually detect whether the FD is async. The Unix version doesn't have any such call, and only sets IsAsync when it is responsible for calling open().

However, new FileStream(sfh,,,true) validates that IsAsync on the SafeFileHandle is true.

FileStream(IntPtr...true) seems to work, but is marked deprecated.

Author: wasabii
Assignees: -
Labels:

area-System.IO, untriaged

Milestone: -

@wasabii wasabii changed the title new FileStream(fd, isAsync:true) doesn't appear to be supported on Unix new FileStream(fd,..., isAsync:true) doesn't appear to be supported on Unix Apr 29, 2023
wasabii added a commit to ikvmnet/ikvm that referenced this issue Apr 29, 2023
…ReadAsync can be positional, and FileStream WriteAsync/ReadAsync isn't exactly thread safe on all platforms (sorta was on Core, but is no longer on NET 6, they added RandomFileAccess), we need a lock around the FD for these operations (position can change between set position and write). This can be fixed by going after the underlying IO directly, but this code is already built around FileStream. So, basically, these AIO methods are now queued per file.

.NET 6 however has a problem. No ability to create a FileStream from a FD on Unix with async: dotnet/runtime#85560
So, we go directly to the private IsAsync field if it exists and set it. Hack.
This mismatch between .NET and Java expectations for IO strike again.
@Gnbrkm41
Copy link
Contributor

Gnbrkm41 commented May 6, 2023

It appears that you can query the flags set on a given file descriptor using fcntl function in F_GETFL mode, could we perhaps return the actual async-ness in the SafeFileHandle instead of storing whether the file descriptor was opened with one?

@adamsitnik
Copy link
Member

Hi @wasabii

O_ASYNC does nothing for regular files on Linux. FileStream uses plain read and write sys-calls for pipes and sockets (no epoll), so there would be no gain from using it for these two file types as well.

From https://man7.org/linux/man-pages/man2/open.2.html

This feature is available only for terminals, pseudoterminals, sockets, and (since Linux 2.6) pipes and FIFOs.

If you are using sockets please use Socket type that provides full async support on Linux (via epoll).

In the future we might implement io_uring support for FileStream. Please follow #51985 for updates.

@adamsitnik adamsitnik closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label May 9, 2023
@wasabii
Copy link
Author

wasabii commented May 9, 2023

@adamsitnik

At the OS level that might all be true, however, and I might be not completely accurate here anymore, because I'd have to fire up my tests again, it's been awhile.... I found that FileStream.ReadAsync/WriteAsync and similar methods were unusable unless FileStream thought it was talking to an async handle. They threw.

So, even though FileStream.ReadAsync would technically function on Unix by doing synchronous operations: it refused to do so unless it thought it was opened for async operations. And the only way to create a new FileStream that thought it was capable of doing so, from a FD, was to pass it a SafeFileHandle marked for isAsync.

Ignore O_ASYNC, as I suppose that was somewhat of a red herring for my purposes here. I need a FileStream that is capable of having ReadAsync called on it. And I need to initialize that FileStream from a FD. And there is no option to create a FileStream with isAsync: true, if the SafeFileHandle.IsAsync is false.

@adamsitnik
Copy link
Member

FileStream.ReadAsync/WriteAsync and similar methods were unusable unless FileStream thought it was talking to an async handle. They threw.

These methods should always work no matter how you create the FileStream.

@wasabii Which .NET version are you using? Could you please provide a repro?

@ghost ghost locked as resolved and limited conversation to collaborators Jun 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants