Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[API Proposal]: Integrate Reactive's System.Linq.Async APIs into the Base Class Libraries #79782

Open
wsugarman opened this issue Dec 17, 2022 · 41 comments
Assignees
Labels
api-suggestion Early API idea and discussion, it is NOT ready for implementation area-System.Linq
Milestone

Comments

@wsugarman
Copy link
Contributor

Background and motivation

As IAsyncEnumerable<T> increases its presence in the BCLs and user libraries, and as new language constructs are added to support asynchronous streams, I think libraries like System.Linq.Async will become more crucial to their adoption by developers. Personally, as I work with our own Azure SDKs, I find myself quickly importing the package to easily aggregate paginated results using ToListAsync(CancellationToken) without a loop while Azure table queries can be easily aggregated locally using GroupBy.

The LINQ APIs have become incredibly popular in .NET, and I think the lack of LINQ support for asynchronous streams is becoming more apparent. I am especially worried that until recently the entire Reactive project itself was at risk of shutting down without active maintainers! The asynchronous LINQ APIs are far too important to leave outside of the BCLs where their support may disappear.

Why should the System.Linq namespace only include support for IEnumerable<T> when System.Collections.Generic includes IAsyncEnumerable<T>?

API Proposal

Functionally equivalent LINQ APIs (as detailed in this older MSDN documentation) used by the asynchronous equivalents of IEnumerable<T>, IOrderedEnumerable<TElement>, and IGrouping<TKey, TElement>. The current library exists here.

This may also be a good opportunity to re-evaluate these APIs if desired.

E.g.

namespace System.Linq;

public static partial class AsyncEnumerable
{
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TResult> selector);
}

API Usage

using System.Collections.Generic;
using System.Linq;

IAsyncEnumerable<Element> elements = /* ... */;
List<string> ids = await elements
    .Where(x => x.IsValid)
    .Select(x => x.Id)
    .Take(10)
    .OrderBy(x => x.Id)
    .ToListAsync(cancellationToken);

Alternative Designs

Alternatively, these could be kept out-of-band.

Risks

While this may include a large number of APIs, they will be quite familiar to those who use LINQ today on their synchronous counterparts.

@wsugarman wsugarman added the api-suggestion Early API idea and discussion, it is NOT ready for implementation label Dec 17, 2022
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Dec 17, 2022
@ghost
Copy link

ghost commented Dec 17, 2022

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

Background and motivation

As IAsyncEnumerable<T> increases its presence in the BCLs and user libraries, and as new language constructs are added to support asynchronous streams, I think libraries like System.Linq.Async will become more crucial to their adoption by developers. Personally, as I work with our own Azure SDKs, I find myself quickly importing the package to easily aggregate paginated results using ToListAsync(CancellationToken) without a loop while Azure table queries can be easily aggregated locally using GroupBy.

The LINQ APIs have become incredibly popular in .NET, and I think the lack of LINQ support for asynchronous streams is becoming more apparent. I am especially worried that until recently the entire Reactive project itself was at risk of shutting down without active maintainers! The asynchronous LINQ APIs are far too important to leave outside of the BCLs where their support may disappear.

Why should the System.Linq namespace only include support for IEnumerable<T> when System.Collections.Generic includes IAsyncEnumerable<T>?

API Proposal

Functionally equivalent LINQ APIs (as detailed in this older MSDN documentation) used by the asynchronous equivalents of IEnumerable<T>, IOrderedEnumerable<TElement>, and IGrouping<TKey, TElement>. The current library exists here.

This may also be a good opportunity to re-evaluate these APIs if desired.

E.g.

namespace System.Linq;

public static partial class AsyncEnumerable
{
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TResult> selector);
}

API Usage

using System.Collections.Generic;
using System.Linq;

IAsyncEnumerable<Element> elements = /* ... */;
List<string> ids = await elements
    .Where(x => x.IsValid)
    .Select(x => x.Id)
    .Take(10)
    .OrderBy(x => x.Id)
    .ToListAsync(cancellationToken);

Alternative Designs

Alternatively, these could be kept out-of-band.

Risks

While this may include a large number of APIs, they will be quite familiar to those who use LINQ today on their synchronous counterparts.

Author: wsugarman
Assignees: -
Labels:

api-suggestion, area-System.Linq

Milestone: -

@jkotas
Copy link
Member

jkotas commented Dec 17, 2022

Duplicate of #31580

@jkotas jkotas marked this as a duplicate of #31580 Dec 17, 2022
@wsugarman
Copy link
Contributor Author

wsugarman commented Dec 17, 2022

While this was indeed discussed previously, in the time since #31580 it has been discovered that there were no active maintainers for Reactive until volunteers came forward this October. And there haven't been any commits since April!

I think it's worth another discussion of whether this API (or a subset) should be deemed "core" enough to be included in the BCLs.

@eiriktsarpalis
Copy link
Member

While this was indeed discussed previously, in the time since #31580 it has been discovered that there were no active maintainers for Reactive until volunteers came forward this October. And there haven't been any commits since April!

I'm curious, why would moving the library to the shared framework solve the problem of maintenance? If anything, it would make the components more difficult to evolve, since it would prevent introducing breaking changes or versioning it independently of dotnet. There are clear benefits to keeping some components out of the shared framework.

@wsugarman
Copy link
Contributor Author

wsugarman commented Dec 19, 2022

I'm curious, why would moving the library to the shared framework solve the problem of maintenance? If anything, it would make the components more difficult to evolve, since it would prevent introducing breaking changes or versioning it independently of dotnet. There are clear benefits to keeping some components out of the shared framework.

I am not (yet) concerned about the speed of changes in the shared framework. Instead, I am more concerned with making sure someone is responsible for maintaining these (what I think are) critical APIs. While it seems that some dotnet foundation members have charitably helped the project in the past, it now appears that the project is entirely dependent on the kindness of unpaid volunteers.

It's wonderful that developers have come forward to help, and for many OSS projects this model is sufficient, but I wonder if these APIs are too important to leave outside the purview of Microsoft. What happens when the current developers move onto other projects? Will the community continue to be so lucky that new volunteers will step forward? It seems that the recent questions of ownership have only arisen due to enough GitHub issues.

Instead, I think the core System.Linq.Async APIs would benefit greatly from being included in the shared library for the following reasons:

  1. A team of developers at Microsoft would be responsible for their ongoing development, maintenance, and quality, ensuring that they continue to grow as both .NET and async streams evolve
  2. Developers can easily discover and use the async stream LINQ APIs without the use of an external package
    • Today, even performing a simple projection from an IAsyncEnumerable<T> requires a NuGet package, but why should that be the case if it can be done for IEnumerable<T>? I think we'd be hard pressed to explain why there are language constructs for IEnumerable<T> and IAsyncEnumerable<T> like foreach and await foreach, but the LINQ APIs are only included for IEnumerable<T>.
  3. A whole new set of features could be possible, such as adding support for asynchronous LINQ queries in a future dotnet SDK!

@eiriktsarpalis
Copy link
Member

I am not (yet) concerned about the speed of changes in the shared framework.

It's not an issue of speed, it is an issue of possibility. Most types of source and binary breaking changes are not allowed for shared framework components, ever.

A team of developers at Microsoft would be responsible for their ongoing development, maintenance, and quality, ensuring that they continue to grow as both .NET and async streams evolve

Bare minimum, it would guarantee security fixes but it's not necessarily the case that we'd be able to fund continued evolution of the library.

There's also the issue of code size, we've rejected API proposals in regular Linq in the past out of size concerns.

@eiriktsarpalis eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Dec 20, 2022
@eiriktsarpalis eiriktsarpalis added this to the Future milestone Dec 20, 2022
@eiriktsarpalis eiriktsarpalis added the wishlist Issue we would like to prioritize, but we can't commit we will get to it yet label Dec 20, 2022
@wsugarman
Copy link
Contributor Author

wsugarman commented Dec 20, 2022

It's not an issue of speed, it is an issue of possibility. Most types of source and binary breaking changes are not allowed for shared framework components, ever.

@eiriktsarpalis I'll defer to you as one of the maintainers of the System.Linq namespace. Do you feel ever hindered by being included in the shared framework? Based on your other comments, has your team considered an out-of-band package (or perhaps an extension package) for new LINQ APIs to help manage the size?

My biggest concern here is that there is (what I believe to be) a critical library in System.Linq.Async. I believe async streams' usability depends on the presence of this 3rd party library, and without some sort of formal funding from the dotnet org/Microsoft its future remains too uncertain.

As argued in this issue, I think its most logical home -- ignoring the history of how the library came to be and perhaps the burdens of being in the shared library -- is within the BCLs; why should the treatment of the LINQ APIs differ between IEnumerable<T> and IAsyncEnumerable<T>? If it's just a matter of their development history, should that be reconciled? However, if it is believed that the cost of integrating this library into the BCL is too high, then I'm certainly open to alternatives for a more formal investment of engineering resources into this library. Although, it sounds like there should also be a separate discussion around how future LINQ APIs should be added given their burdensome size.

Perhaps most importantly, as the new owner of the library @HowardvanRooijen, would you prefer more direct investment by the dotnet team in the System.Linq.Async library? If so, would you have a preference as to how?

@HowardvanRooijen
Copy link

FYI first alpha release of System.Reactive.Async is available on NuGet: https://www.nuget.org/packages/System.Reactive.Async/6.0.0-alpha.3

@WeihanLi
Copy link
Contributor

As there are more and more IAsyncEnumerable APIs in BCL, it makes me feel the IAsyncEnumerable in BCL is not completed without System.Linq.Async

@stephentoub
Copy link
Member

As there are more and more IAsyncEnumerable APIs in BCL, it makes me feel the IAsyncEnumerable in BCL is not completed without System.Linq.Async

What does it mean to be "in BCL"? We ship many core libraries as nuget packages... if we shipped a System.Linq.Async as a nuget package, how is that different from the status quo?

@Arithmomaniac
Copy link

What does it mean to be "in BCL"? We ship many core libraries as nuget packages... if we shipped a System.Linq.Async as a nuget package, how is that different from the status quo?

The library as-is is functional, but:

  • Does not receive the same level of maintenance, whether for bugfixes or performance improvements
  • Does not use analogous signatures
  • Sometimes is missing new operators entirely.

Compare Linq Chunk to Interactive.Async Buffer (putting aside that it's not even in System.Linq.Async, and progress on changing that is stuck).

@julealgon
Copy link

As there are more and more IAsyncEnumerable APIs in BCL, it makes me feel the IAsyncEnumerable in BCL is not completed without System.Linq.Async

What does it mean to be "in BCL"? We ship many core libraries as nuget packages... if we shipped a System.Linq.Async as a nuget package, how is that different from the status quo?

@stephentoub maybe he is talking about discoverability? Many devs are not aware of the System.Linq.Async package. Perhaps it should be made a default package in worker and web SDKs at least?

@Arithmomaniac
Copy link

I just noticed this exact conversation is also being had at dotnet/reactive#2102; maybe we should redirect further discussion of this to there.

@WeihanLi
Copy link
Contributor

WeihanLi commented Jun 18, 2024

As mentioned by @julealgon, some of us even don't know the System.Linq.Async library, we could not get IntelliSense without adding a package reference for the library.

As @Arithmomaniac mentions, the library may be missing some performance improvements, love to see each performance improvement by @stephentoub and the annual performance improvements blog(book),
when it's outside of BCL, we may miss the performance improvements and refactorings that may be also beneficial for System.Linq.Async, and there's been no new release for the last two years and more.

The LINQ in BCL provides synchronous methods like ToArray, Sum, and Any(and more...), but it lacks asynchronous methods like ToArrayAsync, SumAsync, and AnyAsync(and more...). This could be a limitation when working with IAsyncEnumerable, and we're adding more linq methods likes Chuck/CountBy/... for IEnumerable in BCL, but not investing for IAsyncEnumerable async methods, it may be not fair.

We're using the System.Linq.Async library actually, and each time I had to add the reference, it made me feel incomplete more, just like IEnumerable gives us bullets and guns, while IAsyncEnumerable gives us bullets only

@viceroypenguin
Copy link

@WeihanLi @julealgon fyi, in the interim, some of these updates are added and maintained in the SuperLinq.Async library.

@wsugarman
Copy link
Contributor Author

As there are more and more IAsyncEnumerable APIs in BCL, it makes me feel the IAsyncEnumerable in BCL is not completed without System.Linq.Async

What does it mean to be "in BCL"? We ship many core libraries as nuget packages... if we shipped a System.Linq.Async as a nuget package, how is that different from the status quo?

@stephentoub - I'm catching up on this thread after some time lol. At least from my original post, I agree that inclusion into the BCLs isn't necessarily the goal. From my perspective, and to quote an earlier post, I think the .NET ecosystem would benefit from DevDiv/a team of developers at Microsoft overseeing the ongoing development, maintenance, and quality of the library to help ensure it continues to flourish as both .NET and async streams evolve (in addition to the other reasons mentioned in this thread). For example, perhaps ensuring new versions of the library are published with each new version of .NET (like the extension libraries).

Furthermore, I think there are some interesting features/parity that could still be done with IAsyncEnumerable<T> like the inclusion of LINQ syntax. If that requires inclusion into the BCLs for such a feature, that may be evidence. But I also understand that these libraries were first written some time ago, and the considerations for inclusion into the BCLs have changed.

@Arithmomaniac
Copy link

I agree that inclusion into the BCLs isn't necessarily the goal [...] I think the .NET ecosystem would benefit from DevDiv/a team of developers at Microsoft overseeing the ongoing development, maintenance, and quality of the library to help ensure it continues to flourish as both .NET and async streams evolve [...]. For example, perhaps ensuring new versions of the library are published with each new version of .NET (like the extension libraries).

Agreed. I think "BCL" is being used loosely by those requesting this feature; what is really wanted is first-class development and support.

@eiriktsarpalis
Copy link
Member

I think the .NET ecosystem would benefit from DevDiv/a team of developers at Microsoft overseeing the ongoing development, maintenance, and quality of the library to help ensure it continues to flourish as both .NET and async streams evolve

I don't think it's particularly beneficial to consider Microsoft or DevDiv as being the sole arbiters of good libraries. Even if we wanted to, It wouldn't be a realistic goal given that our resources are finite. I probably speak for the rest of the team in saying that we have strong belief in the wider community's ability to deliver libraries that match or even exceed our own bar for quality.

@HowardvanRooijen
Copy link

HowardvanRooijen commented Jun 19, 2024

So the gist behind dotnet/reactive#2102 is that while we're happy to support Reactive Extensions for .NET (and by we, I mean @endjin) - but we're a small < 15 people organisation, and we have to generate our own revenue (none of which is related to Rx BTW). In the last year we've dedicated 868 hours (108 days) of effort into maintaining Rx .NET - we don't have the capacity to support Interactive Extensions (incl System.Linq.Async) too. Consumers of the library need to step up and own their software supply chain.

(As an aside - bundling the two projects into the single repo only makes sense because the original PG developed both libraries - it's added friction for Rx because potential contributors look at the repo and get put off by how complicated it is. )

While I agree that the .NET PG may not be the sole arbiters of good quality libraries... these did originate from them, and as per dotnet/reactive#2102 @stephentoub has some strong opinions that the design guidelines have evolved in the 14 years since these libraries were originally open sourced (on CodePlex!) - but I'm not sure we, as external folk, have all that knowledge, or enough information to align to those guidelines or design ambitions.

@idg10
Copy link
Contributor

idg10 commented Jun 19, 2024

To follow up on my colleague @HowardvanRooijen 's comment, I also work at endjin and am doing most of the maintenance work on Rx.NET. We've had some sporadic conversations with @davidfowl and @stephentoub about the future of System.Async.Linq.

One upshot of this that @stephentoub pointed out that the library in its current form wouldn't pass muster as an official .NET runtime library, because it does not align with current guidelines:

there were a bunch of choices made in the APIs exposed here that we would not want to bring in, e.g. an AggregateAsync method is fine, but not AggregateAwaitAsync or AggregateAwaitWithCancellationAsync... that's simply not how we'd choose to expose such functionality. We would also be able to design the new methods taking into account features that didn't previously exist, e.g. generic math

Something that I suspect would also be an issue is the efficiency of the code. It was all written quite a long time ago. Today, .NET runtime library code is expected to be much more frugal with memory allocation than was common back then. And although we've not done any profiling to see how well today's System.Linq.Async stacks up against the non-async equivalents in the .NET runtime libraries, I know that some parts of System.Reactive cause allocations at a rate that looks alarmingly high by modern standards, so I would expect the same to be true of System.Linq.Async.

So there are at least two substantial jobs to be done for System.Linq.Async:

  1. modify the API design to align with current .NET runtime library guidelines
  2. bring the performance into line with modern expectations

Since, as Howard points out, a) nobody pays us to do any of this work and b) we only ever really wanted to maintain System.Reactive and we effectively acquired System.Interactive and System.Linq.Async as a side effect of this, we don't really have the capacity to do this.

If somebody wanted it enough to pay for it that might be a different matter, but open source projects rarely seem to work that way.

We would much prefer not to be in charge of System.Linq.Async, but for that to happen, someone would need to take ownership. The obvious candidate would be the same bit of Microsoft that is already in charge of System.Linq, but that's not something we get to decide.

@TomGathercole
Copy link

TomGathercole commented Jun 19, 2024

As there are more and more IAsyncEnumerable APIs in BCL, it makes me feel the IAsyncEnumerable in BCL is not completed without System.Linq.Async

What does it mean to be "in BCL"? We ship many core libraries as nuget packages... if we shipped a System.Linq.Async as a nuget package, how is that different from the status quo?

I'm personally having some grief at the moment due to needing to support queryables coming from both entity framework core and from a custom async-compatible IQueryable provider (Since entity framework did not adopt the IAsyncQueryable interface from System.Linq.Async.Queryable).

I realize that async queryables are a somewhat separate concerns but, in my view, I think this is a symptom of the lack of full built-in async enumerable/queryable support. I.e. if entity framework is unwilling to take a dependency on System.Linq.Async, then other library authors will feel the same and the ecosystem will end up being a little fractured.

Again, I know that queryables aren't explicitly in scope for this discussion, but I think it illustrates a problem caused by important functionality not existing in the framework (and, from a purely selfish perspective, I think this is probably a nice step towards easily supporting async queryables from multiple sources)

@stephentoub
Copy link
Member

stephentoub commented Oct 6, 2024

If the maintainers of the current async linq really don't want to continue maintaining it / don't see a good future for it, I can put together a proposal for what a version would look like as part of the core libs. I just want to reiterate, though, that such a thing will not be 100% binary / source compatible. Existing consumers of the library would almost certainly experience ambiguities / errors and be forced to resolve them upon upgrading.

@TomGathercole
Copy link

If the maintainers of the current async linq really don't want to continue maintaining it / don't see a good future for it, I can put together a proposal for what a version would look like as part of the core libs. I just want to reiterate, though, that such a thing will not be 100% binary / source compatible. Existing consumers of the library would almost certainly experience ambiguities / errors and be forced to resolve them upon upgrading.

By 'upon upgrading' to do you mean upgrading to a hypothetical .netN version that ships with a System.Linq.Async? If so, I can certainly see that causing more friction than typical .net updates have had recently - but it's certainly something we'd be willing to deal with (just as one example).

I suppose some <DoNotUseBuiltInSystemLinqAsync>true></DoNotUseBuiltInSystemLinqAsync> project element could be helpful as a transitional tool (which could make the build in System.Linq.Async behave like a transient package with privateassets = compile)

@stephentoub
Copy link
Member

By 'upon upgrading' to do you mean upgrading to a hypothetical .netN version that ships with a System.Linq.Async?

Yes (or to a newer version of the nuget package, assuming the same name was used).

@idg10
Copy link
Contributor

idg10 commented Oct 7, 2024

If the maintainers of the current async linq really don't want to continue maintaining it

My view is that I believe it would be better if the .NET runtime supplied an implementation of LINQ for IAsyncEnumerable<T>. It would enable us to focus on Rx, and given the very limited resources we can bring to bear on this, focus is good.

If we believed that this was definitely not going to happen, then I think we would put some effort into bringing the existing Ix.NET implementation of this functionality back up to speed. We want this functionality to exist somewhere, and if the .NET runtime libraries will never fill this gap, I think we'd want the Ix.NET implementation to remain viable. (Although that decision is above my pay grade, so I wouldn't ultimately be making the call. Maybe @HowardvanRooijen could provide a perspective here?)

But one of the reasons we've done very little with the Ix.NET implementation of LINQ for IAsyncEnumerable<T> is that @davidfowl has, over the last two years, said on a couple of occasions that he thinks this functionality probably does belong in the .NET runtime libraries. If that is ultimately where we're heading, then the only investments in the Ix.NET implementation for this that really make any sense are a) preparatory steps to make that transition easier and b) continuing to maintain any non-standard Rx-specific operators that the .NET runtime is unlikely ever to support.

(If I thought that by keeping Ix.NET's more closely aligned with LINQ to Objects, we could enable the existing implementation to migrate into the .NET runtime codebase, then that would be worth doing. But as I've mentioned before, my understanding of @stephentoub 's initial analysis of our implementation is that it wouldn't be accepted. Its API design is not compatible with current standards. It seems quite possible that 'migration' would turn into 'complete rewrite', in which case development effort at this point will ultimately be wasted.)

@viceroypenguin
Copy link

@stephentoub If it does look like there will be an attempt to add LINQ for IAsyncEnumerable<T> into the BCL, I'd be interested in helping the team in the process.

@stephentoub
Copy link
Member

stephentoub commented Jan 14, 2025

I went through the effort of creating what I think this would look like if we were to add it into .NET 10. I have an implementation that would just need a bit more cleanup to be ready (e.g. XML docs and tests).

Some design notes, outlining decisions I made, with my justifications for why. We would want to discuss all of these.

  • Naming. There are competing forces at play about naming of the APIs here. On the one hand, we have conventions that async methods, both ValueTask-returning and IAsyncEnumerable-returning, should be suffixed with Async; that serves as an indication in the code about where work is possibly being forked, an important distinction. And we follow this fairly strictly throughout the core libraries. On the other hand, "LINQ" is in part a specification for what methods many different providers should implement, where naming is a key part of that. And that's even codified to some extent in the C#/VB languages, where query comprehension syntax is bound to specific API names, which don't include an "Async" suffix; if we don’t offer the relevant methods without an “Async” suffix, query comprehensions wouldn’t work. You also often end up with chains of methods, and if every method has an "Async" suffix, that starts to get unwieldy, e.g. await foreach (var item in GetDataAsync(...).WhereAsync(...).TakeAsync(...).SelectAsync(...), without adding much benefit. Decision: Use an "Async" suffix for sink methods, which return ValueTask, but don't use an "Async" suffix for IAsyncEnumerable-returning methods; that includes static source methods (e.g. Range, Repeat), where you're already writing "AsyncEnumerable" (similar to why we don't have "Async" as a suffix on Task.WhenAll).
  • Overload explosion. One of our long-term objections to shipping an async LINQ is the sheer possible number of overloads if you want to ship the full matrix. Consider a method like public static IEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IEnumerable<TOuter> outer, IEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector, IEqualityComparer<TKey>? comparer);. In the extreme, there are 2^6-1 possible async overloads we’d need for that one sync overload, where each of the 6 arguments could have an asynchronous equivalent, and any combination of them could in theory be valid (e.g. IEnumerable vs IAsyncEnumerable for inner, IEnumerable vs IAsyncEnumerable for outer, a TKey vs ValueTask<TKey> return for outerKeySelector, etc.). I made a few simplifying decisions here. Decision: 1) any such overload gets at most 2 overloads (with a few specific exemptions): one where all of the IEnumerables are instead IAsyncEnumerable and all of the Func<>s are synchronous, and one where all of the Func<>s are asynchronous. 2) We don’t have asynchronous equivalents for I{Equality}Comparer: there’s not a standard .NET representation for that, nor has significant need been shown for it, nor should there be a significant need because comparison is almost invariably about data already available in the instances rather than something that requires I/O to get. (The exemption mentioned is there are one or two methods, namely SelectMany, where a Func<> returns an enumerable. Where it makes sense, I have an additional overload, e.g. Func<TInput, IEnumerable<TOutput>>, Func<TInput, ValueTask<TIEnumerable<TOutput>>, and Func<TInput, IAsyncEnumerable<TOutput>>.)
  • Comparers. LINQ was initially added before C# got optional parameters, and so many methods have one overload that takes an IEqualityComparer and another that doesn’t. I’ve collapsed those into just one taking an optional IEqualityComparer.
  • Cancellation. In theory, every method should accept a CancellationToken. However, that makes sequences of LINQ methods significantly more verbose, e.g. T result = GetDataAsync(…, cancellationToken).WhereAsync(…, cancellationToken).TakeAsync(…, cancellationToken).SelectAsync(…, cancellationToken).AverageAsync(cancellationToken). We can instead rely on the cancellation token being threaded through, using either WithCancellation (e.g. Select(…).Where(…).WithCancellation(cancellationToken)) or having sink methods taking tokens (e.g. Select(…).Where(…).AverageAsync(cancellationToken)), where the token is just supplied once, to the same effect. Decision: only accept a CancellationToken into the XxAsync sink methods. (All of the overloads that take a Func<…, ValueTask<…>> also accept a non-optional CancellationToken.) We could add an optional cancellation token to all the methods, and then folks can only use them if they want to; that’d be a reasonable decision. If we do that, though, we should also invest in fixing analyzers that warn when you don’t pass a cancellation token into a method that can possibly accept one.
  • Removals. Max/Min have overloads that are no longer relevant, where initially type-specific methods were added but later a generic was added that can also handle those types. I removed the type-specific ones in such situations.
  • Tweaks. Enumerable.Cast/OfType both work on the non-generic IEnumerable, and there’s no non-generic IAsyncEnumerable. I thus added Cast/OfType as generic methods that take both the source and result type, both of which need to be specified (until C# gets partial generic inference). We could also choose to simply not have these methods.
  • Additions. I added an ToAsyncEnumerable(IEnumerable<>) and ToObservable(IAsyncEnumerable), the former because it's critical to reducing overload count, and the latter because IObservable is a core interface but more because the current System.Linq.Async has one.

Implementation-wise, I also significantly simplified the implementation, eschewing most of the special-cases present in the synchronous LINQ implementation. There are lots of code paths in the IEnumerable LINQ that special-case for arrays, lists, etc., and I removed all of those for IAsyncEnumerable (in theory ToAsyncEnumerable could expose internally any details about the nature of the wrapped collection, and the implementation could optimize for that, but we’d need to see really strong motivation for that in real use in order to add all that maintenance burden). There are also many code paths that special-case combinations of operaters; in the IEnumerable LINQ, this forces us to implement the state machines by hand, which is a much more complicated endevour to do well for IAsyncEnumerable… it’s also got more overhead, and often involves I/O, both of which change the equation for how worthwhile it is. And I expect the use cases are meaningfully different, such that assumptions we made about what would be useful for typical consumption with IEnumerable don’t necessarily hold for IAsyncEnumerable. I think it makes sense to start clean, and then only add the complication when use in this domain proves it worthwhile. I also did some quick-and-dirty benchmarking against the current System.Linq.Async, and the simple thing can often end up being faster.

Distribution-wise, my proposal would be to have a System.Linq.Async.dll that would ship both in the shared netcoreapp framework as well as a nuget package, ala System.Threading.Channels, System.Text.Json, etc. The current System.Linq.Async package would simply be replaced by this one, with ownership moving to the .NET team. Developers could choose to continue using the current v6.0.1 of the package and upgrade to the 10.0.0 package when ready to accept the breaking changes. However, a developer moving to .NET 10 would be forced to work through those breaking changes as part of the upgrade.

Surface area:

namespace System.Linq;

public static partial class AsyncEnumerable
{
    public static ValueTask<TSource> AggregateAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, TSource, CancellationToken, ValueTask<TSource>> func, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> AggregateAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, TSource, TSource> func, CancellationToken cancellationToken = default);
    public static ValueTask<TAccumulate> AggregateAsync<TSource, TAccumulate>(this IAsyncEnumerable<TSource> source, TAccumulate seed, Func<TAccumulate, TSource, CancellationToken, ValueTask<TAccumulate>> func, CancellationToken cancellationToken = default);
    public static ValueTask<TAccumulate> AggregateAsync<TSource, TAccumulate>(this IAsyncEnumerable<TSource> source, TAccumulate seed, Func<TAccumulate, TSource, TAccumulate> func, CancellationToken cancellationToken = default);
    public static ValueTask<TResult> AggregateAsync<TSource, TAccumulate, TResult>(this IAsyncEnumerable<TSource> source, TAccumulate seed, Func<TAccumulate, TSource, CancellationToken, ValueTask<TAccumulate>> func, Func<TAccumulate, CancellationToken, ValueTask<TResult>> resultSelector, CancellationToken cancellationToken = default);
    public static ValueTask<TResult> AggregateAsync<TSource, TAccumulate, TResult>(this IAsyncEnumerable<TSource> source, TAccumulate seed, Func<TAccumulate, TSource, TAccumulate> func, Func<TAccumulate, TResult> resultSelector, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<KeyValuePair<TKey, TAccumulate>> AggregateBy<TSource, TKey, TAccumulate>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TKey, CancellationToken, ValueTask<TAccumulate>> seedSelector, Func<TAccumulate, TSource, CancellationToken, ValueTask<TAccumulate>> func, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static IAsyncEnumerable<KeyValuePair<TKey, TAccumulate>> AggregateBy<TSource, TKey, TAccumulate>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, TAccumulate seed, Func<TAccumulate, TSource, CancellationToken, ValueTask<TAccumulate>> func, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static IAsyncEnumerable<KeyValuePair<TKey, TAccumulate>> AggregateBy<TSource, TKey, TAccumulate>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TKey, TAccumulate> seedSelector, Func<TAccumulate, TSource, TAccumulate> func, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static IAsyncEnumerable<KeyValuePair<TKey, TAccumulate>> AggregateBy<TSource, TKey, TAccumulate>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, TAccumulate seed, Func<TAccumulate, TSource, TAccumulate> func, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static ValueTask<bool> AllAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<bool> AllAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<bool> AnyAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<bool> AnyAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<bool> AnyAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TSource> Append<TSource>(this IAsyncEnumerable<TSource> source, TSource element);
    public static ValueTask<decimal> AverageAsync(this IAsyncEnumerable<decimal> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync(this IAsyncEnumerable<double> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync(this IAsyncEnumerable<int> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync(this IAsyncEnumerable<long> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> AverageAsync(this IAsyncEnumerable<decimal?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync(this IAsyncEnumerable<double?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync(this IAsyncEnumerable<int?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync(this IAsyncEnumerable<long?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float?> AverageAsync(this IAsyncEnumerable<float?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float> AverageAsync(this IAsyncEnumerable<float> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, double> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, long> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, decimal?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, double?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, long?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, float?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, float> selector, CancellationToken cancellationToken);
    public static ValueTask<decimal> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<decimal>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<double>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<int>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<long>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<decimal?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<double?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<int?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<long?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float?> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<float?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float> AverageAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<float>> selector, CancellationToken cancellationToken);
    public static IAsyncEnumerable<TTarget> Cast<TSource, TTarget>(this IAsyncEnumerable<TSource> source);
    public static IAsyncEnumerable<TSource[]> Chunk<TSource>(this IAsyncEnumerable<TSource> source, int size);
    public static IAsyncEnumerable<TSource> Concat<TSource>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second);
    public static ValueTask<bool> ContainsAsync<TSource>(this IAsyncEnumerable<TSource> source, TSource value, IEqualityComparer<TSource>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<int> CountAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<int> CountAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<int> CountAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<KeyValuePair<TKey, int>> CountBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static IAsyncEnumerable<KeyValuePair<TKey, int>> CountBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? keyComparer = null) where TKey : notnull;
    public static IAsyncEnumerable<TSource?> DefaultIfEmpty<TSource>(this IAsyncEnumerable<TSource> source);
    public static IAsyncEnumerable<TSource> DefaultIfEmpty<TSource>(this IAsyncEnumerable<TSource> source, TSource defaultValue);
    public static IAsyncEnumerable<TSource> DistinctAsync<TSource>(this IAsyncEnumerable<TSource> source, IEqualityComparer<TSource>? comparer = null);
    public static IAsyncEnumerable<TSource> DistinctBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> DistinctBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static ValueTask<TSource> ElementAtAsync<TSource>(this IAsyncEnumerable<TSource> source, System.Index index, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> ElementAtAsync<TSource>(this IAsyncEnumerable<TSource> source, int index, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> ElementAtOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, System.Index index, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> ElementAtOrDefault<TSource>(this IAsyncEnumerable<TSource> source, int index, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TResult> Empty<TResult>();
    public static IAsyncEnumerable<TSource> ExceptBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TKey> second, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> ExceptBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TKey> second, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> Except<TSource>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, IEqualityComparer<TSource>? comparer = null);
    public static ValueTask<TSource> FirstAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> FirstAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> FirstAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> FirstOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, TSource defaultValue, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<IGrouping<TKey, TSource>> GroupBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<IGrouping<TKey, TSource>> GroupBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<IGrouping<TKey, TElement>> GroupBy<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TSource, CancellationToken, ValueTask<TElement>> elementSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TKey, IEnumerable<TSource>, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<IGrouping<TKey, TElement>> GroupBy<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TSource, TElement> elementSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TKey, IEnumerable<TSource>, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TElement, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TSource, CancellationToken, ValueTask<TElement>> elementSelector, Func<TKey, IEnumerable<TElement>, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TElement, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TSource, TElement> elementSelector, Func<TKey, IEnumerable<TElement>, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, CancellationToken, ValueTask<TKey>> outerKeySelector, Func<TInner, CancellationToken, ValueTask<TKey>> innerKeySelector, Func<TOuter, IEnumerable<TInner>, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, IEnumerable<TInner>, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<(int Index, TSource Item)> Index<TSource>(this IAsyncEnumerable<TSource> source);
    public static IAsyncEnumerable<TSource> IntersectBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TKey> second, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> IntersectBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TKey> second, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> Intersect<TSource>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, IEqualityComparer<TSource>? comparer = null);
    public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, CancellationToken, ValueTask<TKey>> outerKeySelector, Func<TInner, CancellationToken, ValueTask<TKey>> innerKeySelector, Func<TOuter, TInner, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer);
    public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector, IEqualityComparer<TKey>? comparer);
    public static ValueTask<TSource> LastAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> LastAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> LastAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> LastOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, TSource defaultValue, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TResult> LeftJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, CancellationToken, ValueTask<TKey>> outerKeySelector, Func<TInner, CancellationToken, ValueTask<TKey>> innerKeySelector, Func<TOuter, TInner?, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> LeftJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner?, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static ValueTask<long> LongCountAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<long> LongCountAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<long> LongCountAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal> MaxAsync(this IAsyncEnumerable<decimal> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> MaxAsync(this IAsyncEnumerable<double> source, CancellationToken cancellationToken = default);
    public static ValueTask<int> MaxAsync(this IAsyncEnumerable<int> source, CancellationToken cancellationToken = default);
    public static ValueTask<long> MaxAsync(this IAsyncEnumerable<long> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> MaxAsync(this IAsyncEnumerable<decimal?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> MaxAsync(this IAsyncEnumerable<double?> source, CancellationToken cancellationToken = default);
    public static ValueTask<int?> MaxAsync(this IAsyncEnumerable<int?> source, CancellationToken cancellationToken = default);
    public static ValueTask<long?> MaxAsync(this IAsyncEnumerable<long?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float?> MaxAsync(this IAsyncEnumerable<float?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float> MaxAsync(this IAsyncEnumerable<float> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MaxAsync<TSource>(this IAsyncEnumerable<TSource> source, IComparer<TSource>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MaxBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MaxBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TResult?> MaxAsync<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TResult> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal> MinAsync(this IAsyncEnumerable<decimal> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> MinAsync(this IAsyncEnumerable<double> source, CancellationToken cancellationToken = default);
    public static ValueTask<int> MinAsync(this IAsyncEnumerable<int> source, CancellationToken cancellationToken = default);
    public static ValueTask<long> MinAsync(this IAsyncEnumerable<long> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> MinAsync(this IAsyncEnumerable<decimal?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> MinAsync(this IAsyncEnumerable<double?> source, CancellationToken cancellationToken = default);
    public static ValueTask<int?> MinAsync(this IAsyncEnumerable<int?> source, CancellationToken cancellationToken = default);
    public static ValueTask<long?> MinAsync(this IAsyncEnumerable<long?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float?> MinAsync(this IAsyncEnumerable<float?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float> MinAsync(this IAsyncEnumerable<float> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MinAsync<TSource>(this IAsyncEnumerable<TSource> source, IComparer<TSource>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TResult?> MinAsync<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TResult> selector, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MinByAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> MinByAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TTarget> OfType<TSource, TTarget>(this IAsyncEnumerable<TSource> source);
    public static IOrderedAsyncEnumerable<TSource> OrderByDescending<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> OrderByDescending<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> OrderBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> OrderBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<T> OrderDescending<T>(this IAsyncEnumerable<T> source, IComparer<T>? comparer = null);
    public static IOrderedAsyncEnumerable<T> Order<T>(this IAsyncEnumerable<T> source, IComparer<T>? comparer = null);
    public static IAsyncEnumerable<TSource> Prepend<TSource>(this IAsyncEnumerable<TSource> source, TSource element);
    public static IAsyncEnumerable<int> Range(int start, int count);
    public static IAsyncEnumerable<TResult> Repeat<TResult>(TResult element, int count);
    public static IAsyncEnumerable<TSource> Reverse<TSource>(this IAsyncEnumerable<TSource> source);
    public static IAsyncEnumerable<TResult> RightJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, CancellationToken, ValueTask<TKey>> outerKeySelector, Func<TInner, CancellationToken, ValueTask<TKey>> innerKeySelector, Func<TOuter?, TInner, CancellationToken, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> RightJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter?, TInner, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, IAsyncEnumerable<TResult>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, IAsyncEnumerable<TResult>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, IEnumerable<TResult>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<IEnumerable<TResult>>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<IEnumerable<TResult>>> selector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, IAsyncEnumerable<TCollection>> collectionSelector, Func<TSource, TCollection, CancellationToken, ValueTask<TResult>> resultSelector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, IEnumerable<TCollection>> collectionSelector, Func<TSource, TCollection, TResult> resultSelector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, IAsyncEnumerable<TCollection>> collectionSelector, Func<TSource, TCollection, CancellationToken, ValueTask<TResult>> resultSelector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, IEnumerable<TCollection>> collectionSelector, Func<TSource, TCollection, TResult> resultSelector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<IEnumerable<TCollection>>> collectionSelector, Func<TSource, TCollection, CancellationToken, ValueTask<TResult>> resultSelector);
    public static IAsyncEnumerable<TResult> SelectMany<TSource, TCollection, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<IEnumerable<TCollection>>> collectionSelector, Func<TSource, TCollection, CancellationToken, ValueTask<TResult>> resultSelector);
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<TResult>> selector);
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, int, TResult> selector);
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TResult>> selector);
    public static IAsyncEnumerable<TResult> Select<TSource, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TResult> selector);
    public static ValueTask<bool> SequenceEqualAsync<TSource>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, IEqualityComparer<TSource>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate, TSource defaultValue, CancellationToken cancellationToken = default);
    public static ValueTask<TSource?> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<TSource> SingleOrDefaultAsync<TSource>(this IAsyncEnumerable<TSource> source, TSource defaultValue, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TSource> SkipLast<TSource>(this IAsyncEnumerable<TSource> source, int count);
    public static IAsyncEnumerable<TSource> SkipWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate);
    public static IAsyncEnumerable<TSource> SkipWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, bool> predicate);
    public static IAsyncEnumerable<TSource> SkipWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<TSource> SkipWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<TSource> Skip<TSource>(this IAsyncEnumerable<TSource> source, int count);
    public static ValueTask<decimal> SumAsync(this IAsyncEnumerable<decimal> source, CancellationToken cancellationToken = default);
    public static ValueTask<double> SumAsync(this IAsyncEnumerable<double> source, CancellationToken cancellationToken = default);
    public static ValueTask<int> SumAsync(this IAsyncEnumerable<int> source, CancellationToken cancellationToken = default);
    public static ValueTask<long> SumAsync(this IAsyncEnumerable<long> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> SumAsync(this IAsyncEnumerable<decimal?> source, CancellationToken cancellationToken = default);
    public static ValueTask<double?> SumAsync(this IAsyncEnumerable<double?> source, CancellationToken cancellationToken = default);
    public static ValueTask<int?> SumAsync(this IAsyncEnumerable<int?> source, CancellationToken cancellationToken = default);
    public static ValueTask<long?> SumAsync(this IAsyncEnumerable<long?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float?> SumAsync(this IAsyncEnumerable<float?> source, CancellationToken cancellationToken = default);
    public static ValueTask<float> SumAsync(this IAsyncEnumerable<float> source, CancellationToken cancellationToken = default);
    public static ValueTask<decimal> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, decimal> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, double> selector, CancellationToken cancellationToken = default);
    public static ValueTask<int> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int> selector, CancellationToken cancellationToken = default);
    public static ValueTask<long> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, long> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, decimal?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, double?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<int?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<long?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, long?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, float?> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, float> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<decimal>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<double>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<int> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<int>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<long> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<long>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<decimal?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<decimal?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<double?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<double?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<int?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<int?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<long?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<long?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float?> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<float?>> selector, CancellationToken cancellationToken = default);
    public static ValueTask<float> SumAsync<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<float>> selector, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TSource> TakeLast<TSource>(this IAsyncEnumerable<TSource> source, int count);
    public static IAsyncEnumerable<TSource> TakeWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate);
    public static IAsyncEnumerable<TSource> TakeWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, bool> predicate);
    public static IAsyncEnumerable<TSource> TakeWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<TSource> TakeWhile<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<TSource> Take<TSource>(this IAsyncEnumerable<TSource> source, int count);
    public static IAsyncEnumerable<TSource> Take<TSource>(this IAsyncEnumerable<TSource> source, System.Range range);
    public static IOrderedAsyncEnumerable<TSource> ThenByDescending<TSource, TKey>(this IOrderedAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> ThenByDescending<TSource, TKey>(this IOrderedAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> ThenBy<TSource, TKey>(this IOrderedAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IComparer<TKey>? comparer = null);
    public static IOrderedAsyncEnumerable<TSource> ThenBy<TSource, TKey>(this IOrderedAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer = null);
    public static ValueTask<TSource[]> ToArrayAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<TSource> ToAsyncEnumerable<TSource>(this IEnumerable<TSource> source);
    public static ValueTask<Dictionary<TKey, TValue>> ToDictionaryAsync<TKey, TValue>(this IAsyncEnumerable<KeyValuePair<TKey, TValue>> source, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<Dictionary<TKey, TValue>> ToDictionaryAsync<TKey, TValue>(this IAsyncEnumerable<(TKey Key, TValue Value)> source, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<Dictionary<TKey, TSource>> ToDictionaryAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<Dictionary<TKey, TSource>> ToDictionaryAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<Dictionary<TKey, TElement>> ToDictionaryAsync<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TSource, CancellationToken, ValueTask<TElement>> elementSelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<Dictionary<TKey, TElement>> ToDictionaryAsync<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TSource, TElement> elementSelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default) where TKey : notnull;
    public static ValueTask<HashSet<TSource>> ToHashSetAsync<TSource>(this IAsyncEnumerable<TSource> source, IEqualityComparer<TSource>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<List<TSource>> ToListAsync<TSource>(this IAsyncEnumerable<TSource> source, CancellationToken cancellationToken = default);
    public static ValueTask<ILookup<TKey, TSource>> ToLookupAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<ILookup<TKey, TSource>> ToLookupAsync<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<ILookup<TKey, TElement>> ToLookupAsync<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, Func<TSource, CancellationToken, ValueTask<TElement>> elementSelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static ValueTask<ILookup<TKey, TElement>> ToLookupAsync<TSource, TKey, TElement>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TSource, TElement> elementSelector, IEqualityComparer<TKey>? comparer = null, CancellationToken cancellationToken = default);
    public static IObservable<TSource> ToObservable<TSource>(this IAsyncEnumerable<TSource> source);
    public static IAsyncEnumerable<TSource> UnionBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, Func<TSource, CancellationToken, ValueTask<TKey>> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> UnionBy<TSource, TKey>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
    public static IAsyncEnumerable<TSource> Union<TSource>(this IAsyncEnumerable<TSource> first, IAsyncEnumerable<TSource> second, IEqualityComparer<TSource>? comparer = null);
    public static IAsyncEnumerable<TSource> Where<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, bool> predicate);
    public static IAsyncEnumerable<TSource> Where<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, bool> predicate);
    public static IAsyncEnumerable<TSource> Where<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, int, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<TSource> Where<TSource>(this IAsyncEnumerable<TSource> source, Func<TSource, CancellationToken, ValueTask<bool>> predicate);
    public static IAsyncEnumerable<(TFirst First, TSecond Second)> Zip<TFirst, TSecond>(this IAsyncEnumerable<TFirst> first, IAsyncEnumerable<TSecond> second);
    public static IAsyncEnumerable<(TFirst First, TSecond Second, TThird Third)> Zip<TFirst, TSecond, TThird>(this IAsyncEnumerable<TFirst> first, IAsyncEnumerable<TSecond> second, IAsyncEnumerable<TThird> third);
    public static IAsyncEnumerable<TResult> Zip<TFirst, TSecond, TResult>(this IAsyncEnumerable<TFirst> first, IAsyncEnumerable<TSecond> second, Func<TFirst, TSecond, CancellationToken, ValueTask<TResult>> resultSelector);
    public static IAsyncEnumerable<TResult> Zip<TFirst, TSecond, TResult>(this IAsyncEnumerable<TFirst> first, IAsyncEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> resultSelector);
}
public partial interface IOrderedAsyncEnumerable<out TElement> : IAsyncEnumerable<TElement>
{
    IOrderedAsyncEnumerable<TElement> CreateOrderedEnumerable<TKey>(Func<TElement, TKey> keySelector, IComparer<TKey>? comparer, bool descending);
    IOrderedAsyncEnumerable<TElement> CreateOrderedEnumerable<TKey>(Func<TElement, CancellationToken, ValueTask<TKey>> keySelector, IComparer<TKey>? comparer, bool descending);
}

Current dotnet/reactive maintainers (@idg10, @HowardvanRooijen), what would you like the outcome to be here? Is it still your desire to stop maintaining the current one and transfer ownership? Are you ok with the resulting breaking changes devs will experience?

@eiriktsarpalis, @terrajobst, @davidfowl, @MadsTorgersen, @jaredpar, @jeffhandley, opinions? Do we want to move forward with this?

@idg10
Copy link
Contributor

idg10 commented Jan 14, 2025

The current System.Linq.Async package would simply be replaced by this one, with ownership moving to the .NET team. Developers could choose to continue using the current v6.0.1 of the package and upgrade to the 10.0.0 package when ready to accept the breaking changes.

So in scenarios such as this:

  • MyApp
    • RandomLibrary1ThatIDidntKnowEvenUsedSystemLinqAsync v1.0
      • System.Linq.Async v6.0.1
    • RandomLibrary2ThatIDidntKnowEvenUsedSystemLinqAsync v1.0
      • System.Linq.Async v6.0.1

if I upgrade RandomLibrary1ThatIDidntKnowEvenUsedSystemLinqAsync from v1.0 to v2.0, and if v2.0 now depends on System.Linq.Async v10.0.0, then I think that means RandomLibrary2ThatIDidntKnowEvenUsedSystemLinqAsync is also going to get upgraded to v10.0.0 of that, right? But if I'm still on v1.0 of that, does that mean I can expect a load of missing method exceptions?

So if I have multiple transient dependencies on System.Linq.Async, I can't upgrade any of them to a new version that requires the v10 version until I'm able to upgrade all of them?

@stephentoub
Copy link
Member

So if I have multiple transient dependencies on System.Linq.Async, I can't upgrade any of them to a new version that requires the v10 version until I'm able to upgrade all of them?

Yup!

@idg10
Copy link
Contributor

idg10 commented Jan 14, 2025

Current dotnet/reactive maintainers (@idg10, @HowardvanRooijen), what would you like the outcome to be here? Is it still your desire to stop maintaining the current one and transfer ownership? Are you ok with the resulting breaking changes devs will experience?

We would like to stop maintaining the current library, and to transfer ownership.

I am concerned about the breaking changes. According to NuGet, System.Linq.Async gets about 1 million downloads a week. I don't know of any way to ask NuGet how many other packages depend on System.Linq.Async, but GitHub's dependency analyser has found 1,082 packages that depend on it.

I'm wondering what the reason is for keeping the identity of this package the same, instead of deprecating it, and giving the new one a different name. I've been burned in the past by NuGet packages introducing incompatible changes, putting projects in a place where they have unresolvable conflicts due to transient dependencies on different, incompatible versions of the 'same' component.

If at all possible, I'd prefer to avoid creating a situation like that. But if there's some reason I've not thought of that makes this impossible to avoid here, then I would still opt to take the hit this one time.

@stephentoub
Copy link
Member

and giving the new one a different name

What name would you give to the nuget package and library? Neither could be the "good" name or else you'd face similar problems. Suggestions?

@idg10
Copy link
Contributor

idg10 commented Jan 14, 2025

Would System.Linq.AsyncEnumerable work?

This package is specific to IAsyncEnumerable. (There are other "async LINQ" implementations out there, such as System.Reactive.Async.) System.Linq.IAsyncEnumerable just looked wrong to me, but System.Linq.AsyncEnumerable seems like a reasonable description of what it does.

@stephentoub
Copy link
Member

That's a possibility.

Note that it will help avoid some but not all such breaking change impact. If both the old and the new package end up getting pulled in, you'd still have conflicts when trying to compile code that uses "AsyncEnumerable", for example, or when using extension methods that end up being ambiguous between the two libs.

You're ok with that level of impact?

@idg10
Copy link
Contributor

idg10 commented Jan 14, 2025

you'd still have conflicts when trying to compile code that uses "AsyncEnumerable", for example, or when using extension methods that end up being ambiguous between the two libs.

Yes, I'm familiar with that problem from our ongoing attempts to reverse the historical decision to merge the UI framework code into the main Rx package... 😢

It's an unpleasant situation, but at least it's not completely impossible to disentangle yourself. (And for many it will offer an unprecedented opportunity to use extern aliases...)

You're ok with that level of impact?

It would be acceptable to me. Of course, since everyone already seems to think Microsoft maintains this library, I might not be the one on the receiving end of most of any vitriol. Are you ok with that level of impact?

@eiriktsarpalis
Copy link
Member

  • In the extreme, there are 2^6-1 possible async overloads we’d need for that one sync overload, where each of the 6 arguments could have an asynchronous equivalent, and any combination of them could in theory be valid (e.g. IEnumerable vs IAsyncEnumerable for inner, IEnumerable vs IAsyncEnumerable for outer, a TKey vs ValueTask<TKey> return for outerKeySelector, etc.)

In practice couldn't this be addressed by exposing an IEnumerable to IAsyncEnumerable wrapper? Any IEnumerable input could be quickly converted using the .AsAsyncEnumerable() extension method.

opinions?

The high level shapes look reasonable, although I'm somewhat sceptical of the utility of chaining these methods LINQ-style, particularly the ones that require buffering the entire source such as Reverse, Sort, or GroupBy. In these cases I would just buffer to a list myself and chain with regular LINQ operations. The sink methods like FirstAsync or MaxAsync seem like useful accelerators, even though they're fairly straightforward to express using await foreach.

I'm also wondering if IAE in particular gives rise Rx-style methods that take timing into account, e.g. Buffer or reactive GroupBy (i.e. one whose grouping values are themselves IAE<T>).

public static IAsyncEnumerable GroupBy<TSource, TKey, TResult>(this IAsyncEnumerable source, Func<TSource, CancellationToken, ValueTask> keySelector, Func<TKey, IEnumerable, CancellationToken, ValueTask> resultSelector, IEqualityComparer? comparer = null);

It seems we could take a break from Linq APIs and feed IReadOnlyList<TSource> to the key selector delegate.

@stephentoub
Copy link
Member

In practice couldn't this be addressed by exposing an IEnumerable to IAsyncEnumerable wrapper? Any IEnumerable input could be quickly converted using the .AsAsyncEnumerable() extension method.

Yes, that's one of the reasons I added the AsAsyncEnumerable and felt comfortable not having those variations.

It seems we could take a break from Linq APIs and feed IReadOnlyList to the key selector delegate.

Can you share a diff of what changes you'd want to see?

I'm also wondering if IAE in particular gives rise Rx-style methods that take timing into account, e.g. Buffer or reactive GroupBy (i.e. one whose grouping values are themselves IAE).

Quite possibly there'd be other APIs that are more impactful in an IAsyncEnumerable world than in the IEnumerable world. However, I want to keep this initial discussions focused on bridging the gap.

@idg10, are there entire categories of APIs in System.Linq.Async today that aren't in Enumerable? (Not just overloads with minor differences in args/names.)

@stephentoub
Copy link
Member

I might not be the one on the receiving end of most of any vitriol.

We'd do our best to engage you so that you don't feel left out 😛

Are you ok with that level of impact?

We still need to decide that.

@eiriktsarpalis
Copy link
Member

Can you share a diff of what changes you'd want to see?

Specifically for the shapes as proposed above I would surface the groupings as IReadOnlyList<T> or IReadOnlyCollection<T> so that users can easily look up the sized of each grouping:

public static IAsyncEnumerable<KeyValuePair<TKey, IReadOnlyList<TSource>> GroupBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TKey, IReadOnlyList<TSource>, TResult> resultSelector, IEqualityComparer<TKey>? comparer = null);

But if we were to consider reactive-style semantics we might instead want to surface the following:

public static IAsyncEnumerable<KeyValuePair<TKey, IAsyncEnumerable<TSource>> GroupBy<TSource, TKey>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, IEqualityComparer<TKey>? comparer = null);
public static IAsyncEnumerable<TResult> GroupBy<TSource, TKey, TResult>(this IAsyncEnumerable<TSource> source, Func<TSource, TKey> keySelector, Func<TKey, IAsyncEnumerable<TSource>, ValueTask<TResult>> resultSelector, IEqualityComparer<TKey>? comparer = null);

@stephentoub stephentoub removed the wishlist Issue we would like to prioritize, but we can't commit we will get to it yet label Jan 14, 2025
@Arithmomaniac
Copy link

although I'm somewhat skeptical of the utility of chaining these methods LINQ-style, particularly the ones that require buffering the entire source such as Reverse, Sort, or GroupBy. In these cases I would just buffer to a list myself and chain with regular LINQ operations.

Not every enumerable is a LINQ-to-Objects enumerable; some are backed by databases (lQueryable inherits IEnumerable). For example, it follows that if #77698 were ever implemented, these would be appropriate signatures for IAsyncQueryable.

@Arithmomaniac
Copy link

Arithmomaniac commented Jan 15, 2025

you'd still have conflicts when trying to compile code that uses "AsyncEnumerable", for example, or when using extension methods that end up being ambiguous between the two libs.

Two possible options:

  1. These can be resolved by declaring a static import for each method group; this is how MoreLinq works. It's a hacky solution for the BCL, though.
  2. Could the conflict behavior be controlled by a compatibility build property, the way EnableUnsafeBinaryFormatterSerialization is?

@stephentoub
Copy link
Member

These can be resolved by declaring a static import for each method group; this is how MoreLinq works. It's a hacky solution for the BCL, though.

Thanks, but yes, that's not a design we'd employ in the core libraries. If we're going to bring these IAsyncEnumerable LINQ APIs into the core libraries, we want it done with an eye towards it being "correct" for the future, not hampered by the past.

Could the conflict behavior be controlled by a compatibility build property, the way EnableUnsafeBinaryFormatterSerialization is?

No, such switches affect run-time behaviors, but this is about build-time and what metadata is exposed to the compiler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-suggestion Early API idea and discussion, it is NOT ready for implementation area-System.Linq
Projects
None yet
Development

No branches or pull requests