[API Proposal]: Infinite stack

[Mr0N]

Background and motivation

I suggest adding a class where you can specify the stack size for the executing method.

API Proposal

 class StackExecute
{
    public  void Run(Action action,float sizeStack);
}

API Usage

int count = 0;
StackExecute execute = new();
execute.Run(()=>{
    TestMethod();
    },2000_000);    
void TestMethod()
{
    count++;
    if(count < 1000_000)
        TestMethod();
}

Alternative Designs

No response

Risks

No response

[tannergooding]

You can already specify the stack size when creating a new thread: https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=net-8.0#system-threading-thread-ctor(system-threading-threadstart-system-int32)

Most platforms do not provide a means through which you can dynamically change the stack size of an existing thread.

That being said, large stack sizes are not always "good" and can often break or hurt hardware level optimizations the CPU does. In general, if you have something where you know deep recursion may be necessary it is better to create some sort of struct or class to hold the per level state and then simply use a Stack<T>, LinkedList<T>, or similar instead. This gives you the best performance and removes the consideration of "stack overflow" (although you do still need to be mindful of the potential for out of memory exceptions and such).

[En3Tho]

Can I have a float.Epsilon stack size please?

Not sure what to pass in? PositiveInfinity to the rescue!

I like this api

[teo-tsirpanis]

You can use Recursion't.

[Mr0N]

You can already specify the stack size when creating a new thread: https://learn.microsoft.com/en-us/dotnet/api/system.threading.thread.-ctor?view=net-8.0#system-threading-thread-ctor(system-threading-threadstart-system-int32)

Most platforms do not provide a means through which you can dynamically change the stack size of an existing thread.

That being said, large stack sizes are not always "good" and can often break or hurt hardware level optimizations the CPU does. In general, if you have something where you know deep recursion may be necessary it is better to create some sort of struct or class to hold the per level state and then simply use a Stack<T>, LinkedList<T>, or similar instead. This gives you the best performance and removes the consideration of "stack overflow" (although you do still need to be mindful of the potential for out of memory exceptions and such).

Creating a new thread to execute a method is not very pleasant. I would like something like a class where I allocate a gigabyte, and once it finishes its work, I can destroy it. I imagine the implementation would create another stack in the same thread that can be closed after the method completes.

[tannergooding]

Creating a new thread to execute a method is not very pleasant.

It's what is fundamentally required for this to work and has almost identical syntax to what you've proposed above.

I would like something like a class where I allocate a gigabyte, and once it finishes its work, I can destroy it.

This would not be recommended as it would fail on most computer systems, especially mobile devices or laptops. It would likely cause thrashing and other issues as well, making your code many times slower, cause UI or other delays, etc.

I imagine the implementation would create another stack in the same thread that can be closed after the method completes.

Most OS do not allow this, once a thread begins execution its stack size is fixed for the lifetime of the thread.

[teo-tsirpanis]

If you are doing an operation that needs 1GB of stack space, the overhead of creating a new thread and Joining it will be negligible.

[colejohnson66]

Most OS do not allow this, once a thread begins execution its stack size is fixed for the lifetime of the thread.

Is it possible to allocate a new stack, then hook up a continuation thunk that would handle returning to the old one? The thunk would shuffle the stack and base pointers, then call into the new code. On return, it would do the same in reverse. I remember reading some ideas or research along those lines. After all, why should the OS care where the stack is?

[tannergooding]

Is it possible to allocate a new stack, then hook up a continuation thunk that would handle returning to the old one

At the hardware level, yes. But most OS don't provide such functionality because it is atypical and unnecessary. There is next to zero practical benefit to having stacks that can grow infinitely and many very real downsides ranging from perf to security and beyond.

The OS cares about where the stack is because its responsible for all memory on the system, it has to manage and share that memory between all processes, understand what is actually in use, what has been allocated but not yet committed, etc. It has to be able to "clean" up that memory when the process terminates (regardless of how it terminated).

This is very much a case where you will rarely hit the stack limit in practice, even though the typical stack size is already "small" on most platforms (it's 1MB on Windows, 8MB on Linux, 512KB on MacOS -- for 64-bit). It is a case where the physical hardware often has specialized handling and presumes stacks will be relatively small and returns will not happen "rapidly" (where it often uses an internal portion of the register file to do mirroring of stack spills, where it tracks a limited number of return addresses so that returns are faster, and other optimizations). Where approaching the upper limits of the already "small" stack sizes can hurt performance, where there are often trivial and far more performant workarounds for scenarios where deep recursion is expected, etc.

Something being technically possible does not necessarily mean it is good to implement or support in practice, which is why most OS do not provide such built-in support (at least not in a way that makes it trivial to use/support).

[Mr0N]

If you are doing an operation that needs 1GB of stack space, the overhead of creating a new thread and Joining it will be negligible.

Well, I used one gigabyte as an example. For instance, for recursively iterating through folders and files on a disk, I think 50-100 megabytes would be more than enough. The downside of a thread is that it’s a primitive, unlike a Task, which is easier to work with. Plus, creating a new thread takes some time, ending it, and so on... It’s easier to do it without recursion.

[Mr0N]

Is it possible to allocate a new stack, then hook up a continuation thunk that would handle returning to the old one

At the hardware level, yes. But most OS don't provide such functionality because it is atypical and unnecessary. There is next to zero practical benefit to having stacks that can grow infinitely and many very real downsides ranging from perf to security and beyond.

The OS cares about where the stack is because its responsible for all memory on the system, it has to manage and share that memory between all processes, understand what is actually in use, what has been allocated but not yet committed, etc. It has to be able to "clean" up that memory when the process terminates (regardless of how it terminated).

This is very much a case where you will rarely hit the stack limit in practice, even though the typical stack size is already "small" on most platforms (it's 1MB on Windows, 8MB on Linux, 512KB on MacOS -- for 64-bit). It is a case where the physical hardware often has specialized handling and presumes stacks will be relatively small and returns will not happen "rapidly" (where it often uses an internal portion of the register file to do mirroring of stack spills, where it tracks a limited number of return addresses so that returns are faster, and other optimizations). Where approaching the upper limits of the already "small" stack sizes can hurt performance, where there are often trivial and far more performant workarounds for scenarios where deep recursion is expected, etc.

Something being technically possible does not necessarily mean it is good to implement or support in practice, which is why most OS do not provide such built-in support (at least not in a way that makes it trivial to use/support).

Well, essentially a stack is just 1 megabyte of data. Can’t you just create an array of 500 megabytes and store everything in it?

Of course, it would be better if this stack started with a minimal size and then grew, like starting with 5 megabytes, then 10 megabytes, and so on. But I’m not sure how realistic that is to implement, so it's probably better to just have a single array where this data can be stored.