Is there a way to temporarily store managed pointers in a stackalloc buffer without incurring unexpected behavior from the GC?

[operation404]

I'm working with C# in a game engine and frequently need to pass around subsets of collections of objects to different synchronous functions. I'd like to avoid making heap allocations for each call, and have investigated a few solutions. One that interested me and seemed elegant conceptually was to stackalloc a buffer and store references to the objects in it, then pass that buffer as a Span to whatever function I needed to call.

However, I know the docs state that stackalloc can only be used with unmanaged types. I've read a few descriptions of why this is the case, including this SO answer. My limited understanding is that the garbage collector needs to track all of the potential references to objects for the purposes of moving and freeing them. But stackalloc creates a value type with no fields, just a raw blob of memory with no type info, so the GC inherently can't know anything about it.

Through use of the Unsafe and MemoryMarshal classes, I've written a workaround that allocates and reinterprets a buffer on the stack:

IntPtr* ptr = stackalloc IntPtr[count];
ref T tRef = ref Unsafe.AsRef<T>(ptr); // T is some object
Span<T> buff = MemoryMarshal.CreateSpan(ref tRef, count);
... // Copy references into buff and pass and call function with it, then deallocate and return

This works as expected so far. However, I haven't used this solution extensively yet and don't know if there will be any unexpected behavior over sustained GC runs.

I know the objects won't be freed as they're still referenced by the original heap allocated collections. But I want to ensure the objects won't be moved by the GC while calling a function with these stack buffers. Is this possible?

Will the GC account for the references stored in this memory because it's now being pointed to by a Span? If not, is it possible to avoid the referenced objects from being moved using the Span with the fixed statement and pinning them? If that won't work, is there any other way to ensure they won't be moved for the duration of the function call, or is this just not possible whatsoever?

[josephmoresena]

You can try using GC.Collect() and GC.WaitForFullGCComplete(). In this example, you can see that the string references were collected.

    using System;
    using System.Runtime.InteropServices;
    using System.Runtime.CompilerServices;

    public class Program
    {
        public static void Main()
        {
            Span<IntPtr> realBuff = stackalloc IntPtr[10];
            ref String tRef = ref Unsafe.As<IntPtr, String>(ref MemoryMarshal.GetReference(realBuff));
            Span<String> buff = MemoryMarshal.CreateSpan(ref tRef, realBuff.Length);

            for(Int32 i = 0; i < buff.Length; i++)
                buff[i] = $"Index: {i} Value: {Random.Shared.NextDouble()}";

            PrintStringSpan(buff);

            GC.Collect();
            GC.WaitForFullGCComplete();

            PrintStringSpan(buff);
        }

        private static void PrintStringSpan(ReadOnlySpan<String> span) {
            foreach(ref readonly String str in span)
                Console.WriteLine(str);
        }
    }

[operation404]

Unfortunately the count is arbitrary. I can perform the filtering ahead of time to figure out how many items I need before allocating, but I never know exactly how many that will be in advance. In practice I've never needed to send more than 2-3~ dozen, and even that is quite rare, it's usually single digits. One thing I did consider was just making a decently large fixed inline array and if I ended up needing more than that I could use a heap allocated array, which should only come up in rare cases. So it's definitely a viable option and probably what I'll fall back on. I think at this point I'm just curious about exploring the possibilities.

Your example with using the GC library calls was very helpful for testing though, I haven't interacted directly with the GC before so wouldn't have thought of that. It seems that even if I wrap the stack allocated memory in a Span of a managed type, the GC still isn't aware of any references it contains. I'm guessing because the GC doesn't bother with Spans in the first place, just whatever underlying memory they're pointing to, and in this case it's the stack buffer which I already know the GC can't track.

I think my only option here would be to pin all of the objects that end up being written into the buffer so they aren't moved, and I'm not sure if that ends up being more costly than just using something like the aforementioned struct with backup heap array method, or an array pool. I'd have to benchmark it.

[josephmoresena]

Check out this approach. It may require creating buffer types as the count increases and memory optimization is needed. If this is useful to you, I can include it as part of josephmoresena/Rxmxnx.PInvoke.Extensions.

    using System;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using System.Runtime.CompilerServices;

    public delegate void ConsumeSpan<TObject>(Span<TObject?> span) where TObject : class;
    public delegate void ConsumeSpan<TObject, TArg>(Span<TObject?> span, in TArg arg) where TObject : class;

    public class Program
    {
        public struct Buffer11 : IObjectBuffer<Buffer11>
        {
            public static Int32 Capacity => 11;
        #pragma warning disable CS0169 // Field is never used
            private Object _0;
            private InlineBuffer2 _1;
            private InlineBuffer4 _3;
            private InlineBuffer4 _7;
        #pragma warning restore CS0169 // Field is never used
        }

        private static readonly Dictionary<Int32, BufferManager> managers = new()
        {
            { InlineBuffer2.Capacity, BufferManager.Create<InlineBuffer2>() },
            { InlineBuffer4.Capacity, BufferManager.Create<InlineBuffer4>() },
            { InlineBuffer8.Capacity, BufferManager.Create<InlineBuffer8>() },
            { Buffer11.Capacity, BufferManager.Create<Buffer11>() },
            { InlineBuffer16.Capacity, BufferManager.Create<InlineBuffer16>() },
            { InlineBuffer32.Capacity, BufferManager.Create<InlineBuffer32>() },
        };

        public static void Main(String[] args)
        {
            Console.Write("Size: ");
            Int32 count = Int32.Parse(Console.ReadLine()!);
            BufferManager man = Program.GetBuffer(count);
            man.WithSpan<String>(count, Program.UseStringSpan);
        }

        private static void UseStringSpan(Span<String?> buff)
        {
            for (Int32 i = 0; i < buff.Length; i++)
                buff[i] = $"Index: {i} Value: {Random.Shared.NextDouble()}";

            Program.PrintStringSpan(buff);

            Console.WriteLine("Begin GC.Collect()");

            GC.Collect();
            GC.WaitForFullGCComplete();

            Console.WriteLine("End GC.Collect()");

            Program.PrintStringSpan(buff);
        }
        private static void PrintStringSpan(ReadOnlySpan<String?> span)
        {
            foreach (ref readonly String? str in span)
                Console.WriteLine(str);
        }

        private static BufferManager GetBuffer(Int32 size)
        {
            BufferManager? result = Program.managers.GetValueOrDefault(size);
            if (result is not null) return result;
            return size switch
            {
                < 2 => Program.managers[2],
                < 4 => Program.managers[4],
                < 8 => Program.managers[8],
                < 11 => Program.managers[11],
                < 16 => Program.managers[16],
                < 32 => Program.managers[32],
                _ => throw new InvalidOperationException()
            };
        }
    }

    public interface IObjectBuffer<TBuffer> where TBuffer : struct, IObjectBuffer<TBuffer>
    {
        public static abstract Int32 Capacity { get; }

        internal static void WithSpan<TObject>(Int32 size, ConsumeSpan<TObject> action) where TObject : class
        {
            TBuffer buffer = new();
            Span<TObject?> span = IObjectBuffer<TBuffer>.GetSpan<TObject>(ref buffer, size);
            action(span);
        }
        internal static void WithSpan<TObject, TArg>(Int32 size, in TArg arg, ConsumeSpan<TObject, TArg> action)
            where TObject : class
        {
            TBuffer buffer = new();
            Span<TObject?> span = IObjectBuffer<TBuffer>.GetSpan<TObject>(ref buffer, size);
            action(span, in arg);
        }

        private static Span<TObject?> GetSpan<TObject>(ref TBuffer buffer, Int32 size) where TObject : class
        {
            if (size > TBuffer.Capacity) throw new InvalidOperationException();
            return MemoryMarshal.CreateSpan(ref Unsafe.As<TBuffer, TObject?>(ref buffer), size);
        }
    }

    public abstract class BufferManager
    {
        public abstract Int32 Capacity { get; }

        private BufferManager() { }

        public abstract void WithSpan<TObject>(Int32 size, ConsumeSpan<TObject> action) where TObject : class;
        public abstract void WithSpan<TObject, TArg>(Int32 size, in TArg arg, ConsumeSpan<TObject, TArg> action)
            where TObject : class;

        public static BufferManager Create<TBuffer>() where TBuffer : struct, IObjectBuffer<TBuffer>
            => new Generic<TBuffer>();

        private sealed class Generic<TBuffer> : BufferManager where TBuffer : struct, IObjectBuffer<TBuffer>
        {
            public override Int32 Capacity => TBuffer.Capacity;

            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            public override void WithSpan<TObject>(Int32 size, ConsumeSpan<TObject> action)
                => IObjectBuffer<TBuffer>.WithSpan(size, action);

            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            public override void WithSpan<TObject, TArg>(Int32 size, in TArg arg, ConsumeSpan<TObject, TArg> action)
                => IObjectBuffer<TBuffer>.WithSpan(size, arg, action);
        }
    }

    [InlineArray(2)]
    public struct InlineBuffer2 : IObjectBuffer<InlineBuffer2>
    {
        public static Int32 Capacity => 2;
        private Object _obj;
    }
    [InlineArray(4)]
    public struct InlineBuffer4 : IObjectBuffer<InlineBuffer4>
    {
        public static Int32 Capacity => 4;
        private Object _obj;
    }
    [InlineArray(8)]
    public struct InlineBuffer8 : IObjectBuffer<InlineBuffer8>
    {
        public static Int32 Capacity => 8;
        private Object _obj;
    }
    [InlineArray(16)]
    public struct InlineBuffer16 : IObjectBuffer<InlineBuffer16>
    {
        public static Int32 Capacity => 16;
        private Object _obj;
    }
    [InlineArray(32)]
    public struct InlineBuffer32 : IObjectBuffer<InlineBuffer32>
    {
        public static Int32 Capacity => 32;
        private Object _obj;
    }

[operation404]

Hm, something like this is definitely a good direction for me, though I'll need to check how viable it would be for me to cache delegates or if I'm going to end up allocating a bunch of those instead.

[josephmoresena]

Sure! I would appreciate it if you could provide feedback on the performance. I actually just wrote that approach based on the challenges I've faced adapting to NativeAOT. The only requirement is .NET 8+.

[operation404]

I ended up writing some quick benchmarking tests using a few of the methods I'd considered. Nothing crazy, just some speed and memory tests using BenchmarkDotNet.

Compared against a baseline of allocating a new array of the exact required length each time and discarding it to be garbage collected after calling the desired function. Tested with subsets ranging from 10-100 elements. Using ArrayPool<T> was around 1.2-1.5x~ baseline, performance being better as required array length increased.

Using InlineArray structs and the unpinned stackalloc buffer approach were practically identical performance wise, both were around 0.85x~ baseline while also not allocating anything to heap. Figures they'd be about the same, they're both just allocating a block of memory on the stack. As for stackalloc though, it's not safe to use those references without pinning. And pinning each object made it considerably slower, multiple times slower than the baseline even when only pinning a small amount of objects. So while that works, it's not practical whatsoever.

I haven't had the opportunity yet to test how your specific implementation utilizing delegates performs in my actual project, but the idea behind it is sound. I think my final approach is going to be using inline array structs for smaller amounts of items, and fall back to an array pool or regular single use array.

[EgorBo]

There is nothing wrong with just storing gc references on stack (not sure why stackalloc doesn't support managed types - there are proposals like #33960 and related links) - you can define a struct with gc fields and put it on stack.

I know the objects won't be freed as they're still referenced by the original heap allocated collections. But I want to ensure the objects won't be moved by the GC while calling a function with these stack buffers. Is this possible?

If you don't want them to be moved, you have to pin every object in your buffer, otherwise GC may change pointers in your buffer while native code is doing something with them (see e.g. GCHandle)

[EgorBo]

As far as I know, an array of ReferenceType elements cannot be pinned. Try calling Object[].AsMemory().Pin(), and you'll get a nice exception, the same for GCHandle.Alloc(Object[], GCHandleType.Pinned).

it's possible, you just need to pin all elements separately.

[operation404]

It's being passed to a mix of C# and native code, though I'm not copying anything and also know that there will always be references to the objects being used in the original heap allocated arrays too, if that matters.

[josephmoresena]

It's not about pinning the object in memory (because a managed reference won't be very useful in an unmanaged context) but rather about keeping it in memory. It seems that my approach with InlineArray works. The buffer is fixed in memory because it's a structure created on the stack; however, unlike the original approach, the buffer is typed and keeps the references alive, preventing the GC from collecting the objects.

[EgorBo]

It's being passed to a mix of C# and native code, though I'm not copying anything and also know that there will always be references to the objects being used in the original heap allocated arrays too, if that matters.

if you don't access any of the elements in the native code - then it's fine (including buffer copying). Although, you original snippet to re-interpretate IntPtr* as string* smells like UB.

e.g. if you enable SkipLocalsInit assembly-wide, your stackalloc will start producing garbage values which GC might take as valid GC references and crash.

[operation404]

This ended up being what worked. I was able to guarantee that the references stayed valid if I pinned the objects they referred to. Unfortunately, all that pinning was pretty expensive and I don't actually need the objects themselves to stay fixed, just the references to stay valid. While I won't be using this approach for my project, it was still interesting to learn about.