Is fixing/pinning memory also page locking?

[msedi]

I'm currently investigating a problem where I have to hand over page-locked memory to an async CUDA call.
For example I need to copy host to device memory using the call

cuMemcpy(IntPtr to, IntPtr from, nint bytes)

For that reason I have to pin the managed memory and hand it over to the cuMemcpy function. Since the function is blocking I can completely decide the scope of the pin block. This works well for regular managed memory and for Span<T>.

Since I'm writing a library that wraps the CUDA calls and hides the internals, I wrote a wrapper class that handles the inner workings and the cuMemcpy call is wrapped in

unsafe void SetData<T>(in ReadOnlySpan<T> data) where T : unmanaged
{
  fixed (T* ptr = data)
  {
    cuMemcpy(devptr, ptr, sizeof(T) * data.Length);
  }
}

Now comes the tricky part with async CUDA invocation. Using async calls you create a stream object (something like a monitor) that "flows" from method to method so I can execute multiple operations in different streams that run in parallel. For example I copy data from the host to the device, execute a kernel on the data and copy the data back from device to host (just in short - not working)

devptr.SetData(mem);
runKernel(devptr);
devptr.GetData();

All is running in sync one after the other. With streams the problem is that the method immediately returns and also the requirement of the method is that the memory is page-locked. For that reason CUDA provides a method to page-lock memory which takes a pinned memory handle/pointer.

cuMemHostRegister(ptr)

so first I need to pin the memory and then I also need to page-lock it.

So my first question is if pinning also means page-locking the memory?

The question I'm asking is that I want to provide a safe way to pin and page-lock memory so that for async call the user of the library has to do the following call:

static unsafe IDisposable PageLock<T>(in ReadOnlySpan<T> data) where T : unmanaged
{
  // Here is the part I cannot do with Span<T> since the lifetime of the fixed is limited
  // If I really need it I have to go with arrays because only here I can say GCHandle.Alloc(data, Pinned).
  // Which is a severe problem since the ReadOnlySpan<T> is the only common denominator because the data can come from 
  // managed and unmanaged sources (either from managed memory, memory mapped files or from managed CUDA memory. 
  // This is what currently prevent from doing this at all.
  fixed(T* ptr = data)
  {
    cuMemHostRegister(ptr);
   return // some diposable handle
  }
}

So that the final async call in the example above should look like

float[] data = new float[100];
using var stream = new CudaStream()
using var lockedmem = CudaMem.Lock(mem);
devptr.SetData(lockedmem, stream);
runKernel(devptr, stream);
devptr.GetData(stream);

Are there any ideas

1. is fixed also page locking?
2. How I can pin a Span longer than the scope of {}?

I know there is memory but it cannot be used in this case.

[DaZombieKiller]

2. How can I pin a Span longer than the scope of {}?

The short answer is that you can't, the long answer is that you can, but really shouldn't. Let that sample code serve as a caution rather than a recommendation (seriously).

Using a Span<T> when you have an async operation like this is bound to cause you trouble. If you need to support unmanaged memory, you can write a custom MemoryManager<T> implementation for a Memory<T> instance (here is an example from within the BCL and here is some additional information and guidelines).

Keep in mind that you need to ensure the MemoryManager<T> is not collected (the Span you get from the Memory will not root the MemoryManager, so you may need GC.KeepAlive calls in some places to ensure safety).

[msedi]

@DaZombieKiller: Thanks for answering. The misunderstanding that I also have with colleagues about async is that the methods are itself not async in the sense of .NET.

Regarding the memory manager the problem is that I have to manage the memory myself and I would need hundreds of thousand individual instances of the memory manager just to get some memory pinned.

Nevertheless your link brough me to the idea to work with `Action<>´ so that my solution would look like this:

    public static unsafe void CudaRunAsync<T>(in ReadOnlySpan<T> data, CudaStream stream, Action<CudaMemory<T>, CudaStream> action) where T : unmanaged
    {
        fixed (T* ptr = data)
        {
            try
            {
                var devptr = cuMemHostRegister(ptr);

                action(devptr, stream);
            }
            finally
            {
                cuMemHostUnregister(ptr);
            }
        }
    }

so that some of the examples above would look like this:

float[] data = new float[100];
using var stream = new CudaStream()

CudaRunAsync(data, stream, (mem, str)=>
{
  devptr.SetData(mem, str);
  runKernel(devptr, str);
  devptr.GetData(str);
});

which should fit all my requirements,

However, one question is still open: "Does fixing/pinning also mean page-locking?"

[teo-tsirpanis]

No. Pinning is a concept of the GC and ensures that an object's virtual address does not change. Page-locking is a concept of the operating system and ensures that a page stays in physical memory.

[msedi]

@teo-tsirpanis: Thanks for pointing this out,