Can Array.Clear() be used to zero out sensitive byte arrays?

[samuel-lucas6]

Can Array.Clear() be used to zero out sensitive byte arrays (e.g. encryption keys)? If not, then how would you go about zeroing sensitive data in .NET/.NET Core?

This question has been asked on StackOverflow before, but the answers aren't particularly helpful (e.g. ProtectedMemory is only available in .NET Framework and SecureString is not recommended). The 'official' answer doesn't seem to be documented.

[bartonjs]

Currently, none of C#, F#, or VB.NET permit the compiler to drop memory writes that it knows no one can read (from a purely deterministic functional flow), so it doesn't have the problem of memset-0+free in C.

Likewise, we don't permit the JIT to skip the writes in similar cases.

We added CryptographicOperations.ZeroMemory to .NET Core 2.1 just so that callers who want to guarantee that the memory write happens have a way to visibly assert that. Like Windows SecureZeroMemory we'll guarantee that no matter what compiler optimizations and runtime optimizations are permitted that we will forcibly clear the target memory.

So... Array.Clear works. Except for when it doesn't. If you are going to fill an array with secrets then you want to have it pinned during the entire time that it contains a secret so that GC compaction won't kick in and make a copy.

byte[] passwordArray = Encoding.UTF8.GetBytes("password");
// break here, go to memory view, see where passwordArray's contents currently live.  Stay there.

// do a lot of things which allocate small objects
Array.Clear(passwordArray);
// break here.  Your memory view might not show zeros.  Open another memory view, look at passwordArray, it's zero.
// what happened? GC compaction decided it would be more convenient of passwordArray lived somewhere else, so it copied it.

A better version of this would be

// Don't have passwords in System.String if you can help it.  You can't clear them.
// If you cheat and clear them by pointer you might crash the runtime, especially in future versions.
// Demonstrative example only.
// Yay for weasel clauses?
string password = "password";
Encoding encoding = Encoding.UTF8;
byte[] passwordArray = new byte[encoding.GetMaxByteCount(password)];

fixed (byte* ignored = passwordArray)
{
    int written = encoding.GetBytes(password, passwordArray);
    Span<byte> passwordSpan = passwordArray.AsSpan(0, written);
    // same as before
    CryptograhicOperations.ZeroMemory(passwordSpan); // Or passwordSpan.Clear().  Whatever.
}

This time there aren't any copies, so it clears as expected, because it was pinned.

You can avoid using fixed with GCHandle, or you can use fancy new API to ask nicely that the array be allocated as pinned:

// Don't have passwords in System.String if you can help it.  You can't clear them.
// If you cheat and clear them by pointer you might crash the runtime, especially in future versions.
// Demonstrative example only.
// Yay for weasel clauses?
string password = "password";
Encoding encoding = Encoding.UTF8;
byte[] passwordArray = GC.AllocateArray<byte>(encoding.GetMaxByteCount(password), pinned: true);
int written = encoding.GetBytes(password, passwordArray);
Span<byte> passwordSpan = passwordArray.AsSpan(0, written);
// same as before
CryptograhicOperations.ZeroMemory(passwordSpan); // Or passwordSpan.Clear().  Whatever.
}

That's probably nicer than fixed or GCHandle since the GC has the option of allocating the array somewhere other than the current Gen0 heap, so it won't end up with (as much) unmovable memory during compaction and subsequent Gen0 allocations.

[samuel-lucas6]

Huge thanks! That is extremely helpful. I have never heard of CryptographicOperations.ZeroMemory. I will definitely start using that and pinning arrays.

[samuel-lucas6]

@bartonjs Just to double check, how does pinning work when it comes to functions/return statements? For example:

private static byte[] GetPasswordBytes(char[] password)
{
    // Pin the passwordBytes
    byte[] passwordBytes = GC.AllocateArray<byte>(Encoding.UTF8.GetMaxByteCount(password.Length), pinned: true);
    passwordBytes = Encoding.UTF8.GetBytes(password);
    Arrays.ZeroMemory(password);
    return passwordBytes;
}

private static void UsePassword(char[] password)
{
    // Is this array pinned?
    byte[] passwordBytes = GetPasswordBytes(password);
    // Use password here...
    CryptograhicOperations.ZeroMemory(passwordBytes);
}

Also is it problematic to just use a return statement like so:

public static byte[] KeyDerivation(byte[] inputKeyingMaterial, byte[] salt, int outputLength)
{
    return GenericHash.HashSaltPersonal(message: Array.Empty<byte>(), inputKeyingMaterial, salt, _personal, outputLength);
}

Should the result of the above actually be stored in a pinned byte array before being returned?

My only concern with all of this pinning is that it makes the code less readable, even with the new API. I'm also getting a lot of IDE0059/Unnecessary assignment messages as a result of using GC.AllocateArray.

Finally, I don't know who to give feedback to about the documentation, but I think it would beneficial if there was an explanation that arrays need to be pinned to prevent copies on the CrytographicOperations.ZeroMemory page. That function could also probably do with being advertised more because I've only ever seen people using Array.Clear.

[PathogenDavid]

how does pinning work when it comes to functions/return statements?

When you allocate an array with GC.AllocateArray with pinned set to true, it is allocated on the pinned object heap and is pinned for its entire lifetime.

passwordBytes = Encoding.UTF8.GetBytes(password);

You're overwriting the pinned array you just allocated here with the unpinned one automatically allocated by GetBytes. (Which is why you're getting those warnings.)

You need to use one of the overloads which receives an output buffer, such as Encoding.GetBytes(ReadOnlySpan<char>, Span<byte>).