Layering encryption without performance cost

[AdrianVovk]

I'd love to hear some feedback about this and possibly get this added to the list of kernel things we want

Problem

We have various situations where it's convenient to layer encryption devices over top of each-other:

• A single encrypted / partition, which contains /home and thus your encrypted /home/<user>.home LUKS volumes
• Inside of your encrypted /home/<user>.home volume, we use fscrypt to encrypt per-app data directories (i.e. ~/.var/app/com.example.App) with per-app keys, as proposed here. This will allow us to drastically increase the security of a non-homed lock without giving up the complete security a homed-lock gives us
• probably many more

The point is: stacking encryption like this is useful. However, we cannot do this in practice, because it also brings a massive performance cost.

Inline encryption

The linux kernel supports a feature called Inline Encryption (aka blk-crypto), which is there to support hardware crypto engines that sit between the CPU and the flash chips. This is functionality available in modern UFS controllers, and potentially other storage hardware

PC platforms have OPAL (aka Self-Encrypting-Drives), but OPAL encryption isn't particularly trustworthy because it is 100% transparent to software. The OS has no say about how the encryption works: basically we just pick an algorithm, upload the volume-key, and the drive does the rest. We have no way of knowing what, exactly, the hardware/firmware is doing; is there a back door? Is it even encrypting anything? We cannot know, thus we cannot trust OPAL. Inline encryption, on hardware that supports it, is different. Yes, the crypto is implemented in hardware. But the OS is in control of the keys being used. If the OS wishes to do so, it can disable the crypto engine and read ciphertext directly off the storage medium and independently verify that everything is actually encrypted correctly. Here's what the kernel has to say about it:

Inline encryption hardware is also very different from "self-encrypting drives", such as those based on the TCG Opal or ATA Security standards. Self-encrypting drives don't provide fine-grained control of encryption and provide no way to verify the correctness of the resulting ciphertext. Inline encryption hardware provides fine-grained control of encryption, including the choice of key and initialization vector for each sector, and can be tested for correctness.

The basic idea is simple. The kernel can upload some keys and encryption settings into special keyslots on the crypto engine, then when sending writes to the storage medium the kernel can say which keyslot to use. The crypto engine will then either encrypt (write) or decrypt (read) the data at line-speed as it travels between CPU and flash chip.

In the kernel, this is implemented by attaching an encryption context to a block I/O request (a "bio"). The block device driver then appropriately handles uploading keys into keyslots, setting up the crypto engine to execute the encryption context, and then passing the data along with the right keyslot attached to it.

The kernel makes an effort to pass the blk-crypto requests through loopback files, device-mapper targets, and other layers to make sure that the actual encryption/decryption happens directly in hardware no matter how layered the storage setup is. This is simply amazing for us, as you'll soon see.

What if our block-device doesn't support blk-crypto? Well, the kernel supports software emulation of blk-crypto via the kernel's crypto API. The crypto API itself, of course, might be hardware-accelerated by the CPU. The kernel calls this blk-crypto-fallback. The performance of a single layer of blk-crypto-fallback compared to a single layer of dm-crypt is yet to be tested. Multiple layers of blk-crypto-fallback will be significantly faster than dm-crypt, of course, because blk-crypto-fallback will encrypt once and never perform double-encryption; I'll explain more in the next section.

Solution

Here's my idea: we need a new device-mapper target called dm-blk-crypto or dm-inline or something like that. It would look and feel like dm-crypt, but it would only implement blk-crypto. Basically for any bio request that it recieves, it'll check if that request has an encryption context attached; if it does, the bio is passed through unchanged, but if it doesn't then it'll attach its own encryption context.

Through this simple mechanism, we can layer dm-blk-crypto as many layers high as we want and it will never actually perform double-encryption. If an upper layer attaches an encryption context, then a lower layer will just pass the encryption right through. The lowest layer will ultimately forward the bio to a real block device, which will either have inline crypto hardware and do the encryption in hardware or use blk-crypto-fallback to do the encryption in software like dm-crypt would.

A possible concern with this idea so far is that layered encryption will not actually include all layers of encryption keys to access data. In other words, you could stack two instances of dm-blk-crypto on top of each-other, and data encrypted in the upper instance would remain readable if the lower instance is suspended and has its keys wiped from memory. In many situations, this outright does not matter. In others, this is a problem. So I propose that we have an alternative mode, where instead of completely passing through bio requests with encryption contexts applied to them, we'll instead take the existing context's key, combine it together with our own key to derive a new key that will actually be used. In situations where it doesn't matter, this behavior will be turned off and it'll just transparently pass through requests.

On its own, of course, the ability to stack dm-blk-crypto isn't particularly useful. You wouldn't stack dm-blk-crypto more than once. But it becomes very powerful when you combine it with other users of blk-crypto (like fscrypt) and the fact that blk-crypto is passed through loop devices. You can have a partition encrypted w/ blk-crypto, then put a loopback file containing a second layer of blk-crypto on top of it, or use fscrypt on top of it, and the encryption contexts will be carefully handed down by the kernel all the way to the underlying storage medium without ever double-encrypting the data.

Let's give a practical example. Here's a partition layout:

ESP /efi
	A UKI	B UKI
	A /usr	B /usr
Rootfs (dm-blk-crypto, don't mix keys) /var /etc /home user1.home (loopback) (dm-blk-crypto, mix keys) /home/user1 /home/user1/.var/app/com.example.App1 (fscrypt) /home/user1/.var/app/com.example.App2 (fscrypt) ... ...

If we weren't using blk-crypto, the files under com.example.App1 would be triple-encrypted! Once by fscrypt protecting the directory, a second time by the LUKS volume in the user1.home loopback file, and a third time by the LUKS volume in the rootfs. With blk-crypto however, fscrypt attaches an encryption context, which is passed through the dm-blk-crypto layer of user1.home. This layer mixes in its own encryption key, then passes the resulting request through the loopback file into the rootfs, which then passes the request through the rootfs's dm-blk-crypto layer unchanged into the hardware to be executed. We get the flexibility of performing multiple layers of encryption without the cost of doing so. If user1.home is suspended (with a homed lock), then com.example.App1 becomes inaccessible because the keys cannot be mixed together anymore.

As another example, let's look at a file /home/user1/Documents/Secret.pdf; a request to access this file reaches the user1.home dm-blk-crypto layer, which detects that it doesn't have an encryption context attached and so attaches its own. Next the IO request is passed through the loopback file into the rootfs, which eventually forwards it to the rootfs's dm-blk-crypto instance. This second dm-blk-crypto instance sees that there's already an encryption context attached, so it passes the request directly down into the hardware to be executed.

Prior art

Android kernels have a special device-mapper target called dm-default-key, which behave pretty much exactly I want: the idea is that dm-default-key encrypts (using blk-crypto) any bio that doesn't have an encryption context attached to it. This way, something like fscrypt will pass encryption clean through dm-default-key, directly to the hardware, thus allowing the data to be protected with fscrypt. Meanwhile, any metadata that fscrypt doesn't protect will be passed via bios that don't have any encryption attached to them, so dm-default-key attaches its own encryption settings so that the data is encrypted. dm-default-key sources in the Android kernel

Google has tried to upstream this functionality in the past (kinda... kernel patch), but it was vehemently rejected as a layering violation (comment). They gave up on this approach, and moved on to trying to add metadata encryption separately to f2fs directly (patchset). As far as I can tell, those f2fs patches went nowhere.

Later on, someone was trying to implement blk-crypto directly into dm-crypt (patchset). A discussion ensues about Android's dm-default-key, how they tried to upstream it, and how it failed. Christoph Hellwig then pointed out that Google's previous approach was rejected as a layering violation because it punched holes into dm-crypt, NOT because the idea of passing through encrpytion in the way dm-default-key does is a bad idea. Google didn't actually try to upstream dm-default-key, but instead wanted to punch holes into dm-crypt. There's the layering violation! Cristoph then goes on to explain that

In a purely inline crypto world a way to assign a key context if there is none before is a little different

Side-note, the cryptsetup maintainer is of the opinion that blk-crypto shouldn't be part of dm-crypt, and should instead be its own device-mapper target, to make the FIPS people happy, reduce the complexity of dm-crypt and cryptsetup, etc.

So TL;DR: our approach here should be palatable, or at least more palatable than some other approaches, to both the kernel folks and cryptsetup upstream