Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for disk encryption key in GCPMachine #1137

Merged
merged 1 commit into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions api/v1beta1/gcpmachine_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ type AttachedDiskSpec struct {
// Defaults to 30GB. For "local-ssd" size is always 375GB.
// +optional
Size *int64 `json:"size,omitempty"`
// EncryptionKey defines the KMS key to be used to encrypt the disk.
// +optional
EncryptionKey *CustomerEncryptionKey `json:"encryptionKey,omitempty"`
}

// IPForwarding represents the IP forwarding configuration for the GCP machine.
Expand Down Expand Up @@ -146,6 +149,72 @@ const (
HostMaintenancePolicyTerminate HostMaintenancePolicy = "Terminate"
)

// KeyType is a type for disk encryption.
type KeyType string

const (
// CustomerManagedKey (CMEK) references an encryption key stored in Google Cloud KMS.
CustomerManagedKey KeyType = "Managed"
// CustomerSuppliedKey (CSEK) specifies an encryption key to use.
CustomerSuppliedKey KeyType = "Supplied"
)

// ManagedKey is a reference to a key managed by the Cloud Key Management Service.
type ManagedKey struct {
// KMSKeyName is the name of the encryption key that is stored in Google Cloud KMS. For example:
// "kmsKeyName": "projects/kms_project_id/locations/region/keyRings/key_region/cryptoKeys/key
// +kubebuilder:validation:Required
// +kubebuilder:validation:Pattern=`projects\/[-_[A-Za-z0-9]+\/locations\/[-_[A-Za-z0-9]+\/keyRings\/[-_[A-Za-z0-9]+\/cryptoKeys\/[-_[A-Za-z0-9]+`
// +kubebuilder:validation:MaxLength=160
KMSKeyName string `json:"kmsKeyName,omitempty"`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does GCP specify any constraints for the name of the KMS key? Upper case? Lower case? Certain special characters allowed? MInimum or maximum length? All of this could be validated at admission time to prevent errors down the line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So unless I'm missing something I don't see constraints defined in the API https://pkg.go.dev/google.golang.org/api/compute/v1#CustomerEncryptionKey or in validations. I'm leery to add additional limit checks here unless its well defined.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When trying to create a key, it gives me

Key names can contain letters, numbers, underscores (_), and hyphens (-). Keys can't be renamed or deleted.

I believe that is also true of project IDs and regions so we probably can limit to that selection plus the slashes required

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for testing that @JoelSpeed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll add a check in the webhook.Validator.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the rule cost come up before or after you added the maxlength? The maxlength is an important factor in the rule cost estimations

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoelSpeed - you make a good point about kubebuilder validations/CEL vs webhook validation. I think part of the current reliance on webhook validation in CAPI is historical because CEL wasn't available for more complex validation logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rule cost error came up when using the 3 XValidation rules. With MaxLength validation plus one XValidation it did not cause an error.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was speaking with @vincepri last week and we think there's merit in having a community wide conversation about API review and CEL vs webhook validations. In a couple of weeks I have some time to put together some ideas which I'll bring to the community call for discussion

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Hopefully we can get this merged and revisit the validations after the discussion.

}

// SuppliedKey contains a key for disk encryption. Either RawKey or RSAEncryptedKey must be provided.
// +kubebuilder:validation:MinProperties=1
bfournie marked this conversation as resolved.
Show resolved Hide resolved
// +kubebuilder:validation:MaxProperties=1
type SuppliedKey struct {
bfournie marked this conversation as resolved.
Show resolved Hide resolved
bfournie marked this conversation as resolved.
Show resolved Hide resolved
// RawKey specifies a 256-bit customer-supplied encryption key, encoded in RFC 4648
// base64 to either encrypt or decrypt this resource. You can provide either the rawKey or the rsaEncryptedKey.
// For example: "rawKey": "SGVsbG8gZnJvbSBHb29nbGUgQ2xvdWQgUGxhdGZvcm0="
// +optional
RawKey []byte `json:"rawKey,omitempty"`
// RSAEncryptedKey specifies an RFC 4648 base64 encoded, RSA-wrapped 2048-bit customer-supplied encryption
// key to either encrypt or decrypt this resource. You can provide either the rawKey or the
// rsaEncryptedKey.
// For example: "rsaEncryptedKey": "ieCx/NcW06PcT7Ep1X6LUTc/hLvUDYyzSZPPVCVPTVEohpeHASqC8uw5TzyO9U+Fka9JFHi
// z0mBibXUInrC/jEk014kCK/NPjYgEMOyssZ4ZINPKxlUh2zn1bV+MCaTICrdmuSBTWlUUiFoDi
// D6PYznLwh8ZNdaheCeZ8ewEXgFQ8V+sDroLaN3Xs3MDTXQEMMoNUXMCZEIpg9Vtp9x2oe=="
// The key must meet the following requirements before you can provide it to Compute Engine:
// 1. The key is wrapped using a RSA public key certificate provided by Google.
// 2. After being wrapped, the key must be encoded in RFC 4648 base64 encoding.
// Gets the RSA public key certificate provided by Google at: https://cloud-certs.storage.googleapis.com/google-cloud-csek-ingress.pem
// +optional
RSAEncryptedKey []byte `json:"rsaEncryptedKey,omitempty"`
}

// CustomerEncryptionKey supports both Customer-Managed or Customer-Supplied encryption keys .
type CustomerEncryptionKey struct {
bfournie marked this conversation as resolved.
Show resolved Hide resolved
// KeyType is the type of encryption key. Must be either Managed, aka Customer-Managed Encryption Key (CMEK) or
// Supplied, aka Customer-Supplied EncryptionKey (CSEK).
// +kubebuilder:validation:Enum=Managed;Supplied
KeyType KeyType `json:"keyType"`
bfournie marked this conversation as resolved.
Show resolved Hide resolved
bfournie marked this conversation as resolved.
Show resolved Hide resolved
// KMSKeyServiceAccount is the service account being used for the encryption request for the given KMS key.
// If absent, the Compute Engine default service account is used. For example:
// "kmsKeyServiceAccount": "name@project_id.iam.gserviceaccount.com.
// The maximum length is based on the Service Account ID (max 30), Project (max 30), and a valid gcloud email
// suffix ("iam.gserviceaccount.com").
// +kubebuilder:validation:MaxLength=85
// +kubebuilder:validation:Pattern=`[-_[A-Za-z0-9]+@[-_[A-Za-z0-9]+.iam.gserviceaccount.com`
// +optional
KMSKeyServiceAccount *string `json:"kmsKeyServiceAccount,omitempty"`
bfournie marked this conversation as resolved.
Show resolved Hide resolved
// ManagedKey references keys managed by the Cloud Key Management Service. This should be set when KeyType is Managed.
// +optional
ManagedKey *ManagedKey `json:"managedKey,omitempty"`
// SuppliedKey provides the key used to create or manage a disk. This should be set when KeyType is Managed.
// +optional
SuppliedKey *SuppliedKey `json:"suppliedKey,omitempty"`
}

// GCPMachineSpec defines the desired state of GCPMachine.
type GCPMachineSpec struct {
// InstanceType is the type of instance to create. Example: n1.standard-2
Expand Down Expand Up @@ -252,6 +321,10 @@ type GCPMachineSpec struct {
// +kubebuilder:validation:Enum=Enabled;Disabled
// +optional
ConfidentialCompute *ConfidentialComputePolicy `json:"confidentialCompute,omitempty"`

// RootDiskEncryptionKey defines the KMS key to be used to encrypt the root disk.
// +optional
RootDiskEncryptionKey *CustomerEncryptionKey `json:"rootDiskEncryptionKey,omitempty"`
bfournie marked this conversation as resolved.
Show resolved Hide resolved
}

// MetadataItem defines a single piece of metadata associated with an instance.
Expand Down
42 changes: 41 additions & 1 deletion api/v1beta1/gcpmachine_webhook.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,11 @@ var _ webhook.Validator = &GCPMachine{}
// ValidateCreate implements webhook.Validator so a webhook will be registered for the type.
func (m *GCPMachine) ValidateCreate() (admission.Warnings, error) {
clusterlog.Info("validate create", "name", m.Name)
return nil, validateConfidentialCompute(m.Spec)

if err := validateConfidentialCompute(m.Spec); err != nil {
return nil, err
}
return nil, validateCustomerEncryptionKey(m.Spec)
}

// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type.
Expand Down Expand Up @@ -117,3 +121,39 @@ func validateConfidentialCompute(spec GCPMachineSpec) error {
}
return nil
}

func checkKeyType(key *CustomerEncryptionKey) error {
switch key.KeyType {
case CustomerManagedKey:
if key.ManagedKey == nil || key.SuppliedKey != nil {
return fmt.Errorf("CustomerEncryptionKey KeyType of Managed requires only ManagedKey to be set")
}
case CustomerSuppliedKey:
if key.SuppliedKey == nil || key.ManagedKey != nil {
return fmt.Errorf("CustomerEncryptionKey KeyType of Supplied requires only SuppliedKey to be set")
}
if len(key.SuppliedKey.RawKey) > 0 && len(key.SuppliedKey.RSAEncryptedKey) > 0 {
return fmt.Errorf("CustomerEncryptionKey KeyType of Supplied requires either RawKey or RSAEncryptedKey to be set, not both")
}
default:
return fmt.Errorf("invalid value for CustomerEncryptionKey KeyType %s", key.KeyType)
}
return nil
bfournie marked this conversation as resolved.
Show resolved Hide resolved
}

func validateCustomerEncryptionKey(spec GCPMachineSpec) error {
if spec.RootDiskEncryptionKey != nil {
if err := checkKeyType(spec.RootDiskEncryptionKey); err != nil {
return err
}
}

for _, disk := range spec.AdditionalDisks {
if disk.EncryptionKey != nil {
if err := checkKeyType(disk.EncryptionKey); err != nil {
return err
}
}
}
return nil
}
80 changes: 80 additions & 0 deletions api/v1beta1/gcpmachine_webhook_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,86 @@ func TestGCPMachine_ValidateCreate(t *testing.T) {
},
wantErr: true,
},
{
name: "GCPMachine with RootDiskEncryptionKey KeyType Managed and Managed field set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
RootDiskEncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerManagedKey,
ManagedKey: &ManagedKey{
KMSKeyName: "projects/my-project/locations/us-central1/keyRings/us-central1/cryptoKeys/some-key",
},
},
},
},
wantErr: false,
},
{
name: "GCPMachine with RootDiskEncryptionKey KeyType Managed and Managed field not set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
RootDiskEncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerManagedKey,
},
},
},
wantErr: true,
},
{
name: "GCPMachine with RootDiskEncryptionKey KeyType Supplied and Supplied field not set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
RootDiskEncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerSuppliedKey,
},
},
},
wantErr: true,
},
{
name: "GCPMachine with AdditionalDisk Encryption KeyType Managed and Managed field not set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
AdditionalDisks: []AttachedDiskSpec{
{
EncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerManagedKey,
},
},
},
},
},
wantErr: true,
},
{
name: "GCPMachine with RootDiskEncryptionKey KeyType Supplied and one Supplied field set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
RootDiskEncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerSuppliedKey,
SuppliedKey: &SuppliedKey{
RawKey: []byte("SGVsbG8gZnJvbSBHb29nbGUgQ2xvdWQgUGxhdGZvcm0="),
},
},
},
},
wantErr: false,
},
{
name: "GCPMachine with RootDiskEncryptionKey KeyType Supplied and both Supplied fields set",
GCPMachine: &GCPMachine{
Spec: GCPMachineSpec{
RootDiskEncryptionKey: &CustomerEncryptionKey{
KeyType: CustomerSuppliedKey,
SuppliedKey: &SuppliedKey{
RawKey: []byte("SGVsbG8gZnJvbSBHb29nbGUgQ2xvdWQgUGxhdGZvcm0="),
RSAEncryptedKey: []byte("SGVsbG8gZnJvbSBHb29nbGUgQ2xvdWQgUGxhdGZvcm0="),
},
},
},
},
wantErr: true,
},
}
for _, test := range tests {
test := test
Expand Down
80 changes: 80 additions & 0 deletions api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading