Skip to content

Commit

Permalink
Render policies (#910)
Browse files Browse the repository at this point in the history
render policies

Signed-off-by: Chip Zoller <chipzoller@gmail.com>
  • Loading branch information
chipzoller authored Jul 9, 2023
1 parent 043e301 commit aae4658
Show file tree
Hide file tree
Showing 33 changed files with 549 additions and 132 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ spec:
any:
- resources:
kinds:
- Application
- Application
validate:
message: >-
`spec.source.path` OR `spec.source.chart` should be specified but never both.
Expand All @@ -55,7 +55,7 @@ spec:
any:
- resources:
kinds:
- Application
- Application
validate:
message: >-
`spec.destination.server` OR `spec.destination.name` should be specified but never both.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: "Add CAST AI Removal Disabled"
category: CAST AI
version:
subject: Job, CronJob
policyType: "mutate"
description: >
CAST AI will not downscale a node that includes a pod with the autoscaling.cast.ai/removal-disabled="true" label on it, this protects sensitive workloads from being evicted and can be attributed to any pod to protect against unwanted downscaling. This policy will mutate jobs and cronjobs to add the removal-disabled label to protect against eviction.
---

## Policy Definition
<a href="https://github.com/kyverno/policies/raw/main//castai/add-castai-removal-disabled/add-castai-removal-disabled.yaml" target="-blank">/castai/add-castai-removal-disabled/add-castai-removal-disabled.yaml</a>

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-castai-removal-disabled
annotations:
policies.kyverno.io/title: Add CAST AI Removal Disabled
policies.kyverno.io/category: CAST AI
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: "Job, CronJob"
kyverno.io/kyverno-version: "1.9"
kyverno.io/kubernetes-version: "1.25"
policies.kyverno.io/description: >-
CAST AI will not downscale a node that includes a pod with the
autoscaling.cast.ai/removal-disabled="true" label on it, this protects
sensitive workloads from being evicted and can be attributed to any pod to
protect against unwanted downscaling. This policy will mutate jobs and
cronjobs to add the removal-disabled label to protect against eviction.
spec:
rules:
- name: do-not-evict-jobs
match:
any:
- resources:
kinds:
- Job
mutate:
patchStrategicMerge:
spec:
template:
metadata:
labels:
autoscaling.cast.ai/removal-disabled: "true"
- name: do-not-evict-cronjobs
match:
any:
- resources:
kinds:
- CronJob
mutate:
patchStrategicMerge:
spec:
jobTemplate:
spec:
template:
metadata:
labels:
autoscaling.cast.ai/removal-disabled: "true"
```
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ spec:
any:
- resources:
kinds:
- ExternalSecret
- external-secrets.io/v1beta1/ExternalSecret
mutate:
foreach:
- list: "request.object.spec.data"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: "Add priority class for DaemonSets to help Karpenter."
category: Karpenter
version: 1.6.0
subject: DaemonSet
policyType: "mutate"
description: >
When a DaemonSet is added to a cluster every node will get a new pod. There may not be enough room for this on every node. Karpenter cannot provision extra nodes just for the DaemonSet because the new pods are not scheduled the way regular pods are. It would require parallel scheduling logic that is not proper to Kubernetes. Therefore, eviction of regular pods should happen instead. This can be achieved with the priority class system-node-critical.
---

## Policy Definition
<a href="https://github.com/kyverno/policies/raw/main//karpenter/add-karpenter-daemonset-priority-class/add-karpenter-daemonset-priority-class.yaml" target="-blank">/karpenter/add-karpenter-daemonset-priority-class/add-karpenter-daemonset-priority-class.yaml</a>

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-karpenter-daemonset-priority-class
annotations:
policies.kyverno.io/title: Add priority class for DaemonSets to help Karpenter.
policies.kyverno.io/subject: DaemonSet
policies.kyverno.io/category: Karpenter
policies.kyverno.io/severity: medium
kyverno.io/kyverno-version: 1.9.3
kyverno.io/kubernetes-version: "1.26"
policies.kyverno.io/minversion: 1.6.0
policies.kyverno.io/description: >-
When a DaemonSet is added to a cluster every node will get a new pod. There may not be
enough room for this on every node. Karpenter cannot provision extra nodes just for the
DaemonSet because the new pods are not scheduled the way regular pods are. It would require
parallel scheduling logic that is not proper to Kubernetes. Therefore, eviction of regular
pods should happen instead. This can be achieved with the priority class system-node-critical.
spec:
rules:
- name: add-karpenter-daemonset-priority-class
match:
any:
- resources:
kinds:
- DaemonSet
mutate:
patchStrategicMerge:
spec:
template:
spec:
priorityClassName: system-node-critical

```
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
title: "Set non-CPU limits for pods to work well with Karpenter."
category: Karpenter, EKS Best Practices
version: 1.6.0
subject: Pod
policyType: "mutate"
description: >
For correct node provisioning Karpenter should know exactly what the non-CPU resources are that the pods will need. Otherwise Karpenter will put as many pods on a node as possible, which may lead to memory pressure on nodes. This is especially important in consolidation mode.
---

## Policy Definition
<a href="https://github.com/kyverno/policies/raw/main//karpenter/set-karpenter-non-cpu-limits/set-karpenter-non-cpu-limits.yaml" target="-blank">/karpenter/set-karpenter-non-cpu-limits/set-karpenter-non-cpu-limits.yaml</a>

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: set-karpenter-non-cpu-limits
annotations:
policies.kyverno.io/title: Set non-CPU limits for pods to work well with Karpenter.
policies.kyverno.io/subject: Pod
policies.kyverno.io/category: Karpenter, EKS Best Practices
policies.kyverno.io/severity: medium
kyverno.io/kyverno-version: 1.9.3
kyverno.io/kubernetes-version: "1.26"
policies.kyverno.io/minversion: 1.6.0
policies.kyverno.io/description: >-
For correct node provisioning Karpenter should know exactly what the non-CPU resources are
that the pods will need. Otherwise Karpenter will put as many pods on a node as possible,
which may lead to memory pressure on nodes. This is especially important in consolidation
mode.
spec:
rules:
- name: set-ephemeral-storage
match:
any:
- resources:
kinds:
- Pod
mutate:
foreach:
- list: request.object.spec.containers
patchStrategicMerge:
spec:
containers:
- (name): "{{element.name}}"
resources:
limits:
+(ephemeral-storage): "{{element.resources.limits.\"ephemeral-storage\" || element.resources.requests.\"ephemeral-storage\"}}"
requests:
ephemeral-storage: "{{element.resources.limits.\"ephemeral-storage\" || element.resources.requests.\"ephemeral-storage\"}}"
- name: set-memory
match:
any:
- resources:
kinds:
- Pod
mutate:
foreach:
- list: request.object.spec.containers
patchStrategicMerge:
spec:
containers:
- (name): "{{element.name}}"
resources:
limits:
+(memory): "{{element.resources.limits.memory || element.resources.requests.memory}}"
requests:
memory: "{{element.resources.limits.memory || element.resources.requests.memory}}"

```
6 changes: 3 additions & 3 deletions content/en/policies/kubevirt/add-services/add-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@ spec:
spec:
ports:
- protocol: TCP
targetPort: 22
selector:
kubevirt.io/domain: "{{request.object.metadata.name}}"
port: 22
selector:
kubevirt.io/domain: "{{request.object.metadata.name}}"
type: ClusterIP

```
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ version:
subject: VirtualMachine
policyType: "validate"
description: >
Check VirtualMachines and validate that they are not using a domain template (`.spec.template.domain`) but instead are based on instaceTypes and preferences only.
Check VirtualMachines and validate that they are using an instance type and preference.
---

## Policy Definition
Expand All @@ -21,25 +21,25 @@ metadata:
policies.kyverno.io/category: KubeVirt
policies.kyverno.io/subject: VirtualMachine
policies.kyverno.io/description: >-
Check VirtualMachines and validate that they are not using a domain template (`.spec.template.domain`)
but instead are based on instaceTypes and preferences only.
Check VirtualMachines and validate that they are using an instance type and preference.
kyverno.io/kyverno-version: "1.8.0-rc2"
kyverno.io/kubernetes-version: "1.24-1.25"
spec:
validationFailureAction: enforce
rules:
- name: k6t-dont-allow-domain-template
- name: k6t-ensure-instance-type-and-preference
match:
any:
- resources:
kinds:
- VirtualMachine
validate:
message: "VirtualMachines must only use instanceTypes and preferences, a domain template is not allowed."
message: "VirtualMachines must use instance types and preferences"
pattern:
spec:
template:
spec:
domain: null
instancetype:
name: ?*
preference:
name: ?*

```
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ version: 1.6.0
subject: APIServer
policyType: "validate"
description: >
Encrption at rest is a security best practice. This policy ensures encryption is enabled for etcd in OpenShift clusters.
Encryption at rest is a security best practice. This policy ensures encryption is enabled for etcd in OpenShift clusters.
---

## Policy Definition
Expand All @@ -25,7 +25,7 @@ metadata:
kyverno.io/kubernetes-version: "1.20"
policies.kyverno.io/subject: APIServer
policies.kyverno.io/description: >-
Encrption at rest is a security best practice. This policy ensures encryption is enabled for etcd in OpenShift clusters.
Encryption at rest is a security best practice. This policy ensures encryption is enabled for etcd in OpenShift clusters.
spec:
validationFailureAction: enforce
background: true
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: "Inject Infrastructure Name"
category: OpenShift
version: 1.10.0
subject: MachineSet
policyType: "mutate"
description: >
A required component of a MachineSet is the infrastructure name which is a random string created in a separate resource. It can be tedious or impossible to know this for each MachineSet created. This policy fetches the value of the infrastructure name from the Cluster resource and replaces all instances of TEMPLATE in a MachineSet with that name.
---

## Policy Definition
<a href="https://github.com/kyverno/policies/raw/main//openshift/inject-infrastructurename/inject-infrastructurename.yaml" target="-blank">/openshift/inject-infrastructurename/inject-infrastructurename.yaml</a>

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: inject-infrastructurename
annotations:
policies.kyverno.io/title: Inject Infrastructure Name
policies.kyverno.io/category: OpenShift
policies.kyverno.io/severity: medium
kyverno.io/kyverno-version: 1.10.0
policies.kyverno.io/minversion: 1.10.0
kyverno.io/kubernetes-version: "1.26"
policies.kyverno.io/subject: MachineSet
policies.kyverno.io/description: >-
A required component of a MachineSet is the infrastructure name which is a random string
created in a separate resource. It can be tedious or impossible to know this for each
MachineSet created. This policy fetches the value of the infrastructure name from the
Cluster resource and replaces all instances of TEMPLATE in a MachineSet with that name.
spec:
schemaValidation: false
rules:
- name: replace-template
match:
any:
- resources:
kinds:
- machine.openshift.io/v1beta1/MachineSet
operations:
- CREATE
context:
- name: cluster
apiCall:
urlPath: /apis/config.openshift.io/v1/infrastructures/cluster
- name: infraid
variable:
jmesPath: cluster.status.infrastructureName
mutate:
patchesJson6902: |-
- op: replace
path: /metadata
value: {{ replace_all(to_string(request.object.metadata),'TEMPLATE', infraid) }}
- op: replace
path: /spec
value: {{ replace_all(to_string(request.object.spec),'TEMPLATE', infraid) }}
```
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ spec:
- Pod
preconditions:
all:
- key: '{{request.object.metadata.annotations."inject-certs"}}'
- key: '{{request.object.metadata.annotations."inject-certs" || ""}}'
operator: Equals
value: enabled
- key: "{{request.operation || 'BACKGROUND'}}"
Expand Down
Loading

0 comments on commit aae4658

Please sign in to comment.