-
Notifications
You must be signed in to change notification settings - Fork 154
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
render policies Signed-off-by: Chip Zoller <chipzoller@gmail.com>
- Loading branch information
1 parent
043e301
commit aae4658
Showing
33 changed files
with
549 additions
and
132 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
62 changes: 62 additions & 0 deletions
62
...t/en/policies/castai/add-castai-removal-disabled/add-castai-removal-disabled.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
--- | ||
title: "Add CAST AI Removal Disabled" | ||
category: CAST AI | ||
version: | ||
subject: Job, CronJob | ||
policyType: "mutate" | ||
description: > | ||
CAST AI will not downscale a node that includes a pod with the autoscaling.cast.ai/removal-disabled="true" label on it, this protects sensitive workloads from being evicted and can be attributed to any pod to protect against unwanted downscaling. This policy will mutate jobs and cronjobs to add the removal-disabled label to protect against eviction. | ||
--- | ||
|
||
## Policy Definition | ||
<a href="https://github.com/kyverno/policies/raw/main//castai/add-castai-removal-disabled/add-castai-removal-disabled.yaml" target="-blank">/castai/add-castai-removal-disabled/add-castai-removal-disabled.yaml</a> | ||
|
||
```yaml | ||
apiVersion: kyverno.io/v1 | ||
kind: ClusterPolicy | ||
metadata: | ||
name: add-castai-removal-disabled | ||
annotations: | ||
policies.kyverno.io/title: Add CAST AI Removal Disabled | ||
policies.kyverno.io/category: CAST AI | ||
policies.kyverno.io/severity: medium | ||
policies.kyverno.io/subject: "Job, CronJob" | ||
kyverno.io/kyverno-version: "1.9" | ||
kyverno.io/kubernetes-version: "1.25" | ||
policies.kyverno.io/description: >- | ||
CAST AI will not downscale a node that includes a pod with the | ||
autoscaling.cast.ai/removal-disabled="true" label on it, this protects | ||
sensitive workloads from being evicted and can be attributed to any pod to | ||
protect against unwanted downscaling. This policy will mutate jobs and | ||
cronjobs to add the removal-disabled label to protect against eviction. | ||
spec: | ||
rules: | ||
- name: do-not-evict-jobs | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- Job | ||
mutate: | ||
patchStrategicMerge: | ||
spec: | ||
template: | ||
metadata: | ||
labels: | ||
autoscaling.cast.ai/removal-disabled: "true" | ||
- name: do-not-evict-cronjobs | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- CronJob | ||
mutate: | ||
patchStrategicMerge: | ||
spec: | ||
jobTemplate: | ||
spec: | ||
template: | ||
metadata: | ||
labels: | ||
autoscaling.cast.ai/removal-disabled: "true" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
...dd-karpenter-daemonset-priority-class/add-karpenter-daemonset-priority-class.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
title: "Add priority class for DaemonSets to help Karpenter." | ||
category: Karpenter | ||
version: 1.6.0 | ||
subject: DaemonSet | ||
policyType: "mutate" | ||
description: > | ||
When a DaemonSet is added to a cluster every node will get a new pod. There may not be enough room for this on every node. Karpenter cannot provision extra nodes just for the DaemonSet because the new pods are not scheduled the way regular pods are. It would require parallel scheduling logic that is not proper to Kubernetes. Therefore, eviction of regular pods should happen instead. This can be achieved with the priority class system-node-critical. | ||
--- | ||
|
||
## Policy Definition | ||
<a href="https://github.com/kyverno/policies/raw/main//karpenter/add-karpenter-daemonset-priority-class/add-karpenter-daemonset-priority-class.yaml" target="-blank">/karpenter/add-karpenter-daemonset-priority-class/add-karpenter-daemonset-priority-class.yaml</a> | ||
|
||
```yaml | ||
apiVersion: kyverno.io/v1 | ||
kind: ClusterPolicy | ||
metadata: | ||
name: add-karpenter-daemonset-priority-class | ||
annotations: | ||
policies.kyverno.io/title: Add priority class for DaemonSets to help Karpenter. | ||
policies.kyverno.io/subject: DaemonSet | ||
policies.kyverno.io/category: Karpenter | ||
policies.kyverno.io/severity: medium | ||
kyverno.io/kyverno-version: 1.9.3 | ||
kyverno.io/kubernetes-version: "1.26" | ||
policies.kyverno.io/minversion: 1.6.0 | ||
policies.kyverno.io/description: >- | ||
When a DaemonSet is added to a cluster every node will get a new pod. There may not be | ||
enough room for this on every node. Karpenter cannot provision extra nodes just for the | ||
DaemonSet because the new pods are not scheduled the way regular pods are. It would require | ||
parallel scheduling logic that is not proper to Kubernetes. Therefore, eviction of regular | ||
pods should happen instead. This can be achieved with the priority class system-node-critical. | ||
spec: | ||
rules: | ||
- name: add-karpenter-daemonset-priority-class | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- DaemonSet | ||
mutate: | ||
patchStrategicMerge: | ||
spec: | ||
template: | ||
spec: | ||
priorityClassName: system-node-critical | ||
|
||
``` |
71 changes: 71 additions & 0 deletions
71
...policies/karpenter/set-karpenter-non-cpu-limits/set-karpenter-non-cpu-limits.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
--- | ||
title: "Set non-CPU limits for pods to work well with Karpenter." | ||
category: Karpenter, EKS Best Practices | ||
version: 1.6.0 | ||
subject: Pod | ||
policyType: "mutate" | ||
description: > | ||
For correct node provisioning Karpenter should know exactly what the non-CPU resources are that the pods will need. Otherwise Karpenter will put as many pods on a node as possible, which may lead to memory pressure on nodes. This is especially important in consolidation mode. | ||
--- | ||
|
||
## Policy Definition | ||
<a href="https://github.com/kyverno/policies/raw/main//karpenter/set-karpenter-non-cpu-limits/set-karpenter-non-cpu-limits.yaml" target="-blank">/karpenter/set-karpenter-non-cpu-limits/set-karpenter-non-cpu-limits.yaml</a> | ||
|
||
```yaml | ||
apiVersion: kyverno.io/v1 | ||
kind: ClusterPolicy | ||
metadata: | ||
name: set-karpenter-non-cpu-limits | ||
annotations: | ||
policies.kyverno.io/title: Set non-CPU limits for pods to work well with Karpenter. | ||
policies.kyverno.io/subject: Pod | ||
policies.kyverno.io/category: Karpenter, EKS Best Practices | ||
policies.kyverno.io/severity: medium | ||
kyverno.io/kyverno-version: 1.9.3 | ||
kyverno.io/kubernetes-version: "1.26" | ||
policies.kyverno.io/minversion: 1.6.0 | ||
policies.kyverno.io/description: >- | ||
For correct node provisioning Karpenter should know exactly what the non-CPU resources are | ||
that the pods will need. Otherwise Karpenter will put as many pods on a node as possible, | ||
which may lead to memory pressure on nodes. This is especially important in consolidation | ||
mode. | ||
spec: | ||
rules: | ||
- name: set-ephemeral-storage | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- Pod | ||
mutate: | ||
foreach: | ||
- list: request.object.spec.containers | ||
patchStrategicMerge: | ||
spec: | ||
containers: | ||
- (name): "{{element.name}}" | ||
resources: | ||
limits: | ||
+(ephemeral-storage): "{{element.resources.limits.\"ephemeral-storage\" || element.resources.requests.\"ephemeral-storage\"}}" | ||
requests: | ||
ephemeral-storage: "{{element.resources.limits.\"ephemeral-storage\" || element.resources.requests.\"ephemeral-storage\"}}" | ||
- name: set-memory | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- Pod | ||
mutate: | ||
foreach: | ||
- list: request.object.spec.containers | ||
patchStrategicMerge: | ||
spec: | ||
containers: | ||
- (name): "{{element.name}}" | ||
resources: | ||
limits: | ||
+(memory): "{{element.resources.limits.memory || element.resources.requests.memory}}" | ||
requests: | ||
memory: "{{element.resources.limits.memory || element.resources.requests.memory}}" | ||
|
||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59 changes: 59 additions & 0 deletions
59
...nt/en/policies/openshift/inject-infrastructurename/inject-infrastructurename.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
--- | ||
title: "Inject Infrastructure Name" | ||
category: OpenShift | ||
version: 1.10.0 | ||
subject: MachineSet | ||
policyType: "mutate" | ||
description: > | ||
A required component of a MachineSet is the infrastructure name which is a random string created in a separate resource. It can be tedious or impossible to know this for each MachineSet created. This policy fetches the value of the infrastructure name from the Cluster resource and replaces all instances of TEMPLATE in a MachineSet with that name. | ||
--- | ||
|
||
## Policy Definition | ||
<a href="https://github.com/kyverno/policies/raw/main//openshift/inject-infrastructurename/inject-infrastructurename.yaml" target="-blank">/openshift/inject-infrastructurename/inject-infrastructurename.yaml</a> | ||
|
||
```yaml | ||
apiVersion: kyverno.io/v1 | ||
kind: ClusterPolicy | ||
metadata: | ||
name: inject-infrastructurename | ||
annotations: | ||
policies.kyverno.io/title: Inject Infrastructure Name | ||
policies.kyverno.io/category: OpenShift | ||
policies.kyverno.io/severity: medium | ||
kyverno.io/kyverno-version: 1.10.0 | ||
policies.kyverno.io/minversion: 1.10.0 | ||
kyverno.io/kubernetes-version: "1.26" | ||
policies.kyverno.io/subject: MachineSet | ||
policies.kyverno.io/description: >- | ||
A required component of a MachineSet is the infrastructure name which is a random string | ||
created in a separate resource. It can be tedious or impossible to know this for each | ||
MachineSet created. This policy fetches the value of the infrastructure name from the | ||
Cluster resource and replaces all instances of TEMPLATE in a MachineSet with that name. | ||
spec: | ||
schemaValidation: false | ||
rules: | ||
- name: replace-template | ||
match: | ||
any: | ||
- resources: | ||
kinds: | ||
- machine.openshift.io/v1beta1/MachineSet | ||
operations: | ||
- CREATE | ||
context: | ||
- name: cluster | ||
apiCall: | ||
urlPath: /apis/config.openshift.io/v1/infrastructures/cluster | ||
- name: infraid | ||
variable: | ||
jmesPath: cluster.status.infrastructureName | ||
mutate: | ||
patchesJson6902: |- | ||
- op: replace | ||
path: /metadata | ||
value: {{ replace_all(to_string(request.object.metadata),'TEMPLATE', infraid) }} | ||
- op: replace | ||
path: /spec | ||
value: {{ replace_all(to_string(request.object.spec),'TEMPLATE', infraid) }} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.