Skip to content

Commit

Permalink
chore: add Expression-Based Assertions KEP (#568)
Browse files Browse the repository at this point in the history
Signed-off-by: Kumar Mallikarjuna <kumarmallikarjuna.work@gmail.com>
  • Loading branch information
kumar-mallikarjuna authored Oct 16, 2024
1 parent 7851703 commit 4c7b28a
Showing 1 changed file with 117 additions and 0 deletions.
117 changes: 117 additions & 0 deletions keps/0009-expression-based-assertions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
kep-number: 9
short-desc: This KEP discusses adding expression-based assertions to Kuttl
title: Expression-Based Assertions
authors:
- "@kumar-mallikarjuna"
owners:
- "@kumar-mallikarjuna"
- "@porridge"
creation-date: 2024-09-23
last-updated: 2024-09-23
status: provisional
---

# Expression-Based Assertions

## Summary

Add expression-based assertions to Kuttl. This can be achieved by integrating support for languages such as Common Expression Language (CEL). This would extend Kuttl's ability to perform assertions based on complex expressions that are currently not possible with the existing syntax.

## Motivation

Currently, Kuttl lacks support for complex data manipulation in assertions.
Here are a few examples where this limitation is apparent:

- Testing whether a Deployment has created _more than `n` Pods_.
- While Kuttl supports equality testing, conditional matching poses a challenge.
- Slice assertion.
- Kuttl's current method for handling slice comparisons is somewhat unclear. This is especially important when
the order of elements in the slice is not fixed, which is common in Kubernetes (see
https://github.com/kudobuilder/kuttl/issues/76#issuecomment-660944596).
- Additionally, partial assertions on slices are not possible (see https://github.com/kudobuilder/kuttl/issues/76).

These issues could be resolved by incorporating a CEL engine into Kuttl, allowing for expression
evaluation and more flexible assertions.

### Goals

1. Add a CEL engine using [github.com/google/cel-go/cel](https://github.com/google/cel-go/cel).
2. Update `TestAssert` CRD to specify CEL identifiers and evaluate expressions.
3. Support any/all assertion of CEL expressions.

### Non-Goals

1. Reading non-Kubernetes resources for assertions.
2. Reading resources from multiple clusters for assertions.

> **Note:** These goals are not in the scope of the first development of this feature but should be targetted in its later phases.
## Proposal

### CRD Changes

```yaml
apiVersion: kuttl.dev/v1beta1
kind: TestAssert
resourceRefs:
- apiVersion: apps/v1
kind: Deployment
name: coredns
namespace: kube-system
ref: coredns_deployment
- apiVersion: apps/v1
kind: Deployment
name: metrics-server
namespace: kube-system
ref: metrics_server
# Success if any expression evaluates to true
assertAny:
- celExpr: "coredns_deployment.status.readyReplicas > 1"
- celExpr: ...
- ...
# Success only if all expressions evaluate to true
assertAll:
- celExpr: "metrics_server.status.readyReplicas > 0"
- celExpr: ...
- ...
```
### Implementation
At a high level, we would use the [github.com/google/cel-go/cel](https://github.com/google/cel-go/cel) library to evaluate expressions as follows:
```go
func EvaluateExpression(resources...*unstructured.Unstructured)(ref.Val, error) {
env, err := cel.NewEnv(
cel.Variable("<id1>", cel.MapType(cel.StringType, cel.DynType)),
cel.Variable("<id2>", cel.MapType(cel.StringType, cel.DynType)),
...
)
if err != nil {
return fmt.Errorf("failed to create environment: %w", err)
}

ast, issues := env.Compile(`<expression>`)
if issues != nil && issues.Err() != nil {
return fmt.Errorf("type-check error: %s", issues.Err())
}

prg, err := env.Program(ast)
if err != nil {
return fmt.Errorf("program construction error: %w", err)
}

out, _, err := prg.Eval(map[string] interface {} {
"<id1>": resources[0].Object,
"<id2>": resources[1].Object,
...
})

if err != nil {
return fmt.Errorf("failed to evaluate program: %w", err)
}

return out, nil
}
```

0 comments on commit 4c7b28a

Please sign in to comment.