Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditional rendering using CEL #84

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
04c3eb7
evaluate expressions with cel
stevendborrelli Oct 23, 2023
2ca7c04
use new sdk fork module
stevendborrelli Oct 23, 2023
8c4eb9c
fix linter errors
stevendborrelli Oct 23, 2023
23bdd38
gci import update
stevendborrelli Oct 23, 2023
1b5ba1c
add test for conditionals
stevendborrelli Oct 24, 2023
3a30fd5
improve type error message
stevendborrelli Oct 24, 2023
e6b16ed
update examples with reand nop provider
stevendborrelli Oct 25, 2023
155a4b3
update go.mod for function-sdk-go updates
stevendborrelli Oct 25, 2023
1feed87
feat: :wrench: eval CEL expression for each composed resource
Nov 13, 2023
a36244b
Merge pull request #1 from jcooklin/pr/stevendborrelli/26
stevendborrelli Nov 14, 2023
0b35fb6
merge upstream main
stevendborrelli Nov 14, 2023
f4bba8d
update repo location
stevendborrelli Nov 14, 2023
20b2fce
lint fixes
stevendborrelli Nov 14, 2023
a0618e5
lint fixes
stevendborrelli Nov 14, 2023
f221df1
Merge pull request #2 from stevendborrelli/cel
stevendborrelli Nov 14, 2023
6c1285a
change fields from condition/expression to condition
stevendborrelli Nov 14, 2023
ea49546
Merge pull request #3 from stevendborrelli/refactor-conditional
stevendborrelli Nov 14, 2023
02b501d
initial push workflow
stevendborrelli Nov 14, 2023
c5fc464
fix package path
stevendborrelli Nov 14, 2023
755c911
use crossplane xpkg login
stevendborrelli Nov 14, 2023
033c1b5
Merge pull request #4 from stevendborrelli/github-actions
stevendborrelli Nov 14, 2023
aea859c
update readme
stevendborrelli Nov 14, 2023
bcd58de
update readme
stevendborrelli Nov 14, 2023
d89a56c
add examples
stevendborrelli Nov 14, 2023
d36f0f1
update readme
stevendborrelli Nov 14, 2023
535f4fc
Merge pull request #5 from stevendborrelli/0.3.0-release
stevendborrelli Nov 14, 2023
de4e713
update readme
stevendborrelli Nov 14, 2023
e6a26c6
Merge pull request #6 from stevendborrelli/0.3.0-release
stevendborrelli Nov 14, 2023
35d96b4
Merge branch 'dev/patch-and-order' into patch-ordering
stevendborrelli Nov 30, 2023
1074aa0
update examples
stevendborrelli Nov 30, 2023
61e3372
update version
stevendborrelli Nov 30, 2023
436ba40
Merge pull request #7 from stevendborrelli/patch-ordering
stevendborrelli Nov 30, 2023
0590290
Merge remote-tracking branch 'upstream/main' into upstream-sync
stevendborrelli Nov 30, 2023
a1a5ccd
Merge pull request #8 from stevendborrelli/upstream-sync
stevendborrelli Nov 30, 2023
03f2829
feat(upboundcare): switch to upboundcare
haarchri Feb 15, 2024
6d164cb
Merge pull request #1 from UpboundCare/feature/upboundcare-adoption
haarchri Feb 15, 2024
511de38
add conditional rendering
stevendborrelli Jan 30, 2024
49d4297
update go modules
stevendborrelli Jan 30, 2024
51e7be6
Merge branch 'main' into cel-conditionals
stevendborrelli Feb 24, 2024
4b208e6
remove render.go
stevendborrelli Feb 24, 2024
faf3317
remove old examples
stevendborrelli Feb 24, 2024
a98d431
change push org
stevendborrelli Feb 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ env:

# The package to push, without a version tag. The default matches GitHub. For
# example xpkg.upbound.io/crossplane/function-template-go.
XPKG: xpkg.upbound.io/${{ github.repository}}
XPKG: xpkg.upbound.io/borrelli-org/function-conditional-patch-and-transform
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Revert


# The package version to push. The default is 0.0.0-gitsha.
XPKG_VERSION: ${{ inputs.version }}
Expand Down Expand Up @@ -144,7 +144,7 @@ jobs:
run: "curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh"

- name: Login to Upbound
uses: docker/login-action@v3
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3
if: env.XPKG_ACCESS_ID != ''
with:
registry: xpkg.upbound.io
Expand Down
2 changes: 1 addition & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ linters-settings:
- default
- prefix(github.com/crossplane/crossplane-runtime)
- prefix(github.com/crossplane/function-sdk-go)
- prefix(github.com/crossplane-contrib/function-patch-and-transform)
- prefix(github.com/upboundcare/function-conditional-patch-and-transform)
- blank
- dot

Expand Down
20 changes: 20 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@

DOCKER := docker
GO := go

GOLANGCI_VERSION := 1.55.2

generate:
$(GO) generate ./...

test:
$(GO) test ./...

lint:
$(DOCKER) run --rm -v $(CURDIR):/app -v ~/.cache/golangci-lint/v$(GOLANGCI_VERSION):/root/.cache -w /app golangci/golangci-lint:v$(GOLANGCI_VERSION) golangci-lint run --fix

reviewable: generate test lint

# run a local process for crossplane render
run-local:
$(GO) run . --debug --insecure
264 changes: 173 additions & 91 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,140 +1,222 @@
# function-patch-and-transform
[![CI](https://github.com/crossplane-contrib/function-patch-and-transform/actions/workflows/ci.yml/badge.svg)](https://github.com/crossplane-contrib/function-patch-and-transform/actions/workflows/ci.yml) ![GitHub release (latest SemVer)](https://img.shields.io/github/release/crossplane-contrib/function-patch-and-transform)
# function-conditional-patch-and-transform

This [composition function][docs-functions] does everything Crossplane's
built-in [patch & transform][docs-pandt] (P&T) composition does. Instead of
specifying `spec.resources` in your Composition, you can use this function.
[![CI](https://github.com/upboundcare/function-conditional-patch-and-transform/actions/workflows/ci.yml/badge.svg)](https://github.com/upboundcare/function-conditional-patch-and-transform/actions/workflows/ci.yml) ![GitHub release (latest SemVer)](https://img.shields.io/github/release/crossplane-contrib/function-conditional-patch-and-transform)

Using this function, P&T looks like this:
This composition function is a fork of the upstream [function-patch-and-transform](https://github.com/crossplane-contrib/function-patch-and-transform)
that adds support for Conditional invocation of the function and the rendering
of individual resources.

## Installing this Function

The function can be installed as a Crossplane package, and runs in a [Composition Function](https://docs.crossplane.io/latest/concepts/composition-functions/). This feature requires a minium Crossplane version of 1.14.

```yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
name: example
name: function-conditional-patch-and-transform
annotations:
render.crossplane.io/runtime: Development
spec:
# Omitted for brevity.
mode: Pipeline
package: xpkg.upbound.io/upboundcare/function-conditional-patch-and-transform:v0.4.0
```

## What this function does

This function enables conditional rendering of the entire function or select resources.

The language used for Conditionals is the [Common Expression Language (CEL)](https://github.com/google/cel-spec), which is widely used in the Kubernetes ecosystem.

### Conditionally Running the Function

Composition authors express a CEL condition, and if it returns `true`, patch-and-transforms defined in the `input` will be processed.

```yaml
mode: Pipeline
pipeline:
- step: patch-and-transform
functionRef:
name: function-patch-and-transform
input:
apiVersion: pt.fn.crossplane.io/v1beta1
apiVersion: conditional-pt.fn.crossplane.io/v1beta1
kind: Resources
resources:
- name: bucket
base:
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
region: us-east-2
patches:
- type: FromCompositeFieldPath
fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.region"
transforms:
- type: map
map:
EU: "eu-north-1"
US: "us-east-2"
condition: observed.composite.resource.spec.env == "prod" && observed.composite.resource.spec.render == true
resources: [...all your resources...]
```

Using the following XR and RunFunctionRequest inputs (click to expand):
<details>

```yaml
apiVersion: nopexample.org/v1alpha1
kind: XNopResource
metadata:
name: test-resource
spec:
env: dev
render: true
```

Notice that it looks very similar to native P&T. The difference is that
everything is under `spec.pipeline[0].input.resources`, not `spec.resources`.
This is the Function's input.
```json
{
"desired": {
"composite": {
"resource": {
"apiVersion": "nopexample.org/v1alpha1",
"kind": "XNopResource",
"metadata": {
"name": "test-resource"
},
"spec": {
"env": "dev",
"render": true
}
}
},
"resources": {
"test": {
"resource": {
"apiVersion": "example.org/v1",
"kind": "CD",
"metadata": {
"name": "cool-42",
"namespace": "default"
}
}
}
}
},
"observed": {
"composite": {
"resource": {
"apiVersion": "nopexample.org/v1alpha1",
"kind": "XNopResource",
"metadata": {
"name": "test-resource"
},
"spec": {
"env": "dev",
"render": true
},
"status": {
"id": "123",
"ready": false
}
}
}
}
}
```

## Okay, but why?
</details>

There are a lot of good reasons to use a function to use a function to do P&T
composition. In fact, it's so compelling that the Crossplane maintainers are
considering deprecating native P&T. See Crossplane issue [#4746] for details.
You can use the [CEL Playground](https://playcel.undistro.io/?content=H4sIAAAAAAAAA%2B1UPW%2FDIBT8K4g5SeW0U9Z27tCh6sDyYl5aVAwIsNUq8n%2BvMY4dGxx16dYNuON93D04Uw4e6IGemSKEMMrRCYuc0QOJR%2F1pqSujnfA4O%2B8hi07XtkyQHgQjXtE6oVWAGVXa4BdURuJO2%2Fe7pgBpPqBgdLO8%2BSkUj3fenrV5GZMkxAo9hB4y%2BXtcQYUxkEfnt1O5c26bBHYGy7WgqJoYk2OT1DTIojjaQPK2xkWuq%2B24ngqYNHWp3KGJrNQ3fCA5K%2BY%2B5JuYTHh8yjNuq0%2FmBpRay%2B3DPhdpZDoDJV60PUEt%2FdKphYCrevaLQVVG9dGhbf4H%2B28HO83lwdfJGF9QMShR7O%2FXkgH%2FDpwTSPerVxRdZ6qlm27ETadKMKn74Fjn1BH4lU5EKDJ8d7vxxdH2B6myt7YTBQAA) to test various queries.

### Mix and match P&T with other functions
Here are some example queries on the XR and RunFunctionRequest:

With this function you can use P&T with other functions. For example you can
create a desired resource using the [Go Templating][fn-go-templating] function,
then patch the result using this function.
- `desired.composite.resource.spec.env == "dev"` evaluates to `true`
- `desired.composite.resource.spec.render == true,` evaluates to `true`
- `desired.composite.resource.spec.render == false"` evaluates to `false`
- `observed.composite.resource.status.ready == true"` evaluates to `false`
- `size(desired.resources) == 0` evaluates to `false`
- `"test" in desired.resources`evaluates to `true`
- `"bad-resource" in desired.resources` evaluates to `false`

To include results from previous functions, simply provide a `resources` entry
for each and specify a `name` field that matches the name of the resource from
the previous function. Also, do not specify any value for the `base` field of
each resource.
### Conditionally Rendering Managed Resources

It's not just patches either. You can use P&T to derive composite resource
connection details from a resource produced by another function, or use it to
determine whether a resource produced by another function is ready.
In a similar manner, individual Managed Resources can also
be rendered conditionally, see the example at [examples/conditional-resources](examples/conditional-resources/).

### Decouple P&T development from Crossplane core
Each resource can have a `condition`.

When P&T development happens in a function, it's not coupled to the Crossplane
release cycle. The maintainers of this function can cut releases more frequently
to add new features to P&T.
```yaml
resources:
- name: blue-resource
condition: observed.composite.resource.spec.deployment.blue == true
base:
apiVersion: nop.crossplane.io/v1alpha1
kind: NopResource
spec:
forProvider:
```

It also becomes easier to fork. You could fork this function, add a new kind of
transform and try it out for a few weeks before sending a PR upstream. Or, if
your new feature is controversial, it's now a lot less work to maintain your own
fork long term.
If this condition is set in the Claim/XR, the resource will be rendered:

### Test P&T locally using the Crossplane CLI
```yaml
apiVersion: nop.example.org/v1alpha1
kind: XNopConditional
metadata:
name: test-resource
spec:
env: dev
render: true
deployment:
blue: true
green: false

```

### Test this function locally using the Crossplane CLI

You can use the Crossplane CLI to run any function locally and see what composed
resources it would create. This only works with functions - not native P&T.

For example, using the files in the [example](example) directory:
For example, using the files in the [examples](examples) directory:

```shell
$ crossplane beta render xr.yaml composition.yaml functions.yaml
cd examples/conditional-rendering
crossplane beta render xr.yaml composition.yaml functions.yaml
```

Produces the following output, showing what resources Crossplane would compose:

```yaml
---
apiVersion: example.crossplane.io/v1
kind: XR
apiVersion: nop.example.org/v1alpha1
kind: XNopResource
metadata:
name: example-xr
name: test-resource
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
apiVersion: nop.crossplane.io/v1alpha1
kind: NopResource
metadata:
annotations:
crossplane.io/composition-resource-name: bucket
generateName: example-xr-
crossplane.io/composition-resource-name: test-resource
generateName: test-resource-
labels:
crossplane.io/composite: example-xr
crossplane.io/composite: test-resource
ownerReferences:
# Omitted for brevity
- apiVersion: nop.example.org/v1alpha1
blockOwnerDeletion: true
controller: true
kind: XNopResource
name: test-resource
uid: ""
spec:
forProvider:
region: us-east-2
conditionAfter:
- conditionStatus: "True"
conditionType: Ready
time: 5s
connectionDetails:
- name: username
value: fakeuser
- name: password
value: verysecurepassword
- name: endpoint
value: 127.0.0.1
fields:
arrayField:
- stringField: array
integerField: 42
objectField:
stringField: object
stringField: string
```

See the [composition functions documentation][docs-functions] to learn how to
use `crossplane beta render`.

## Differences from the native implementation

This function has a few small, intentional breaking changes compared to the
native implementation.

These fields are now required. This makes P&T configuration less ambiguous:

* `resources[i].name`
* `resources[i].connectionDetails[i].name`
* `resources[i].connectionDetails[i].type`
* `resources[i].patches[i].transforms[i].string.type`
* `resources[i].patches[i].transforms[i].math.type`

Also, the `resources[i].patches[i].policy.mergeOptions` field is no longer
supported. The same capability can be achieved by setting
`resources[i].patches[i].policy.toFieldPath` to:
- `MergeObject` - equivalent to
`resources[i].patches[i].policy.mergeOptions.keepMapValues: true`
- `AppendArray` - equivalent to
`resources[i].patches[i].policy.mergeOptions.appendSlice: false`

## Developing this function

This function uses [Go][go], [Docker][docker], and the [Crossplane CLI][cli] to
Expand All @@ -158,9 +240,9 @@ $ crossplane xpkg build -f package --embed-runtime-image=runtime
[docs-composition]: https://docs.crossplane.io/v1.14/getting-started/provider-aws-part-2/#create-a-deployment-template
[docs-functions]: https://docs.crossplane.io/v1.14/concepts/composition-functions/
[docs-pandt]: https://docs.crossplane.io/v1.14/concepts/patch-and-transform/
[fn-go-templating]: https://github.com/crossplane-contrib/function-go-templating
[fn-go-templating]: https://github.com/upboundcare/function-go-templating
[#4617]: https://github.com/crossplane/crossplane/issues/4617
[#4746]: https://github.com/crossplane/crossplane/issues/4746
[go]: https://go.dev
[docker]: https://www.docker.com
[cli]: https://docs.crossplane.io/latest/cli
[cli]: https://docs.crossplane.io/latest/cli
Loading
Loading