From 25c0a0087e267f70de602ad2f015581833830f95 Mon Sep 17 00:00:00 2001 From: Philippe Scorsolini Date: Tue, 15 Oct 2024 11:47:03 +0100 Subject: [PATCH] update composition environment docs to function-environment-configs Signed-off-by: Philippe Scorsolini --- .../master/concepts/environment-configs.md | 560 ++++++++++-------- 1 file changed, 320 insertions(+), 240 deletions(-) diff --git a/content/master/concepts/environment-configs.md b/content/master/concepts/environment-configs.md index 7369218e..ff7a7991 100644 --- a/content/master/concepts/environment-configs.md +++ b/content/master/concepts/environment-configs.md @@ -11,12 +11,12 @@ TODO: Add Policies --> -A Crossplane EnvironmentConfig is a cluster scoped +A Crossplane EnvironmentConfig is a cluster-scoped, strongly-typed, [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/)-like resource used by Compositions. Compositions can use the environment to store information from individual resources or to apply patches. -Crossplane supports multiple EnvironmentConfigs, each acting as a unique +Crossplane supports multiple `EnvironmentConfigs`, each acting as a unique data store. When Crossplane creates a composite resource, Crossplane merges all the @@ -32,39 +32,6 @@ A composite resource can't read data in another composite resource's environment. {{< /hint >}} -## Enable EnvironmentConfigs -EnvironmentConfigs are an alpha feature. Alpha features aren't enabled by -default. - -Enable EnvironmentConfig support by -[changing the Crossplane pod setting]({{}}) -and enabling -{{}}--enable-environment-configs{{}} -argument. - -```yaml {label="deployment",copy-lines="12"} -$ kubectl edit deployment crossplane --namespace crossplane-system -apiVersion: apps/v1 -kind: Deployment -spec: -# Removed for brevity - template: - spec: - containers: - - args: - - core - - start - - --enable-environment-configs -``` - -{{}} - -The [Crossplane install guide]({{}}) -describes enabling feature flags like -{{}}--enable-environment-configs{{}} -with Helm. -{{< /hint >}} - ## Create an EnvironmentConfig @@ -95,30 +62,47 @@ data: - item2 ``` +## Access EnvironmentConfigs + +`EnvironmentConfigs` can be accessed by [Composition Functions] supporting +[extra-resources], e.g. [function-environment-configs] or +[function-go-templating]. + +## Migration from Alpha Composition Environment + +Crossplane (`<=v1.17`) natively supported selecting `EnvironmentConfigs`, +merging them into an `in-memory environment` and patching between that, +composed and composite resources. From `v1.18`, this native capability has been +removed, in favor of [Composition Functions]. + +Users that enabled Alpha Composition Environments +(`--enable-environment-configs`) and leveraged the native functionality +(`spec.environment.patches`, `spec.environment.environmentConfigs` and +`*Environment` patches), will have to migrate to Composition Functions to +continue doing so. + +Automated migration to `Pipeline` mode is available through `crossplane beta +convert pipeline-composition`, which will move a composition using `Resource` +mode, to [function-patch-and-transform] and, if needed, +[function-environment-configs]. + +See [function-environment-configs]' docs for more details about manual +migration. + -## Select an EnvironmentConfig +## Select an EnvironmentConfig using function-environment-configs -Select the EnvironmentConfigs to use -inside a Composition's -{{}}environment{{}} field. +Select the EnvironmentConfigs to use through [function-environment-configs]'s `Input`. -The {{}}environmentConfigs{{}} field is a -list of environments this Composition can use. +The `environmentConfigs` field is a list of `EnvironmentConfigs` we want +retrieved, merged and passed to the next step in the pipeline through the +[Context] at a well-known key, `apiextensions.crossplane.io/environment`. -Select an environment by -{{}}Reference{{}} or -by -{{}}Selector{{}}. +Select an environment by `Reference` or by `Selector`: -A -{{}}Reference{{}} -selects an environment by -{{}}name{{}}. -The -{{}}Selector{{}} selects an environment -based on the -{{}}Labels{{}} applied to the environment. +* A `Reference` selects an `EnvironmentConfig` by name. +* The `Selector` selects an `EnvironmentConfig` by labels. ```yaml {label="comp",copy-lines="none"} apiVersion: apiextensions.crossplane.io/v1 @@ -126,42 +110,41 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Reference - ref: - name: example-environment - - type: Selector - selector: - matchLabels: - # Removed for brevity + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Reference + ref: + name: example-environment + - type: Selector + selector: + matchLabels: + # Removed for brevity + # the environment will be passed to the next function in the pipeline + # as part of the context + +# Next step consuming the merged environment removed for brevity... ``` -If a Composition uses multiple -{{}}environmentConfigs{{}} -Crossplane merges them together in the order they're listed. - -{{}} -If multiple -{{}}environmentConfigs{{}} -use the same key, the Composition uses the value of the last environment listed. -{{}} +If a Composition uses multiple `EnvironmentConfigs`, +[function-environment-configs] merges them together in the order they're +listed. ### Select by name -Select an environment by name with -{{}}type: Reference{{}}. +Select an `EnvironmentConfig` by name with `type: Reference`. -Define the -{{}}ref{{}} object and the -{{}}name{{}} matching the exact name of -the environment. +Define `ref.name` to match the exact name of the environment. -For example, select the -{{}}environmentConfig{{}} -named -{{}}example-environment{{}} +For example, select the `EnvironmentConfig` named `example-environment`: ```yaml {label="byName",copy-lines="all"} apiVersion: apiextensions.crossplane.io/v1 @@ -169,38 +152,33 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Reference - ref: - name: example-environment + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Reference + ref: + name: example-environment ``` ### Select by label -Select an environment by labels with a -{{}}type: Selector{{}}. - -Define the {{}}selector{{}} object. +Select an `EnvironmentConfig` by labels with a `type: Selector`. -The -{{}}matchLabels{{}} object contains a -list of labels to match on. - -Selecting a label requires matching both the label -{{}}key{{}} -and the value of key. +Define `selector.matchLabels` to a list of selectors either of type `Value`, or `FromCompositeFieldPath`. When matching the label's value, provide an exact value with a -{{}}type: Value{{}} and provide the value -to match in the -{{}}value{{}} field. +`type: Value and provide the value to match in the `value` field. -Crossplane can also match a label's value based on an input in the composite -resource. Use -{{}}type: FromCompositeFieldPath{{}} -and provide the field to match in the -{{}}valueFromFieldPath{{}} field. +[function-environment-configs] can also match a label's value based on an input +in the composite resource. Use `type: FromCompositeFieldPath` and provide the +field to match in the `valueFromFieldPath` field. ```yaml {label="byLabel",copy-lines="all"} apiVersion: apiextensions.crossplane.io/v1 @@ -208,42 +186,39 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Selector - selector: - matchLabels: - - key: my-label-key - type: Value - value: my-label-value - - key: my-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Selector + selector: + matchLabels: + - key: my-label-key + type: Value + value: my-label-value + - key: my-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy # Removed for brevity ``` #### Manage selector results Selecting environments by labels may return more than one environment. -The Composition sorts all the results by the name of the environments and -only uses the first environment in the sorted list. - -Set the {{}}mode{{}} as -{{}}mode: Multiple{{}} to return -all matched environments. Use -{{}}mode: Single{{}} to -return a single environment. - -{{}} -Sorting and the selection -{{}}mode{{}} -only applies to a single -{{}}type: Selector{{}}. - -This doesn't change how Compositions merge multiple -{{}}environmentConfigs{{}}. -{{< /hint >}} +[function-environment-configs], by default, sorts all the results by name and +only uses the first environment in the sorted list. + +Set the `selector.mode` to `Multiple` to return all matched EnvironmentConfigs. +Use `mode: Single` to return a single environment, and error out if more than +one match is found. +Sorting and the selection mode only applies to a single `Selector`. ```yaml {label="selectResults"} apiVersion: apiextensions.crossplane.io/v1 @@ -251,42 +226,45 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Selector - selector: - mode: Multiple - matchLabels: - - key: my-label-key - type: Value - value: my-label-value - - key: my-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy - - type: Selector - selector: - mode: Single - matchLabels: - - key: my-other-label-key - type: Value - value: my-other-label-value - - key: my-other-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Selector + selector: + mode: Multiple + matchLabels: + - key: my-label-key + type: Value + value: my-label-value + - key: my-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy + - type: Selector + selector: + mode: Single + matchLabels: + - key: my-other-label-key + type: Value + value: my-other-label-value + - key: my-other-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy ``` -When using -{{}}mode: Multiple{{}} limit the -number of returned environments with -{{}}maxMatch{{}} and define the -maximum number of environments returned. +When using `mode: Multiple` limit the number of returned `EnvironmentConfigs` +with `maxMatch` and define the maximum number to select. -Use `minMatch` and define the minimum -number of environments returned. +Use `minMatch` and define the minimum number of environments returned. -The Composition sorts the returned environments alphabetically by name. Sort the -environments on a different field with -{{}}sortByFieldPath{{}} and define +The Function sorts the returned environments alphabetically by name by defaul. +Sort the environments on a different field with `sortByFieldPath` and define the field to sort by. @@ -296,36 +274,38 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Selector - selector: - mode: Multiple - maxMatch: 4 - sortByFieldPath: metadata.annotations[sort.by/weight] - matchLabels: - - key: my-label-key - type: Value - value: my-label-value - - key: my-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Selector + selector: + mode: Multiple + maxMatch: 4 + sortByFieldPath: metadata.annotations[sort.by/weight] + matchLabels: + - key: my-label-key + type: Value + value: my-label-value + - key: my-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy ``` -The environments selected by -{{}}matchLabels{{}} are then merged -into any other environments listed in the -{{}}environmentConfigs{{}}. +The EnvironmentConfigs selected by `matchLabels` are then merged with all the +other ones specified. #### Optional selector labels -By default, Crossplane issues an error if a -{{}}valueFromFieldPath{{}} +By default, Crossplane issues an error if the specified `valueFromFieldPath` field doesn't exist in the composite resource. -Add -{{}}fromFieldPathPolicy{{}} -as {{}}Optional{{}} -to ignore a field if it doesn't exist. +Set `fromFieldPathPolicy` to `Optional` to ignore a field if it doesn't exist. ```yaml {label="byLabelOptional",copy-lines="all"} apiVersion: apiextensions.crossplane.io/v1 @@ -333,34 +313,40 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Selector - selector: - matchLabels: - - key: my-first-label-key - type: Value - value: my-first-label-value - - key: my-second-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy - fromFieldPathPolicy: Optional + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Selector + selector: + mode: Multiple + maxMatch: 4 + sortByFieldPath: metadata.annotations[sort.by/weight] + matchLabels: + - key: my-label-key + type: Value + value: my-label-value + - key: my-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy + fromFieldPathPolicy: Optional # Removed for brevity ``` -Set a default value for an optional label by setting the default -{{}}value{{}} for the -{{}}key{{}} first, then -define the -{{}}Optional{{}} label. +Set a default value for an optional label by setting the default `value` for +the `key` first using a `Value` selector, then define the `Optional` +`FromCompositeFieldPath` one. -For example, this Composition defines -{{}}value: my-default-value{{}} -for the key {{}}my-second-label-key{{}}. -If the label -{{}}my-second-label-key{{}} -exists, Crossplane uses the value from the label instead. +For example, this Composition defines `value: my-default-value` for the key +`my-second-label-key`. If the Composite resource defines +`spec.parameters.deploy`, [function-environment-configs] will use that instead. ```yaml {label="byLabelOptionalDefault",copy-lines="all"} apiVersion: apiextensions.crossplane.io/v1 @@ -368,47 +354,141 @@ kind: Composition metadata: name: example-composition spec: - environment: - environmentConfigs: - - type: Selector - selector: - matchLabels: - - key: my-first-label-key - type: Value - value: my-label-value - - key: my-second-label-key - type: Value - value: my-default-value - - key: my-second-label-key - type: FromCompositeFieldPath - valueFromFieldPath: spec.parameters.deploy - fromFieldPathPolicy: Optional + mode: Pipeline + pipeline: + - step: environmentConfigs + functionRef: + name: function-environment-configs + input: + apiVersion: environmentconfigs.fn.crossplane.io/v1beta1 + kind: Input + spec: + environmentConfigs: + - type: Selector + selector: + matchLabels: + - key: my-first-label-key + type: Value + value: my-label-value + - key: my-second-label-key + type: Value + value: my-default-value + - key: my-second-label-key + type: FromCompositeFieldPath + valueFromFieldPath: spec.parameters.deploy + fromFieldPathPolicy: Optional # Removed for brevity ``` {{}} -Crossplane applies values in order. The value of the last key defined always takes precedence. +[function-environment-configs] applies values in order. The value of the last +key defined always takes precedence. Defining the default value _after_ the label always overwrites the label value. {{< /hint >}} -## Use EnvironmentConfigs in a Composition +## Patching with EnvironmentConfigs using [function-patch-and-transform] -When Crossplane creates or updates a composite resource, it merges all the -specified EnvironmentConfigs into an in-memory environment. +`EnvironmentConfigs` selected as explained above, are then merged in an +`in-memory environment` by [function-environment-configs] and passed to the +next function in the pipeline at a well-known key, +`apiextensions.crossplane.io/environment`. -Crossplane sends the merged, in-memory environment to the composition function -pipeline using the -[pipeline context]({{}}). -It writes the environment to the `apiextensions.crossplane.io/environment` -context key. - -Some composition functions can read the environment from the pipeline context -and use it to compose resources. +[function-patch-and-transform] can be used to read or write data between the in-memory environment and +composite resource or individual composed resources. {{}} The Patch and Transform function can use the environment to patch composed resources. Read about EnvironmentConfig patch types in the [Patch and Transform function documentation]({{}}). {{< /hint >}} + +### Patch between Composite resource and environment + +To patch between Composite resource and environment define patches at +`spec.environment.patches` in [function-patch-and-transform]'s `Resources` +input. + +Use the `ToCompositeFieldPath` patch type to copy data from the in-memory +environment to the Composite resource. Use the `FromCompositeFieldPath` to +copy data from the Composite resource to the in-memory environment. + +```yaml {label="xrpatch",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: example-composition +spec: + mode: Pipeline + pipeline: + # Removed for Brevity + - step: patch-and-transform + functionRef: + name: function-patch-and-transform + input: + apiVersion: pt.fn.crossplane.io/v1beta1 + kind: Resources + environment: + patches: + - type: ToCompositeFieldPath + fromFieldPath: tags + toFieldPath: metadata.labels[envTag] + - type: FromCompositeFieldPath + fromFieldPath: metadata.name + toFieldPath: newEnvironmentKey +# Removed for Brevity +``` + +Individual resources can use any data written to the in-memory environment. + +`CombineFromComposite` and `CombineToComposite` can be used to combine multiple +values and write the result either to the in-memory environment or the +Composite resource, respectively. + +### Patch an individual resource + +To patch between individual resources and the in-memory environment, inside the +patches of the resource, use `ToEnvironmentFieldPath` to copy data from the +resource to the in-memory environment. Use `FromEnvironmentFieldPath` to copy +data to the resource from the in-memory environment. + +```yaml {label="envpatch",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: example-composition +spec: + mode: Pipeline + pipeline: + # Removed for Brevity + - step: patch-and-transform + functionRef: + name: function-patch-and-transform + input: + apiVersion: pt.fn.crossplane.io/v1beta1 + kind: Resources + # Removed for Brevity + resources: + - name: vpc + base: + apiVersion: ec2.aws.upbound.io/v1beta1 + kind: VPC + spec: + forProvider: + cidrBlock: 172.16.0.0/16 + patches: + - type: ToEnvironmentFieldPath + fromFieldPath: status.atProvider.id + toFieldPath: vpcId + - type: FromEnvironmentFieldPath + fromFieldPath: tags + toFieldPath: spec.forProvider.tags +``` + +The [Patch and Transform]({{}}) documentation has +more information on patching individual resources. + +[extra-resources]: {{}} +[function-environment-configs]: https://github.com/crossplane-contrib/function-environment-configs +[function-patch-and-transform]: {{}} \ No newline at end of file