An application is a group of k8s resources related by some common purpose, e.g. a load balancer in front of a webserver backed by a database. Resource labelling, naming and metadata schemes have historically served to group resources together for collective operations like list and remove.
This proposal describes a new k8s resource called application to more formally describe this idea and provide support for application-level operations and dashboards.
kustomize configures k8s resources, and the proposed application resource is just another resource.
The verb apply in the context of k8s refers to a kubectl command and an in-progress API endpoint for mutating a cluster.
One applies a statement of what one wants to a cluster in the form of a complete resource list.
The cluster merges this with the previously applied state and the actual state to arrive at a new desired state, which the cluster's reconciliation loop attempts to create. This is the foundation of level-based state management in k8s.
A base is a kustomization referred to by some other kustomization.
Any kustomization, including an overlay, can be a base to another kustomization.
A base has no knowledge of the overlays that refer to it.
For simple gitops management, a base configuration could be the sole content of a git repository dedicated to that purpose. Same with overlays. Changes in a repo could generate a build, test and deploy cycle.
A bespoke configuration is a kustomization and some resources created and maintained internally by some organization for their own purposes.
The workflow associated with a bespoke config is simpler than the workflow associated with an off-the-shelf config, because there's no notion of periodically capturing someone else's upgrades to the off-the-shelf config.
One can extend the k8s API by making a Custom Resource Definition (CRD spec).
This defines a custom resource (CD), an entirely new resource that can be used alongside native resources like ConfigMaps, Deployments, etc.
Kustomize can customize a CD, but to do so kustomize must also be given the corresponding CRD so that it can interpret the structure correctly.
Kustomize aspires to support Declarative Application Management, a set of best practices around managing k8s clusters.
In brief, kustomize should
- Work with any configuration, be it bespoke, off-the-shelf, stateless, stateful, etc.
- Support common customizations, and creation of variants (e.g. development vs. staging vs. production).
- Expose and teach native k8s APIs, rather than hide them.
- Add no friction to version control integration to support reviews and audit trails.
- Compose with other tools in a unix sense.
- Eschew crossing the line into templating, domain specific languages, etc., frustrating the other goals.
A generator makes resources that can be used as is, or fed into a transformer.
Devops or CICD workflows that use a git repository as a single source of truth and take action (e.g., build, test or deploy) when that truth changes.
The term kustomization refers to a
kustomization.yaml file, or more generally to a
directory (the root) containing the
kustomization.yaml file and all the relative file
paths that it immediately references (all the local
data that doesn't require a URL specification).
I.e. if someone gives you a kustomization for use with kustomize, it could be in the form of
- one file called
kustomization.yaml, - a tarball (containing that YAML file plus what it references),
- a git archive (ditto),
- a URL to a git repo (ditto), etc.
A kustomization file contains fields falling into four categories:
-
resources - what existing resources are to be customized. Example fields: resources, crds.
-
generators - what new resources should be created. Example fields: configMapGenerator (legacy), secretGenerator (legacy), generators (v2.1).
-
transformers - what to do to the aforementioned resources. Example fields: namePrefix, nameSuffix, images, commonLabels, patchesJson6902, etc. and the more general transformers (v2.1) field.
-
meta - fields which may influence all or some of the above. Example fields: vars, namespace, apiVersion, kind, etc.
The directory that immediately contains a
kustomization.yaml file.
When a kustomization file is processed, it may or may not be able to access files outside its root.
Data files like resource YAML files, or text files containing name=value pairs intended for a ConfigMap or Secret, or files representing a patch to be used in a patch transformation, must live within or below the root, and as such are specified via relative paths exclusively.
A special flag (in v2.1), --load_restrictions none,
is provided to relax this security feature, to, say,
allow a patch file to be shared by more than one
kustomization.
Other kustomizations (other directories containing a
kustomization.yaml file) may be referred to by URL, by
absolute path, or by relative path.
If kustomization A depends on kustomization B, then
- B may not contain A.
- B may not depend on A, even transitively.
A may contain B, but in this case it might be simplest to have A directly depend on B's resources and eliminate B's kustomization.yaml file (i.e. absorb B into A).
Conventionally, B is in a directory that's sibling to A, or B is off in a completely independent git repository, referencable from any kustomization.
A common layout is
├── base │ ├── deployment.yaml │ ├── kustomization.yaml │ └── service.yaml └── overlays ├── dev │ ├── kustomization.yaml │ └── patch.yaml ├── prod │ ├── kustomization.yaml │ └── patch.yaml └── staging ├── kustomization.yaml └── patch.yaml
The three roots dev, prod and staging
(presumably) all refer to the base root. One would
have to inspect the kustomization.yaml files to be
sure.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
It's often abbreviated as k8s.
An object, expressed in a YAML or JSON file, with the fields required by kubernetes. Basically just a kind field to identify the type, a metadata/name field to identify the particular instance, and an apiVersion field to identify the version (if there's more than one version).
kustomize is a command line tool supporting template-free, structured customization of declarative configuration targetted to k8s-style objects.
Targetted to k8s means that kustomize has some understanding of API resources, k8s concepts like names, labels, namespaces, etc. and the semantics of resource patching.
kustomize is an implementation of DAM.
An off-the-shelf configuration is a kustomization and resources intentionally published somewhere for others to use.
E.g. one might create a github repository like this:
github.com/username/someapp/ kustomization.yaml deployment.yaml configmap.yaml README.md
Someone could then fork this repo (on github) and clone their fork to their local disk for customization.
This clone could act as a base for the user's own overlays to do further customization.
An overlay is a kustomization that depends on another kustomization.
The kustomizations an overlay refers to (via file path, URI or other method) are called bases.
An overlay is unusable without its bases.
An overlay may act as a base to another overlay.
Overlays make the most sense when there is more than one, because they create different variants of a common base - e.g. development, QA, staging and production environment variants.
These variants use the same overall resources, and vary in relatively simple ways, e.g. the number of replicas in a deployment, the CPU to a particular pod, the data source used in a ConfigMap, etc.
One configures a cluster like this:
kustomize build someapp/overlays/staging |\ kubectl apply -f - kustomize build someapp/overlays/production |\ kubectl apply -f -
Usage of the base is implicit - the overlay's kustomization points to the base.
See also root.
The word package has no meaning in kustomize, as kustomize is not to be confused with a package management tool in the tradition of, say, apt or rpm.
General instructions to modify a resource.
There are two alternative techniques with similar power but different notation - the strategic merge patch and the JSON patch.
A patchStrategicMerge is strategic-merge-style patch (SMP).
An SMP looks like an incomplete YAML specification of
a k8s resource. The SMP includes TypeMeta
fields to establish the group/version/kind/name of the
resource to patch, then just enough remaining fields
to step into a nested structure to specify a new field
value, e.g. an image tag.
By default, an SMP replaces values. This is usually desired when the target value is a simple string, but may not be desired when the target value is a list.
To change this default behavior, add a directive. Recognized directives in YAML patches are replace (the default) and delete (see these notes).
Note that for custom resources, SMPs are treated as json merge patches.
Fun fact - any resource file can be used as an SMP, overwriting matching fields in another resource with the same group/version/kind/name, but leaving all other fields as they were.
TODO(monopole): add ptr to example.
A patchJson6902 refers to a kubernetes resource and a JSONPatch specifying how to change the resource.
A patchJson6902 can do almost everything a patchStrategicMerge can do, but with a briefer syntax. See this example.
A chunk of code used by kustomize, but not necessarily compiled into kustomize, to generate and/or transform a kubernetes resource as part of a kustomization.
Details here.
A resource in the context of a REST-ful API is the target object of an HTTP operation like GET, PUT or POST. k8s offers a REST-ful API surface to interact with clients.
A resource, in the context of a kustomization, is a root relative path to a YAML or JSON file describing a k8s API object, like a Deployment or a ConfigMap, or it's a path to a kustomization, or a URL that resolves to a kustomization.
More generally, a resource can be any correct YAML file that defines an object with a kind and a metadata/name field.
See kustomization root.
A sub-whatever is not a thing. There are only bases and overlays.
The target is the argument to kustomize build, e.g.:
kustomize build $target
$target must be a path or a url to a kustomization.
The target contains, or refers to, all the information needed to create customized resources to send to the apply operation.
A target can be a base or an overlay.
A transformer can modify a resource, or merely
visit it and collect information about it in the
course of a kustomize build.
A variant is the outcome, in a cluster, of applying an overlay to a base.
E.g., a staging and production overlay both modify some common base to create distinct variants.
The staging variant is the set of resources exposed to quality assurance testing, or to some external users who'd like to see what the next version of production will look like.
The production variant is the set of resources exposed to production traffic, and thus may employ deployments with a large number of replicas and higher cpu and memory requests.