diff --git a/README.md b/README.md index 403503f848..9477da3c50 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ The `distributions` directory contains manifests for specific, opinionated distr The `docs`, `hack`, and `tests` directories will also be gradually phased out. -Starting Kubeflow 1.3, all components should be deployable using `kustomize` only. Any automation tooling for deployment on top of the manifests should be maintained externally by distribution owners. +Starting from Kubeflow 1.3, all components should be deployable using `kustomize` only. Any automation tooling for deployment on top of the manifests should be maintained externally by distribution owners. ## Kubeflow components versions @@ -43,22 +43,30 @@ This repo periodically syncs all official Kubeflow components from their respect | - | - | - | | Training Operator | apps/training-operator/upstream | [v1.3.0](https://github.com/kubeflow/tf-operator/tree/v1.3.0/manifests) | | MPI Operator | apps/mpi-job/upstream | [v0.3.0](https://github.com/kubeflow/mpi-operator/tree/v0.3.0/manifests) | -| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/notebook-controller/config) | -| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/tensorboard-controller/config) | -| Central Dashboard | apps/centraldashboard/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/centraldashboard/manifests) | -| Profiles + KFAM | apps/profiles/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/profile-controller/config) | -| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/admission-webhook/manifests) | -| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/crud-web-apps/jupyter/manifests) | -| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/crud-web-apps/tensorboards/manifests) | -| Volumes Web App | apps/volumes-web-app/upstream | [v1.4-rc.0](https://github.com/kubeflow/kubeflow/tree/v1.4-rc.0/components/crud-web-apps/volumes/manifests) | +| Notebook Controller | apps/jupyter/notebook-controller/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/notebook-controller/config) | +| Tensorboard Controller | apps/tensorboard/tensorboard-controller/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/tensorboard-controller/config) | +| Central Dashboard | apps/centraldashboard/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/centraldashboard/manifests) | +| Profiles + KFAM | apps/profiles/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/profile-controller/config) | +| PodDefaults Webhook | apps/admission-webhook/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/admission-webhook/manifests) | +| Jupyter Web App | apps/jupyter/jupyter-web-app/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/crud-web-apps/jupyter/manifests) | +| Tensorboards Web App | apps/tensorboard/tensorboards-web-app/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/crud-web-apps/tensorboards/manifests) | +| Volumes Web App | apps/volumes-web-app/upstream | [v1.4.0](https://github.com/kubeflow/kubeflow/tree/v1.4.0/components/crud-web-apps/volumes/manifests) | | Katib | apps/katib/upstream | [v0.12.0](https://github.com/kubeflow/katib/tree/v0.12.0/manifests/v1beta1) | | KFServing | apps/kfserving/upstream | [v0.6.1](https://github.com/kubeflow/kfserving/releases/tag/v0.6.1) | | Kubeflow Pipelines | apps/pipeline/upstream | [1.7.0](https://github.com/kubeflow/pipelines/tree/1.7.0/manifests/kustomize) | | Kubeflow Tekton Pipelines | apps/kfp-tekton/upstream | [v1.0.0](https://github.com/kubeflow/kfp-tekton/tree/v1.0.0/manifests/kustomize) | +The following is also a matrix with versions from common components that are +used from the different projects of Kubeflow: + +| Component | Local Manifests Path | Upstream Revision | +| - | - | - | +| Istio | common/istio-1-9 | [1.9.6](https://github.com/istio/istio/releases/tag/1.9.6) | +| Knative | common/knative | [0.22.1](https://github.com/knative/serving/releases/tag/v0.22.1) | + ## Installation -Starting Kubeflow 1.3, the Manifests WG provides two options for installing Kubeflow official components and common services with kustomize. The aim is to help end users install easily and to help distribution owners build their opinionated distributions from a tested starting point: +Starting from Kubeflow 1.3, the Manifests WG provides two options for installing Kubeflow official components and common services with kustomize. The aim is to help end users install easily and to help distribution owners build their opinionated distributions from a tested starting point: 1. Single-command installation of all components under `apps` and `common` 2. Multi-command, individual components installation for `apps` and `common` @@ -72,9 +80,9 @@ The `example` directory contains an example kustomization for the single command ### Prerequisites -- `Kubernetes` (tested with version `1.17`) with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) +- `Kubernetes` (tested with version `1.19`) with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) - `kustomize` (version `3.2.0`) ([download link](https://github.com/kubernetes-sigs/kustomize/releases/tag/v3.2.0)) - - :warning: Kubeflow 1.3.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can. + - :warning: Kubeflow 1.4.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can. - `kubectl` --- @@ -398,8 +406,8 @@ For security reasons, we don't want to use the default password for the default ## Frequently Asked Questions -- **Q:** What versions of Istio, Knative, Cert-Manager, Argo, ... are compatible with Kubeflow 1.3? \ +- **Q:** What versions of Istio, Knative, Cert-Manager, Argo, ... are compatible with Kubeflow 1.4? \ **A:** Please refer to each individual component's documentation for a dependency compatibility range. For Istio, Knative, Dex, Cert-Manager and OIDC-AuthService, the versions in `common` are the ones we have validated. - **Q:** Can I use the latest Kustomize version (`v4.x`)? \ - **A:** Kubeflow 1.3.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can. + **A:** Kubeflow 1.4.0 is not compatible with the latest versions of of kustomize 4.x. This is due to changes in the order resources are sorted and printed. Please see [kubernetes-sigs/kustomize#3794](https://github.com/kubernetes-sigs/kustomize/issues/3794) and [kubeflow/manifests#1797](https://github.com/kubeflow/manifests/issues/1797). We know this is not ideal and are working with the upstream kustomize team to add support for the latest versions of kustomize as soon as we can. diff --git a/apps/admission-webhook/upstream/base/kustomization.yaml b/apps/admission-webhook/upstream/base/kustomization.yaml index d3fbf3df0b..385d7fc8bc 100644 --- a/apps/admission-webhook/upstream/base/kustomization.yaml +++ b/apps/admission-webhook/upstream/base/kustomization.yaml @@ -16,7 +16,7 @@ commonLabels: images: - name: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook newName: public.ecr.aws/j1r0q0g6/notebooks/admission-webhook - newTag: v1.4-rc.1 + newTag: v1.4 namespace: kubeflow generatorOptions: disableNameSuffixHash: true diff --git a/apps/centraldashboard/upstream/base/kustomization.yaml b/apps/centraldashboard/upstream/base/kustomization.yaml index c01e72e4ba..2c1eb9a734 100644 --- a/apps/centraldashboard/upstream/base/kustomization.yaml +++ b/apps/centraldashboard/upstream/base/kustomization.yaml @@ -18,7 +18,7 @@ commonLabels: images: - name: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard newName: public.ecr.aws/j1r0q0g6/notebooks/central-dashboard - newTag: v1.4-rc.1 + newTag: v1.4 configMapGenerator: - envs: - params.env diff --git a/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml b/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml index 8a3da8db5e..f9b9aadf92 100644 --- a/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml +++ b/apps/jupyter/jupyter-web-app/upstream/base/configs/spawner_ui_config.yaml @@ -17,23 +17,23 @@ spawnerFormDefaults: image: # The container Image for the user's Jupyter Notebook - value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.4-rc.1 + value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.4 # The list of available standard container Images options: - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.4-rc.1 - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.4-rc.1 - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.4-rc.1 - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.4-rc.1 - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.4-rc.1 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-scipy:v1.4 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-full:v1.4 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-pytorch-cuda-full:v1.4 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-full:v1.4 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/jupyter-tensorflow-cuda-full:v1.4 imageGroupOne: # The container Image for the user's Group One Server # The annotation `notebooks.kubeflow.org/http-rewrite-uri: /` # is applied to notebook in this group, configuring # the Istio rewrite for containers that host their web UI at `/` - value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.4-rc.1 + value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.4 # The list of available standard container Images options: - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.4-rc.1 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/codeserver-python:v1.4 imageGroupTwo: # The container Image for the user's Group Two Server # The annotation `notebooks.kubeflow.org/http-rewrite-uri: /` @@ -42,10 +42,10 @@ spawnerFormDefaults: # The annotation `notebooks.kubeflow.org/http-headers-request-set` # is applied to notebook in this group, configuring Istio # to add the `X-RStudio-Root-Path` header to requests - value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.4-rc.1 + value: public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.4 # The list of available standard container Images options: - - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.4-rc.1 + - public.ecr.aws/j1r0q0g6/notebooks/notebook-servers/rstudio-tidyverse:v1.4 # If true, hide registry and/or tag name in the image selection dropdown hideRegistry: true hideTag: false diff --git a/apps/jupyter/jupyter-web-app/upstream/base/kustomization.yaml b/apps/jupyter/jupyter-web-app/upstream/base/kustomization.yaml index a4a9bfd091..c967886445 100644 --- a/apps/jupyter/jupyter-web-app/upstream/base/kustomization.yaml +++ b/apps/jupyter/jupyter-web-app/upstream/base/kustomization.yaml @@ -23,7 +23,7 @@ commonLabels: images: - name: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app newName: public.ecr.aws/j1r0q0g6/notebooks/jupyter-web-app - newTag: v1.4-rc.1 + newTag: v1.4 # We need the name to be unique without the suffix because the original name is what # gets used with patches configMapGenerator: diff --git a/apps/jupyter/notebook-controller/upstream/base/kustomization.yaml b/apps/jupyter/notebook-controller/upstream/base/kustomization.yaml index 114edbd246..a44beace4d 100644 --- a/apps/jupyter/notebook-controller/upstream/base/kustomization.yaml +++ b/apps/jupyter/notebook-controller/upstream/base/kustomization.yaml @@ -5,4 +5,4 @@ resources: images: - name: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller newName: public.ecr.aws/j1r0q0g6/notebooks/notebook-controller - newTag: v1.4-rc.1 + newTag: v1.4 diff --git a/apps/profiles/upstream/base/kustomization.yaml b/apps/profiles/upstream/base/kustomization.yaml index 91f08bfa58..28290c6cc3 100644 --- a/apps/profiles/upstream/base/kustomization.yaml +++ b/apps/profiles/upstream/base/kustomization.yaml @@ -12,7 +12,7 @@ patchesStrategicMerge: images: - name: public.ecr.aws/j1r0q0g6/notebooks/profile-controller newName: public.ecr.aws/j1r0q0g6/notebooks/profile-controller - newTag: v1.4-rc.1 + newTag: v1.4 configMapGenerator: - name: namespace-labels-data diff --git a/apps/profiles/upstream/overlays/kubeflow/kustomization.yaml b/apps/profiles/upstream/overlays/kubeflow/kustomization.yaml index d162934664..7467918225 100644 --- a/apps/profiles/upstream/overlays/kubeflow/kustomization.yaml +++ b/apps/profiles/upstream/overlays/kubeflow/kustomization.yaml @@ -28,4 +28,4 @@ vars: images: - name: public.ecr.aws/j1r0q0g6/notebooks/access-management newName: public.ecr.aws/j1r0q0g6/notebooks/access-management - newTag: v1.4-rc.1 + newTag: v1.4 diff --git a/apps/tensorboard/tensorboard-controller/upstream/base/kustomization.yaml b/apps/tensorboard/tensorboard-controller/upstream/base/kustomization.yaml index 5414c64d92..d8e74f367f 100644 --- a/apps/tensorboard/tensorboard-controller/upstream/base/kustomization.yaml +++ b/apps/tensorboard/tensorboard-controller/upstream/base/kustomization.yaml @@ -36,4 +36,4 @@ patches: images: - name: public.ecr.aws/j1r0q0g6/notebooks/tensorboard-controller newName: public.ecr.aws/j1r0q0g6/notebooks/tensorboard-controller - newTag: v1.4-rc.1 + newTag: v1.4 diff --git a/apps/tensorboard/tensorboards-web-app/upstream/base/kustomization.yaml b/apps/tensorboard/tensorboards-web-app/upstream/base/kustomization.yaml index f0c9c0a9f9..45afdcf2a3 100644 --- a/apps/tensorboard/tensorboards-web-app/upstream/base/kustomization.yaml +++ b/apps/tensorboard/tensorboards-web-app/upstream/base/kustomization.yaml @@ -14,7 +14,7 @@ commonLabels: images: - name: public.ecr.aws/j1r0q0g6/notebooks/tensorboards-web-app newName: public.ecr.aws/j1r0q0g6/notebooks/tensorboards-web-app - newTag: v1.4-rc.1 + newTag: v1.4 # We need the name to be unique without the suffix because the original name is what # gets used with patches configMapGenerator: diff --git a/apps/volumes-web-app/upstream/base/kustomization.yaml b/apps/volumes-web-app/upstream/base/kustomization.yaml index 1d3658573f..38b5f7ea48 100644 --- a/apps/volumes-web-app/upstream/base/kustomization.yaml +++ b/apps/volumes-web-app/upstream/base/kustomization.yaml @@ -14,7 +14,7 @@ commonLabels: images: - name: public.ecr.aws/j1r0q0g6/notebooks/volumes-web-app newName: public.ecr.aws/j1r0q0g6/notebooks/volumes-web-app - newTag: v1.4-rc.1 + newTag: v1.4 # We need the name to be unique without the suffix because the original name is what # gets used with patches configMapGenerator: diff --git a/common/knative/knative-serving/overlays/gateways/kustomization.yaml b/common/knative/knative-serving/overlays/gateways/kustomization.yaml new file mode 100644 index 0000000000..b9a3b155b8 --- /dev/null +++ b/common/knative/knative-serving/overlays/gateways/kustomization.yaml @@ -0,0 +1,10 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: +- ../../base + +# We want to Knative to use Istio's local Gateway and not the Ingress Gateway +# See: https://github.com/kubeflow/manifests/issues/1966 +patchesStrategicMerge: +- patches/gateway-selectors.yaml diff --git a/common/knative/knative-serving/overlays/gateways/patches/gateway-selectors.yaml b/common/knative/knative-serving/overlays/gateways/patches/gateway-selectors.yaml new file mode 100644 index 0000000000..fd290ec35e --- /dev/null +++ b/common/knative/knative-serving/overlays/gateways/patches/gateway-selectors.yaml @@ -0,0 +1,19 @@ +apiVersion: networking.istio.io/v1alpha3 +kind: Gateway +metadata: + name: knative-local-gateway + namespace: knative-serving +spec: + selector: + app: cluster-local-gateway + istio: cluster-local-gateway +--- +apiVersion: v1 +kind: Service +metadata: + name: knative-local-gateway + namespace: istio-system +spec: + selector: + app: cluster-local-gateway + istio: cluster-local-gateway