Install Sveltos as a Provider Template

[wahabmk]

Description

This fixes #269.

1. Sveltos is installed (via it's helm chart) as a provider type template.
2. The reason it is installed as a separate template rather than as a subchart within the HMC template is because Sveltos needs to be installed in the projectsveltos namespace for it to function.
3. We already tried installing Sveltos in hmc-system namespace but it didn't work properly, the details of which can be seen here.

Manual Testing

Installed with AWS provider and waited for everything to be in running state.

Management Cluster

• Verified that Sveltos template has been created:

➜  ~ kubectl get template -A
NAMESPACE    NAME                         TYPE         VALID
hmc-system   aws-hosted-cp                deployment   true
hmc-system   aws-standalone-cp            deployment   true
hmc-system   azure-hosted-cp              deployment   true
hmc-system   azure-standalone-cp          deployment   true
hmc-system   cluster-api                  core         true
hmc-system   cluster-api-provider-aws     provider     true
hmc-system   cluster-api-provider-azure   provider     true
hmc-system   hmc                          core         true
hmc-system   k0smotron                    provider     true
hmc-system   projectsveltos               provider     true

• Verified all HelmReleases have been successfully reconciled:

➜  ~ kubectl get helmreleases -A
NAMESPACE    NAME                         AGE   READY   STATUS
hmc-system   cluster-api                  15m   True    Helm install succeeded for release hmc-system/cluster-api.v1 with chart cluster-api@0.1.1
hmc-system   cluster-api-provider-aws     15m   True    Helm install succeeded for release hmc-system/cluster-api-provider-aws.v1 with chart cluster-api-provider-aws@0.1.1
hmc-system   cluster-api-provider-azure   15m   True    Helm install succeeded for release hmc-system/cluster-api-provider-azure.v1 with chart cluster-api-provider-azure@0.0.1
hmc-system   hmc                          15m   True    Helm upgrade succeeded for release hmc-system/hmc.v2 with chart hmc@0.1.0
hmc-system   k0smotron                    15m   True    Helm install succeeded for release hmc-system/k0smotron.v1 with chart k0smotron@0.1.3
hmc-system   projectsveltos               15m   True    Helm install succeeded for release projectsveltos/projectsveltos.v1 with chart projectsveltos@0.37.1
hmc-system   wali-aws-dev                 10m   True    Helm install succeeded for release hmc-system/wali-aws-dev.v1 with chart aws-standalone-cp@0.1.2

• Verified that components of Sveltos can be seen running in the projectsveltos namespace:

➜  ~ kubectl get deployments -A
NAMESPACE            NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
hmc-system           azureserviceoperator-controller-manager       1/1     1            1           12m
hmc-system           capa-controller-manager                       1/1     1            1           13m
hmc-system           capi-controller-manager                       1/1     1            1           14m
hmc-system           capz-controller-manager                       1/1     1            1           12m
hmc-system           helm-controller                               1/1     1            1           15m
hmc-system           hmc-cert-manager                              1/1     1            1           15m
hmc-system           hmc-cert-manager-cainjector                   1/1     1            1           15m
hmc-system           hmc-cert-manager-webhook                      1/1     1            1           15m
hmc-system           hmc-cluster-api-operator                      1/1     1            1           14m
hmc-system           hmc-controller-manager                        1/1     1            1           15m
hmc-system           k0smotron-controller-manager-bootstrap        1/1     1            1           13m
hmc-system           k0smotron-controller-manager-control-plane    1/1     1            1           13m
hmc-system           k0smotron-controller-manager-infrastructure   1/1     1            1           12m
hmc-system           source-controller                             1/1     1            1           15m
kube-system          coredns                                       2/2     2            2           16m
local-path-storage   local-path-provisioner                        1/1     1            1           16m
projectsveltos       access-manager                                1/1     1            1           13m
projectsveltos       addon-controller                              1/1     1            1           13m
projectsveltos       classifier-manager                            1/1     1            1           13m
projectsveltos       conversion-webhook                            1/1     1            1           13m
projectsveltos       event-manager                                 1/1     1            1           13m
projectsveltos       hc-manager                                    1/1     1            1           13m
projectsveltos       sc-manager                                    1/1     1            1           13m
projectsveltos       shard-controller                              1/1     1            1           13m
projectsveltos       sveltos-agent-manager                         1/1     1            1           12m

• Verified that Sveltos's default classifier has been installed:

➜  ~ kubectl get classifier
NAME                 AGE
default-classifier   13m

• Labeled the target cluster with env=fv:

➜  ~ kubectl get cluster -A --show-labels
NAMESPACE    NAME           CLUSTERCLASS   PHASE         AGE   VERSION   LABELS
hmc-system   wali-aws-dev                  Provisioned   11m             app.kubernetes.io/managed-by=Helm,env=fv,helm.toolkit.fluxcd.io/name=wali-aws-dev,helm.toolkit.fluxcd.io/namespace=hmc-system,sveltos-agent=present

• Finally created the following cluster profile:

apiVersion: config.projectsveltos.io/v1beta1
kind: ClusterProfile
metadata:
  name: kyverno
spec:
  clusterSelector:
    matchLabels:
      env: fv
  helmCharts:
    - repositoryURL:    https://kyverno.github.io/kyverno/
      repositoryName:   kyverno
      chartName:        kyverno/kyverno
      chartVersion:     v3.2.5
      releaseName:      kyverno-latest
      releaseNamespace: kyverno
      helmChartAction:  Install

➜  ~ kubectl get clusterprofile                
NAME      AGE
kyverno   118m

Target Cluster

As can be seen below, Sveltos is running in projectsveltos namespace and kyverno has also been installed in the target cluster.

➜  ~ kubectl get deployments -A
NAMESPACE        NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
kube-system      calico-kube-controllers         1/1     1            1           7m33s
kube-system      coredns                         2/2     2            2           7m37s
kube-system      ebs-csi-controller              2/2     2            2           7m17s
kube-system      metrics-server                  1/1     1            1           7m28s
kyverno          kyverno-admission-controller    1/1     1            1           40s
kyverno          kyverno-background-controller   1/1     1            1           40s
kyverno          kyverno-cleanup-controller      1/1     1            1           40s
kyverno          kyverno-reports-controller      1/1     1            1           40s
projectsveltos   sveltos-agent-manager           1/1     1            1           7m3s

[a13x5]

Sveltos chart should be referenced, not copied over. This way we doomed to copy it on every update.
Just create an empty chart and reference sveltos as the only subchart.

[wahabmk]

@Kshatrix @a13x5 Included an empty helm chart and added projectsveltos chart as its dependency and rebased from main branch to pull in changes made in #287.

[a13x5]

@wahabmk minor typo in version.
Rest looks good

[a13x5]

Inconsistent version here

[wahabmk]

Oh that was intentional because the 0.37.1 chart version did not have change in app version. However, Sveltos has since released 0.38.1 so I've updated it to that.

[a13x5]

LGTM