kubernetes_manifest crashes during plan of argocd applicationset with matrix generator CR

[kacurez]

Terraform Version, Provider Version and Kubernetes Version

Terraform version: v1.5.7
Kubernetes provider version: v2.30.0
Kubernetes version: v1.27

Affected Resource(s)

• kubernetes_manifest

Terraform Configuration Files

# manifest similar to https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Generators-Matrix/#overriding-parameters-from-one-child-generator-in-another-child-generator
resource "kubernetes_manifest" "application_set" {
  manifest = {
    apiVersion = "argoproj.io/v1alpha1"
    kind       = "ApplicationSet"
    metadata = {
      name      = "my-argocd-app-set"
      namespace = "argocd"
    }
    spec = {
      generators = [
        {
          matrix = {
            generators = [
              {
                git = {
                  repoURL         = "https://github.com/argoproj/argocd-example-apps"
                  revision        = "main"
                  pathParamPrefix = "firstApp"
                  files = [
                    {
                      path = "apps/templates/*.yaml"
                    }
                  ]
                }
              },
              {
                git = {
                  repoURL         = "https://github.com/argoproj/argocd-example-apps"
                  revision        = "main"
                  pathParamPrefix = "secondApp"
                  files = [
                    {
                      path = "apps/templates/*.yaml"
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
      template = {
        metadata = {
          name = "{{firstApp.path.basename}}"
        }
        spec = {
          project = "default"
          source = {
            repoURL        = "https://github.com/argoproj/argocd-example-apps"
            targetRevision = "main"
          }
          destination = {
            server    = "https://kubernetes.default.svc"
            namespace = "default"
          }
        }
      }
    }
  }
}

Debug Output

https://gist.github.com/kacurez/d823f2de22766dbf8f4ba45df7a72447

Panic Output

https://gist.github.com/kacurez/d823f2de22766dbf8f4ba45df7a72447

Steps to Reproduce

1. install argocd e.g via helm https://github.com/argoproj/argo-helm/tree/main/charts/argo-cd
2. define the kubernetes_manifest as specified above
3. terraform init
4. terraform plan

Expected Behavior

terraform plan doesn't crash and succeed to plan. Consequent terraform apply works as well by creating ApplicationSet CR

Actual Behavior

terraform plan crashes, see https://gist.github.com/kacurez/d823f2de22766dbf8f4ba45df7a72447

Important Factoids

If I run it with a single git generator then it works, for example:

      generators = [
        {
          matrix = {
            generators = [
              {
                git = {
                  repoURL         = "https://github.com/argoproj/argocd-example-apps"
                  revision        = "main"
                  pathParamPrefix = "secondApp"
                  files = [
                    {
                      path = "apps/templates/*.yaml"
                    }
                  ]
                }
              }
            ]
          }
        }
      ]

so the problem is most likely related to the generators array failing to proceed with 2 items.

References

• same issue but got closed kubernetes_manifest: can't use tftypes.Object #1636

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[eli-nomasec]

Same here

[gablemire]

I'm having the same behaviour (with the same error stack) with version 2.31.0. When I apply the ApplicationSet manually through kubectl apply -f <my-appset.yaml>, it is successfully applied.

[ivankorn]

I logged similar issue, the only difference - I use the new fancy provider function

resource "kubernetes_manifest" "argocd_application_set" {
  manifest = provider::kubernetes::manifest_decode(file("argocd/application-set.yaml"))

  depends_on = [helm_release.chart]
}

If anyone is interested in temporary workaround for this issue, below is the code I use for that.

# Workaround for https://github.com/hashicorp/terraform-provider-kubernetes/issues/2580
module "kubectl-argocd-application-set" {
  source  = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"
  version = "~> 3.4"

  project_id        = local.gcp_project_id
  cluster_name      = module.gke.name
  cluster_location  = module.gke.location
  module_depends_on = [module.gke.endpoint]

  kubectl_create_command  = "kubectl create -f argocd/application-set.yaml"
  kubectl_destroy_command = "kubectl delete -f argocd/application-set.yaml"
  skip_download           = true
}