After modifying the virtualrouter/virtualnode k8s manifest file it not update changes inPlace

[gaurav04]

We are managing the appMesh resource from K8s manifest file for that we installed appMesh controller in K8s. Every time when we make small changes in virtualNode or virtualRouter and run terraform again it ask for destroy and create instead of in-place upgrade. When it try to delete the resource it failed because virtualNode has dependency on VirtualRouter, VirtualService and VirtualGateway.

Terraform Version, Provider Version and Kubernetes Version

Terraform version: v1.5.0
Kubernetes provider version: v2.22.0 
Kubernetes version: 1.26

Affected Resource(s)

Terraform Configuration Files

  manifest = {
    apiVersion = "appmesh.k8s.aws/v1beta2"
    kind       = "VirtualRouter"

    metadata = {
      name      = var.name
    }
    spec = {
      listeners = [
        {
          portMapping = {
            port     = 8080
            protocol = "http"
          }
        },
      ]

      routes = [
        {
          httpRoute = {
            action = {
              weightedTargets = [
                {
                  virtualNodeRef = {
                    name = var.name
                  }
                  weight = 100
                }
              ]
            }
            match = {
              prefix = "/"
            }
            retryPolicy = {
              httpRetryEvents = ["server-error", "gateway-error"]
              maxRetries      = 3
              perRetryTimeout = {
                unit  = "ms"
                value = 5000
              }
              tcpRetryEvents = ["connection-error"]
            }
          }
          name = var.name
        },
      ]
    }
  }
}

Panic Output

-/+ resource "kubernetes_manifest" "virtualrouter" {
      ~ manifest = {
          ~ spec       = {
              ~ routes    = [
                  ~ {
                      ~ httpRoute = {
                          ~ retryPolicy = {
                              ~ perRetryTimeout = {
                                  ~ value = 10000 -> 5000
                                    # (1 unchanged attribute hidden)
                                }
                                # (3 unchanged attributes hidden)
                            }
                            # (2 unchanged attributes hidden)
                        }
                        name      = "test"
                    },
                ]
                # (1 unchanged attribute hidden)
            }
            # (3 unchanged attributes hidden)
        }
      ~ object   = {
          ~ metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              ~ deletionGracePeriodSeconds = 0 -> (known after apply)
              ~ deletionTimestamp          = "2023-07-21T09:10:32Z" -> (known after apply)
              ~ finalizers                 = [
                  - "finalizers.appmesh.k8s.aws/aws-appmesh-resources",
                ] -> (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
                name                       = "test"
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply) 

Steps to Reproduce

After changing perRetryTimeout from 10000 to 5000

Expected Behavior

Plan: 0 to add, 1 to change, 0 to destroy.

Actual Behavior

Plan: 1 to add, 1 to change, 1 to destroy.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[gaurav04]

This issue impacting our prod environment. Please help on this. Thanks

[sheneska]

Hi @gaurav04, Terraform doesn't tolerate schema change through a lifecycle and the schema changes because of the attribute in the CRD.

[gaurav04]

hi @sheneska thanks for reply. Can you help with any workaround to fix this issue that would really help us. Thanks

[gaurav04]

fixed.