ignoreDifferences doesn't apply desired manifest

[wustus]

Checklist:

• [ x ] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [ x ] I've included steps to reproduce the bug.
• [ x ] I've pasted the output of argocd version.

Describe the bug

Originally we were looking for a way to update immutable fields in StatefulSets. A solution was proposed here #5882.
After some more digging, I found this issue #5172 describing a solution which executes kubectl apply --force thus deleting and replacing the violated StatefulSet after 5 retries.
This generally works, but the application is stuck in a sync-cycle if it is deployed by a parent application. The entire

operation:
  sync:
    syncStrategy:
      apply:
        force: true

is added to the live manifest, then removed immediately, resulting in a sync every few seconds.

To circumvent this, it was proposed to add

spec:
  ignoreDifferences:
    - group: argoproj.io
      kind: Application
      jsonPointers:
        - /operation

to the parent‚ application. This should ignore any differences under the operation key but still apply the desired manifest as is. This is described here:

By default, Argo CD uses the ignoreDifferences config just for computing the diff between the live and desired state which defines if the application is synced or not. However during the sync stage, the desired state is applied as-is.

So when adding ignoreDifferences... to the parent application and operation... to the application, the sync cycle indeed stops but operation... is not applied to to the live manifest as expected. The desired manifest does show the operation object though.

The only (found) working solutions are to

1. add operation... to status.operationState, which messes with Argos internally handled state and has unknown consequences or
2. let the application wallow in the sync-cycle, which somehow seems worse

To Reproduce

Deploy a parent Application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: bootstrap-app
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  source:
    repoURL: https://github.com/username/argocd-apps.git
    targetRevision: master
    path: "argocd-apps"
    directory:
      recurse: true

  destination:
    name: my-destination
    namespace: system

  syncPolicy:
    automated:
      prune: false
      selfHeal: true
      allowEmpty: true
  
  ignoreDifferences:
    - group: argoproj.io
      kind: Application
      jsonPointers:
        - /operation

and the actual Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io

spec:
  source:
    helm:        
      valueFiles:
        - values.yaml

    repoURL: https://github.com/username/argocd-apps.git
    path: helm-charts/mychart
    targetRevision: master
  
  destination:
    name: my-destination
    namespace: application-ns
  syncPolicy:
    automated:
      prune: true
      allowEmpty: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

operation:
  sync:
    syncStrategy:
      apply:
        force: true

Expected behavior

The parent Application deploys the 'myapp' Application and shows

operation:
  sync:
    syncStrategy:
      apply:
        force: true

in its live manifest.

Screenshots

Version

{
    "Version": "v2.10.4+f5d63a5",
    "BuildDate": "2024-03-18T08:09:23Z",
    "GitCommit": "f5d63a5c77d2e804e51ef94bee3db441e0789d00",
    "GitTreeState": "clean",
    "GoVersion": "go1.21.3",
    "Compiler": "gc",
    "Platform": "linux/amd64",
    "KustomizeVersion": "v5.2.1 2023-10-19T20:13:51Z",
    "HelmVersion": "v3.14.3+gf03cc04",
    "KubectlVersion": "v0.26.11",
    "JsonnetVersion": "v0.20.0"
}

Thanks for your help. :)

[crenshaw-dev]

I think probably what we really need is this: argoproj/gitops-engine#560

I'm not sure that managing the operation field via GitOps is even meant to be a supported use case.

[wustus]

It's at least documented:
https://argo-cd.readthedocs.io/en/stable/user-guide/sync-kubectl/#apply-and-hook-synchronization-strategies

But I agree. This would fix the root problem.

[wustus]

Closing this as #17866 seems to be moving.
Also Argo seems to have 'caught itself' after a few days, so that the ignoreDifferences did take after a while.