Better support for downgrades/rollbacks

[lbernick]

Pipelines currently tries to ensure backwards compatibility where possible, and our upgrade tests ensure that users can upgrade to a new version of Pipelines without breaking existing resources defined on their cluster.

However, we haven't put as much effort into smooth rollbacks. We don't explicitly support rollbacks (and I don't think we're at a point where it makes sense to make guarantees or policies-- see this comment from @vdemeester), but I'm creating this issue to gauge interest and start discussion about ways we could better support rollbacks.

Two ideas:

1. Change our feature flag logic: Right now we only allow certain fields if the feature flags gating those fields are turned on when a CRD is created/updated. In order to support safe rollbacks, we should allow a field if either the feature flag is enabled, or the object is being updated and already has the field present. This is what k8s does: see discussion here. This could be done as part of work on per feature flags (Per Feature Feature Flags #5632).
2. Introduce some kind of "downgrade testing" that tests that objects don't fail validation when downgrading tekton versions. Ex: On a kind cluster that has the latest tekton version and some pipelines/tasks, downgrade to the previous tekton version, check that existing pipelines/tasks don't fail validation (maybe via some trivial updates), and run previous release's integration tests.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale