Run acceptance tests across multiple versions of Kubernetes

[coryflucas]

Description

Per the discussion in PR 2561 it'd be good to run the acceptance tests against multiple versions of Kubernetes to ensure that changes work across the different versions.

A couple questions I have with regards to this:

• What range of versions do we want to test against? My suggestion would be to a newer version of kind and use the latest minor versions supported by that version. As an example we could use kind v0.23.0 and Kubernetes versions 1.25-1.30.
• Do we want to make this change to the acceptance tests run against cloud providers? Given those have a cost associated and the kind tests should be covering roughly the same thing I would expect not. If we do, what version range? If we don't, should we increase the version used in those?

pinging @BBBmau from the PR discussion.

References

PR with initial discussion: PR 2561

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[BBBmau]

What range of versions do we want to test against? My suggestion would be to a newer version of kind and use the latest minor versions supported by that version. As an example we could use kind v0.23.0 and Kubernetes versions 1.25-1.30.

we actually have a range already when testing the manifest resource so we can have it match what's already covered there:

terraform-provider-kubernetes/.github/workflows/manifest_acc.yaml

Lines 28 to 35 in ef67d17

	matrix:
	kubernetes_version:
	# kind images: https://github.com/kubernetes-sigs/kind/releases
	- v1.30.0@sha256:047357ac0cfea04663786a612ba1eaba9702bef25227a794b52890dd8bcd692e
	- v1.28.9@sha256:dca54bc6a6079dd34699d53d7d4ffa2e853e46a20cd12d619a09207e35300bd0
	- v1.26.6@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1
	- v1.25.11@sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315
	- v1.23.15@sha256:ef453bb7c79f0e3caba88d2067d4196f427794086a7d0df8df4f019d5e336b61

Do we want to make this change to the acceptance tests run against cloud providers? Given those have a cost associated and the kind tests should be covering roughly the same thing I would expect not. If we do, what version range? If we don't, should we increase the version used in those?

Agreed, we wouldn't want to do the multiple k8s test on the cloud provider tests, kind tests are sufficient.

Thanks for opening this @coryflucas !