The Kubernetes Conformance test suite is a subset of e2e tests that SIG Architecture has approved to define the core set of interoperable features that all conformant Kubernetes clusters must support. The tests verify that the expected behaviour works as a user might encounter it in the wild.ii.coop have test wrties that create e2e test for testing API Endpoints to confirm conformance.
ii.coop have made the process easier for to on-board test writers by creating tooling that make a very complect task manageble. In bdorder for ii’s test writers to be able to create conformance test for endpoints that are not covered yet, an analitic tool is needed. The tools in our toolbox are: APISnoop for identifying on untested end points, Test are writen in .org file using the ii version of Spacemacs called Kubemacs, as well as some special org file to bring up the test environment and templates.
Creating new tests: In Kubemacs the the test-writing.org file is used to set-up the environment to test writing. Mock ticket template.org file is used to run a query to identify remaining endpoints which are untested.
Once suiteble endpoints have been identified a mock test is created:
- A test outline is created in plain language explaining the test steps.
- The Test functionality is written in Go to test if the intended end points is covered.
3.Verify the increase of coverage with APISnoop. 4.Update the file with the number of new en points hit by the new test.
- Export the file to .MD format
The .org & .MD file is pushed to a branch on https://github.com/cncf/apisnoop and a PR (pull request) is created and asigned to the Project mananager. The ii.coop Project manager must process the PR and create and Issue.
Test lifecycle: The Project manager will create a new Issue in https://github.com/kubernetes/kubernetes The heading of the issue will contain the endpoints as discribed in the .org file name as well as the number of new endpoint coved by the new test. For example Write RESOURCENAME test+promote - +X endpoint coverage The MD file will be copied into the body of the Issue. Hyperlinks will be created to the .org file as well as the Issue in the heading of the issue. The new issue must be placed in the Kubernetes project board in Issues to Triage colmn. In the every bi-weekly Conformance office hours new Issues will be discussed and agreement will be reached on the next steps. When an Issue is given the go ahead, the Test writer can write the test, including all changes proposed in the Issue by the meeting participants and broader community. When the test is ready to be submitted the .org and MD files is push to a new branch in https://github.com/ii/kubernetes/ and the Project Manager is informed via a Pull Request. The branch must be named: ii-RESOURCENAME. The Project manager will create a Pull Request for the new test.