Skip to content

Latest commit

 

History

History
264 lines (190 loc) · 9.12 KB

testing.md

File metadata and controls

264 lines (190 loc) · 9.12 KB
Raw

WARNING WARNING WARNING WARNING WARNING

PLEASE NOTE: This document applies to the HEAD of the source tree

If you are using a released version of Kubernetes, you should refer to the docs that go with that version.

The latest release of this document can be found [here](http://releases.k8s.io/release-1.4/docs/devel/testing.md).

Documentation for other releases can be found at releases.k8s.io.

Testing guide

Updated: 5/21/2016

Table of Contents

This assumes you already read the development guide to install go, godeps, and configure your git client. All command examples are relative to the kubernetes root directory.

Before sending pull requests you should at least make sure your changes have passed both unit and integration tests.

Kubernetes only merges pull requests when unit, integration, and e2e tests are passing, so it is often a good idea to make sure the e2e tests work as well.

Unit tests

  • Unit tests should be fully hermetic
    • Only access resources in the test binary.
  • All packages and any significant files require unit tests.
  • The preferred method of testing multiple scenarios or input is table driven testing
  • Unit tests must pass on OS X and Windows platforms.
    • Tests using linux-specific features must be skipped or compiled out.
    • Skipped is better, compiled out is required when it won't compile.
  • Concurrent unit test runs must pass.
  • See coding conventions.

Run all unit tests

make test is the entrypoint for running the unit tests that ensures that GOPATH is set up correctly. If you have GOPATH set up correctly, you can also just use go test directly.

cd kubernetes
make test  # Run all unit tests.

Set go flags during unit tests

You can set go flags by setting the KUBE_GOFLAGS environment variable.

Run unit tests from certain packages

make test accepts packages as arguments; the k8s.io/kubernetes prefix is added automatically to these:

make test WHAT=pkg/api                # run tests for pkg/api

To run multiple targets you need quotes:

make test WHAT="pkg/api pkg/kubelet"  # run tests for pkg/api and pkg/kubelet

In a shell, it's often handy to use brace expansion:

make test WHAT=pkg/{api,kubelet}  # run tests for pkg/api and pkg/kubelet

Run specific unit test cases in a package

You can set the test args using the KUBE_TEST_ARGS environment variable. You can use this to pass the -run argument to go test, which accepts a regular expression for the name of the test that should be run.

# Runs TestValidatePod in pkg/api/validation with the verbose flag set
make test WHAT=pkg/api/validation KUBE_GOFLAGS="-v" KUBE_TEST_ARGS='-run ^TestValidatePod$'

# Runs tests that match the regex ValidatePod|ValidateConfigMap in pkg/api/validation
make test WHAT=pkg/api/validation KUBE_GOFLAGS="-v" KUBE_TEST_ARGS="-run ValidatePod\|ValidateConfigMap$"

For other supported test flags, see the golang documentation.

Stress running unit tests

Running the same tests repeatedly is one way to root out flakes. You can do this efficiently.

# Have 2 workers run all tests 5 times each (10 total iterations).
make test PARALLEL=2 ITERATION=5

For more advanced ideas please see flaky-tests.md.

Unit test coverage

Currently, collecting coverage is only supported for the Go unit tests.

To run all unit tests and generate an HTML coverage report, run the following:

make test KUBE_COVER=y

At the end of the run, an HTML report will be generated with the path printed to stdout.

To run tests and collect coverage in only one package, pass its relative path under the kubernetes directory as an argument, for example:

make test WHAT=pkg/kubectl KUBE_COVER=y

Multiple arguments can be passed, in which case the coverage results will be combined for all tests run.

Benchmark unit tests

To run benchmark tests, you'll typically use something like:

go test ./pkg/apiserver -benchmem -run=XXX -bench=BenchmarkWatch

This will do the following:

  1. -run=XXX is a regular expression filter on the name of test cases to run
  2. -bench=BenchmarkWatch will run test methods with BenchmarkWatch in the name
  • See grep -nr BenchmarkWatch . for examples
  1. -benchmem enables memory allocation stats

See go help test and go help testflag for additional info.

Integration tests

Install etcd dependency

Kubernetes integration tests require your PATH to include an etcd installation. Kubernetes includes a script to help install etcd on your machine.

# Install etcd and add to PATH

# Option a) install inside kubernetes root
hack/install-etcd.sh  # Installs in ./third_party/etcd
echo export PATH="$PATH:$(pwd)/third_party/etcd" >> ~/.profile  # Add to PATH

# Option b) install manually
grep -E "image.*etcd" cluster/saltbase/etcd/etcd.manifest  # Find version
# Install that version using yum/apt-get/etc
echo export PATH="$PATH:<LOCATION>" >> ~/.profile  # Add to PATH

Etcd test data

Many tests start an etcd server internally, storing test data in the operating system's temporary directory.

If you see test failures because the temporary directory does not have sufficient space, or is on a volume with unpredictable write latency, you can override the test data directory for those internal etcd instances with the TEST_ETCD_DIR environment variable.

Run integration tests

The integration tests are run using make test-integration. The Kubernetes integration tests are writting using the normal golang testing package but expect to have a running etcd instance to connect to. The test- integration.sh script wraps make test and sets up an etcd instance for the integration tests to use.

make test-integration  # Run all integration tests.

This script runs the golang tests in package test/integration.

Run a specific integration test

You can use also use the KUBE_TEST_ARGS environment variable with the hack /test-integration.sh script to run a specific integration test case:

# Run integration test TestPodUpdateActiveDeadlineSeconds with the verbose flag set.
make test-integration KUBE_GOFLAGS="-v" KUBE_TEST_ARGS="-run ^TestPodUpdateActiveDeadlineSeconds$"

If you set KUBE_TEST_ARGS, the test case will be run with only the v1 API version and the watch cache test is skipped.

End-to-End tests

Please refer to End-to-End Testing in Kubernetes.

Analytics