(MINOR) New methods for Kubernetes job assertion and EKS interactions (…

…#26)

### SUMMARY

This resolves #21 , with the exception that currently the reason for the job failure (e.g. job logs) is not outputted by the test. Streaming / capturing logs proved to be more complex, so a separate issue will be opened to enhance the method in the future.

* A new package `k8s` has been added with some methods around asserting things against Kubernetes clusters.
* New methods have been added to the `aws` package around interacting with EKS clusters. 
* An integration test using `kind` has been added for this new method. A new Make target of `k8s-integration-test` has been added to run this.
* The GitHub workflow for validating PRs has been updated to run these integrations tests with a matrix of k8s versions from 1.19-1.21.

### DEVELOPER IMPACT

#### ADDED
* A new `k8s` package has been added with numerous methods around asserting tests against Kubernetes clusters:
  * `k8s.AssertJobSucceeds`: This method runs a job against a Kubernetes cluster, and asserts that it finishes successfully.
  * `k8s.GetClientsetE`: This method is a wrapper around some Kubernetes API methods that makes constructing a new [`Clientset`](https://pkg.go.dev/k8s.io/client-go@v0.23.1/kubernetes#Clientset) object easy in various ways. Several functional option methods have been added in support of this, including:
    * `k8s.WithGetClientsetEHost`: This method configures a host name to be used by the clientset.
    * `k8s.WithGetClientsetEToken`: This method configures a bearer token to be used by the clientset.
    * `k8s.WithGetClientsetETLSCAData`: This method configures the CA certificate to be trusted by the clientset when connecting to the Kubernetes API endpoint.
    * `k8s.WithGetClientsetEKubeconfigPath`: This method configures the clientset using an existing config file. It should only be used in isolation for best results. 
* New methods around EKS clusters have been added to the `aws` package.
  * `aws.GetEKSClusterE`: This method retrieves useful information around EKS clusters, such as cluster endpoints.
  * `aws.GetEKSTokenE`: This method returns a bearer token that can be used to authenticate against EKS clusters using the Kubernetes `client-go` library and other methods in the `k8s` package.
#### CHANGED
* The module now requires Go v1.17 at a minimum. This is due to an upstream issue with the Kubernetes API library; please see [this PR comment](kubernetes/kubernetes#106666 (comment)).

.github/workflows/main.yaml

@@ -16,7 +16,7 @@ jobs:
       - name: Set up Golang
         uses: actions/setup-go@v1
         with:
-          go-version: 1.16
+          go-version: 1.17
       - name: Check out source code
         uses: actions/checkout@v2
         with:

.github/workflows/pull_request.yaml

@@ -10,14 +10,29 @@ jobs:
       - name: Set up Golang
         uses: actions/setup-go@v1
         with:
-          go-version: 1.16
+          go-version: 1.17
       - name: Check out source code
         uses: actions/checkout@v2
       - name: Run tests
         run: make
-      # Left here for future consideration
-      # - name: Comment PR
-      #   uses: thollander/actions-comment-pull-request@v1
-      #   with:
-      #     message: 'Example of message !'
-      #     GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+  k8s_integration_test:
+    name: Run k8s integration tests
+    runs-on: ubuntu-20.04
+    needs: test
+    strategy:
+      fail-fast: false
+      matrix:
+        k8s_version:
+          - 1.22.4          
+          - 1.21.2
+          - 1.20.7
+          - 1.19.11
+    steps:
+      - name: Set up Golang
+        uses: actions/setup-go@v1
+        with:
+          go-version: 1.17
+      - name: Check out source code
+        uses: actions/checkout@v2
+      - name: Run k8s integration tests
+        run: make k8s-integration-test K8S_VERSION=${{ matrix.k8s_version }}

Makefile

@@ -2,6 +2,13 @@ SOURCE = ./...
 
 .DEFAULT_GOAL := test
 
+export SHELL:=/bin/bash
+export SHELLOPTS:=$(if $(SHELLOPTS),$(SHELLOPTS):)pipefail:errexit
+
+export K8S_VERSION:=1.21.1
+
+.ONESHELL:
+
 vet:
 	go vet $(SOURCE)
 .PHONY: vet
@@ -11,7 +18,7 @@ test-fmt:
 .PHONY: test-fmt
 
 test: vet test-fmt
-	go test -cover $(SOURCE) -count=1
+	go test -cover ./pkg/... -count=1
 .PHONY: test
 
 tools:
@@ -22,4 +29,12 @@ tools:
 mock: tools
 	mockgen -source pkg/aws/dax.go -destination mock/dax.go -package mock
 	mockgen -source pkg/aws/ec2.go -destination mock/ec2.go -package mock
+	mockgen -source pkg/aws/eks.go -destination mock/eks.go -package mock
+	mockgen -source pkg/k8s/jobs.go -destination mock/k8s_jobs.go -package mock
+	mockgen -source pkg/k8s/util.go -destination mock/k8s_util.go -package mock
 .PHONY: mock
+
+.PHONY: k8s-integration-test
+k8s-integration-test:
+	echo K8S_VERSION: $(K8S_VERSION)
+	go test -v -timeout 10m -count 1 ./integration/k8s_test.go

README.md

@@ -91,9 +91,10 @@ used unless there is a compelling reason to use something else.
 
 | Library Name  / URL                             | Used For                                                                  |
 |-------------------------------------------------|---------------------------------------------------------------------------|
-| github.com/stretchr/testify/assert              | Asserting actual values equal some expected value.                        |
+| github.com/stretchr/testify                     | Asserting actual values equal some expected value.                        |
 | github.com/aws/aws-sdk-go-v2                    | All AWS related interactions.                                             |
 | gopkg.in/square/go-jose.v2/json                 | JSON manipulation, marshalling, etc                                       |
+| k8s.io/client-go                                | Kubernetes interactions                                                   |
 
 ### Use of interfaces rather than direct types
 

go.mod

@@ -2,17 +2,90 @@
 // See the LICENSE file for license information.
 module github.com/hbocodelabs/infratest
 
-go 1.16
+go 1.17
 
 require (
 	github.com/aws/aws-sdk-go-v2/service/dax v1.6.0
 	github.com/aws/aws-sdk-go-v2/service/ec2 v1.12.0
+	github.com/aws/aws-sdk-go-v2/service/eks v1.18.0
 	github.com/aws/aws-sdk-go-v2/service/iam v1.11.0
 	github.com/aws/aws-sdk-go-v2/service/route53 v1.12.0
-	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/golang/mock v1.6.0
-	github.com/kr/pretty v0.2.0 // indirect
-	github.com/stretchr/testify v1.6.1
-	gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
-	gopkg.in/square/go-jose.v2 v2.2.2
+	github.com/gruntwork-io/terratest v0.38.8
+	github.com/stretchr/testify v1.7.0
+	gopkg.in/square/go-jose.v2 v2.5.1
+	k8s.io/api v0.23.1
+	k8s.io/apimachinery v0.23.1
+	k8s.io/client-go v0.23.1
+	sigs.k8s.io/aws-iam-authenticator v0.5.3
+	sigs.k8s.io/kind v0.11.1
+)
+
+require (
+	cloud.google.com/go v0.83.0 // indirect
+	github.com/BurntSushi/toml v0.3.1 // indirect
+	github.com/alessio/shellescape v1.4.1 // indirect
+	github.com/aws/aws-sdk-go v1.40.56 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.13.0 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.4 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.2.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.2.1 // indirect
+	github.com/aws/smithy-go v1.10.0 // indirect
+	github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect
+	github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/evanphx/json-patch/v5 v5.2.0 // indirect
+	github.com/go-errors/errors v1.0.2-0.20180813162953-d98b870cc4e0 // indirect
+	github.com/go-logr/logr v1.2.0 // indirect
+	github.com/go-sql-driver/mysql v1.4.1 // indirect
+	github.com/gofrs/flock v0.7.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/google/go-cmp v0.5.6 // indirect
+	github.com/google/gofuzz v1.1.0 // indirect
+	github.com/google/uuid v1.2.0 // indirect
+	github.com/googleapis/gnostic v0.5.5 // indirect
+	github.com/gruntwork-io/go-commons v0.8.0 // indirect
+	github.com/hashicorp/errwrap v1.0.0 // indirect
+	github.com/hashicorp/go-multierror v1.1.0 // indirect
+	github.com/imdario/mergo v0.3.11 // indirect
+	github.com/inconshreveable/mousetrap v1.0.0 // indirect
+	github.com/jmespath/go-jmespath v0.4.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/mattn/go-isatty v0.0.12 // indirect
+	github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 // indirect
+	github.com/mitchellh/go-homedir v1.1.0 // indirect
+	github.com/moby/spdystream v0.2.0 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pelletier/go-toml v1.9.3 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/pquerna/otp v1.2.0 // indirect
+	github.com/russross/blackfriday/v2 v2.1.0 // indirect
+	github.com/sirupsen/logrus v1.8.1 // indirect
+	github.com/spf13/cobra v1.2.1 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	github.com/urfave/cli v1.22.2 // indirect
+	golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
+	golang.org/x/mod v0.4.2 // indirect
+	golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect
+	golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect
+	golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e // indirect
+	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect
+	golang.org/x/text v0.3.7 // indirect
+	golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
+	golang.org/x/tools v0.1.5 // indirect
+	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/protobuf v1.27.1 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
+	k8s.io/klog/v2 v2.30.0 // indirect
+	k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect
+	k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b // indirect
+	sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.1.2 // indirect
+	sigs.k8s.io/yaml v1.2.0 // indirect
 )