Adding multi-platform support for linux/amd64 and linux/arm64 (#83)

* Adding multi-platform support for linux/amd64 and linux/arm64 * Bumped GHA versions Added `--platform` option to `docker build` commands in e2e tests Added `PLATFORM` variable to Makefile for local builds Golang version update Dependency updates Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Need to use `docker buildx` to get platform flags in Dockerfile Signed-off-by: Craig Ingram <cingram@us.ibm.com> * `docker buildx` is apparently not supported in GHA Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Trying to get `BUILDPLATFORM` down to Dockerfiles. Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Defaulted `BUILDPLATFORM` Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Updated tests referring to old v1beta policy Problem only shows when updating the kubebuilder-tools to 1.25.0 (when the v1beta policy was deprecated) Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Updating e2e-tests Added podman support for e2d tests Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Testing e2e-tests w/`docker buildx build` Tied to use the `--load` option instead of `--push` but didn't work May be a Rancher Desktop issue > error: docker exporter does not currently support exporting manifest lists Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Added logic to handle podman buildx behavior differences `podman buildx` auto-loads the image into the local docker repo `podman buildx` doesn't support `--push` arg `docker pull` is not necessary with podman Refactored create_dev_cluster.sh Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Updated e2e-test node versions Added both platforms to docker build commands in e2e-tests Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Can't specify multiple platforms w/regular docker build Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Updated quick-start.md Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Major refactoring of create_dev_cluster.sh Mainly for testing different platform and build configurations Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Typos and logic Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Typos and logic Signed-off-by: Craig Ingram <cingram@us.ibm.com> * Typos and logic Signed-off-by: Craig Ingram <cingram@us.ibm.com> Signed-off-by: Craig Ingram <cingram@us.ibm.com>
IBM · Oct 26, 2022 · 3f50ebb · 3f50ebb
1 parent bb23f7a
commit 3f50ebb
Show file tree

Hide file tree

Showing 14 changed files with 306 additions and 92 deletions.
diff --git a/.github/workflows/containers.yml b/.github/workflows/containers.yml
@@ -13,18 +13,19 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@v2
       - name: Get version
         id: get_version
         run: echo ::set-output name=VERSION::${GITHUB_REF##*/}
       - name: Login to DockerHub
-        uses: docker/login-action@v1
+        uses: docker/login-action@v2
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME}}
           password: ${{ secrets.DOCKERHUB_PASSWORD}}
       - name: Build and push
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@v3
         with:
+          platforms: linux/amd64,linux/arm64
           tags: ibmcom/varnish-operator:${{ steps.get_version.outputs.VERSION }}
           file: ./Dockerfile
           build-args: |
@@ -34,37 +35,39 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@v2
       - name: Get version
         id: get_version
         run: echo ::set-output name=VERSION::${GITHUB_REF##*/}
       - name: Login to DockerHub
-        uses: docker/login-action@v1
+        uses: docker/login-action@v2
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME}}
           password: ${{ secrets.DOCKERHUB_PASSWORD}}
       - name: Build and push
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@v3
         with:
+          platforms: linux/amd64,linux/arm64
           tags: ibmcom/varnish:${{ steps.get_version.outputs.VERSION }}
           file: ./Dockerfile.varnishd
           push: ${{ github.event_name == 'release' && github.event.action == 'created' }}
   varnish-controller:
     runs-on: ubuntu-latest
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@v2
       - name: Get version
         id: get_version
         run: echo ::set-output name=VERSION::${GITHUB_REF##*/}
       - name: Login to DockerHub
-        uses: docker/login-action@v1
+        uses: docker/login-action@v2
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME}}
           password: ${{ secrets.DOCKERHUB_PASSWORD}}
       - name: Build and push
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@v3
         with:
+          platforms: linux/amd64,linux/arm64
           tags: ibmcom/varnish-controller:${{ steps.get_version.outputs.VERSION }}
           file: ./Dockerfile.controller
           build-args: |
@@ -74,18 +77,19 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@v2
       - name: Get version
         id: get_version
         run: echo ::set-output name=VERSION::${GITHUB_REF##*/}
       - name: Login to DockerHub
-        uses: docker/login-action@v1
+        uses: docker/login-action@v2
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME}}
           password: ${{ secrets.DOCKERHUB_PASSWORD}}
       - name: Build and push
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@v3
         with:
+          platforms: linux/amd64,linux/arm64
           tags: ibmcom/varnish-metrics-exporter:${{ steps.get_version.outputs.VERSION }}
           file: ./Dockerfile.exporter
           push: ${{ github.event_name == 'release' && github.event.action == 'created' }}
diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml
@@ -10,16 +10,16 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        kubernetes-version: [ "1.24", "1.23", "1.22", "1.21"]
+        kubernetes-version: [ "1.25", "1.24", "1.23", "1.22"]
         include:
+        - kubernetes-version: "1.25"
+          kind-node: kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace
         - kubernetes-version: "1.24"
-          kind-node: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8e
+          kind-node: kindest/node:v1.24.6@sha256:97e8d00bc37a7598a0b32d1fabd155a96355c49fa0d4d4790aab0f161bf31be1
         - kubernetes-version: "1.23"
-          kind-node: kindest/node:v1.23.6@sha256:b1fa224cc6c7ff32455e0b1fd9cbfd3d3bc87ecaa8fcb06961ed1afb3db0f9ae
+          kind-node: kindest/node:v1.23.12@sha256:9402cf1330bbd3a0d097d2033fa489b2abe40d479cc5ef47d0b6a6960613148a
         - kubernetes-version: "1.22"
-          kind-node: kindest/node:v1.22.9@sha256:8135260b959dfe320206eb36b3aeda9cffcb262f4b44cda6b33f7bb73f453105
-        - kubernetes-version: "1.21"
-          kind-node: kindest/node:v1.21.12@sha256:f316b33dd88f8196379f38feb80545ef3ed44d9197dca1bfd48bcb1583210207
+          kind-node: kindest/node:v1.22.15@sha256:bfd5eaae36849bfb3c1e3b9442f3da17d730718248939d9d547e86bbac5da586
     steps:
       - name: Set up Go
         uses: actions/setup-go@v3
@@ -31,10 +31,10 @@ jobs:
         run: go mod download
       - name: build images
         run: |
-          docker build -f Dockerfile -t ibmcom/varnish-operator:local .
-          docker build -f Dockerfile.varnishd -t ibmcom/varnish:local .
-          docker build -f Dockerfile.controller -t ibmcom/varnish-controller:local .
-          docker build -f Dockerfile.exporter -t ibmcom/varnish-metrics-exporter:local .
+          docker build --platform linux/amd64 -f Dockerfile -t ibmcom/varnish-operator:local .
+          docker build --platform linux/amd64 -f Dockerfile.varnishd -t ibmcom/varnish:local .
+          docker build --platform linux/amd64 -f Dockerfile.controller -t ibmcom/varnish-controller:local .
+          docker build --platform linux/amd64 -f Dockerfile.exporter -t ibmcom/varnish-metrics-exporter:local .
       - name: Create k8s Kind Cluster ${{ matrix.kubernetes-version }}
         uses: helm/kind-action@v1.3.0
         with:

diff --git a/Dockerfile b/Dockerfile
@@ -1,4 +1,5 @@
-FROM golang:1.19.1-bullseye AS builder
+ARG BUILDPLATFORM=linux/amd64
+FROM --platform=$BUILDPLATFORM golang:1.19.2-bullseye AS builder
 
 ENV DEBIAN_FRONTEND=noninteractive INSTALL_DIRECTORY=/usr/local/bin
 
@@ -18,17 +19,20 @@ COPY cmd/ cmd/
 COPY pkg/ pkg/
 COPY api/ api/
 
+ARG TARGETOS=linux
+ARG TARGETARCH=amd64
 ARG VERSION=undefined
+
 # Build
-RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
+RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH \
     go build \
     -ldflags "-X main.Version=$VERSION" \
     -a \
     -o varnish-operator \
     github.com/ibm/varnish-operator/cmd/varnish-operator
 
 
-FROM debian:bullseye-slim
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim
 
 LABEL maintainer="Alex Lytvynenko <oleksandr.lytvynenko@ibm.com>, Tomash Sidei <tomash.sidei@ibm.com>"
 

diff --git a/Dockerfile.controller b/Dockerfile.controller
@@ -1,4 +1,5 @@
-FROM golang:1.19.1-bullseye AS builder
+ARG BUILDPLATFORM=linux/amd64
+FROM --platform=$BUILDPLATFORM golang:1.19.2-bullseye AS builder
 
 ENV DEBIAN_FRONTEND=noninteractive INSTALL_DIRECTORY=/usr/local/bin
 
@@ -15,23 +16,25 @@ COPY cmd ./cmd
 COPY pkg ./pkg
 COPY api/ api/
 
+ARG TARGETOS=linux
+ARG TARGETARCH=amd64
 ARG VERSION=undefined
 
-RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
+RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH \
     go build \
     -ldflags "-X main.Version=$VERSION" \
     -a \
     -o varnish-controller \
     ./cmd/varnish-controller/
 
 
-FROM debian:bullseye-slim as binary
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim as binary
 
 RUN apt-get update && apt-get install -y --no-install-recommends varnish
 
 
 # Build Final Varnish image
-FROM debian:bullseye-slim
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim
 LABEL maintainer="Alex Lytvynenko <oleksandr.lytvynenko@ibm.com>, Tomash Sidei <tomash.sidei@ibm.com>"
 
 RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libc6 libedit2 libncursesw6 libtinfo6 libvarnishapi2 \

diff --git a/Dockerfile.exporter b/Dockerfile.exporter
@@ -1,20 +1,23 @@
-FROM debian:bullseye-slim as importer
+ARG BUILDPLATFORM=linux/amd64
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim as importer
+
 ARG PROMETHEUS_VARNISH_EXPORTER_VERSION=1.6.1
+
 RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends varnish curl tar gzip ca-certificates \
 && curl -L https://github.com/jonnenauha/prometheus_varnish_exporter/releases/download/${PROMETHEUS_VARNISH_EXPORTER_VERSION}/prometheus_varnish_exporter-${PROMETHEUS_VARNISH_EXPORTER_VERSION}.linux-amd64.tar.gz | \
     tar -xz  && cp prometheus_varnish_exporter-${PROMETHEUS_VARNISH_EXPORTER_VERSION}.linux-amd64/prometheus_varnish_exporter /prometheus-varnish-exporter \
     && chmod +x /prometheus-varnish-exporter
 
-FROM debian:bullseye-slim
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim
 LABEL maintainer="Alex Lytvynenko <oleksandr.lytvynenko@ibm.com>, Tomash Sidei <tomash.sidei@ibm.com>"
 RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libc6 libedit2 libncursesw6 libtinfo6 libvarnishapi2 \
     && adduser --quiet --system --no-create-home --home /nonexistent --group varnish \
     && mkdir -p /etc/varnish /var/lib/varnish \
     && chown -R varnish /etc/varnish /var/lib/varnish \
     && rm -rf /var/lib/apt/lists/*
 
-COPY  --from=importer /prometheus-varnish-exporter /usr/bin/
-COPY  --from=importer /usr/bin/varnishadm /usr/bin/varnishstat /usr/bin/
+COPY --from=importer /prometheus-varnish-exporter /usr/bin/
+COPY --from=importer /usr/bin/varnishadm /usr/bin/varnishstat /usr/bin/
 
 USER varnish
 

diff --git a/Dockerfile.varnishd b/Dockerfile.varnishd
@@ -1,4 +1,5 @@
-FROM debian:bullseye-slim
+ARG BUILDPLATFORM=linux/amd64
+FROM --platform=$BUILDPLATFORM debian:bullseye-slim
 LABEL maintainer="Alex Lytvynenko <oleksandr.lytvynenko@ibm.com>, Tomash Sidei <tomash.sidei@ibm.com>"
 
 RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \

diff --git a/Makefile b/Makefile
@@ -11,6 +11,7 @@ VARNISH_METRICS_PUBLISH_IMG ?= varnish-metrics-exporter:${VERSION}
 VARNISH_METRICS_IMG ?= ${VARNISH_METRICS_PUBLISH_IMG}-dev
 NAMESPACE ?= "default"
 CRD_OPTIONS ?= "crd:crdVersions=v1"
+PLATFORM ?= "linux/amd64"
 
 # all: test varnish-operator
 all: test varnish-operator varnish-controller
@@ -65,7 +66,7 @@ helm-upgrade: manifests
 # Build the docker image
 # docker-build: test
 docker-build: test
-	docker build ${ROOT_DIR} -t ${IMG} -f Dockerfile
+	docker build --platform ${PLATFORM} ${ROOT_DIR} -t ${IMG} -f Dockerfile
 
 # Tag and push the docker image
 docker-tag-push:
@@ -85,7 +86,7 @@ varnish-controller: fmt vet
 
 # Build the docker image with varnishd itself and varnish modules
 docker-build-varnish:
-	docker build ${ROOT_DIR} -t ${VARNISH_IMG} -f Dockerfile.varnishd
+	docker build --platform ${PLATFORM} ${ROOT_DIR} -t ${VARNISH_IMG} -f Dockerfile.varnishd
 
 docker-tag-push-varnish:
 ifndef REPO_PATH
@@ -101,7 +102,7 @@ endif
 
 # Build the docker image with varnish controller
 docker-build-varnish-controller: fmt vet
-	docker build ${ROOT_DIR} -t ${VARNISH_CONTROLLER_IMG} -f Dockerfile.controller
+	docker build --platform ${PLATFORM} ${ROOT_DIR} -t ${VARNISH_CONTROLLER_IMG} -f Dockerfile.controller
 
 docker-tag-push-varnish-controller:
 ifndef REPO_PATH
@@ -117,7 +118,7 @@ endif
 
 # Build the docker image with varnish metrics exporter
 docker-build-varnish-exporter:
-	docker build ${ROOT_DIR} -t ${VARNISH_METRICS_IMG} -f Dockerfile.exporter
+	docker build --platform ${PLATFORM} ${ROOT_DIR} -t ${VARNISH_METRICS_IMG} -f Dockerfile.exporter
 
 docker-tag-push-varnish-exporter:
 ifndef REPO_PATH

diff --git a/api/v1alpha1/zz_generated.deepcopy.go b/api/v1alpha1/zz_generated.deepcopy.go
diff --git a/docs/quick-start.md b/docs/quick-start.md
@@ -48,24 +48,22 @@ varnish-operator-fd96f48f-gn6mc   1/1     Running             0          40s
 
     ```bash
     $ cat <<EOF | kubectl create -f -
-    apiVersion: caching.ibm.com/v1alpha1
-    kind: VarnishCluster
-    metadata:
-      name: varnishcluster-example
-      namespace: varnish-cluster # the namespace we've created above
-    spec:
-      vcl:
-        configMapName: vcl-config # name of the config map that will store your VCL files. Will be created if doesn't exist.
-        entrypointFileName: entrypoint.vcl # main file used by Varnish to compile the VCL code.
-      backend:
-        port: 80
-        selector:
-          app: nginx-backend # labels that identify your backend pods
-      service:
-        port: 80 # Varnish pods will be exposed on that port
-    EOF
- 
-    varnishcluster.ibm.com/varnishcluster-example created  
+   apiVersion: caching.ibm.com/v1alpha1
+   kind: VarnishCluster
+   metadata:
+     name: varnishcluster-example
+     namespace: varnish-cluster # the namespace we've created above
+   spec:
+     vcl:
+       configMapName: vcl-config # name of the config map that will store your VCL files. Will be created if doesn't exist.
+       entrypointFileName: entrypoint.vcl # main file used by Varnish to compile the VCL code.
+     backend:
+       port: 80
+       selector:
+         app: nginx-backend # labels that identify your backend pods
+     service:
+       port: 80 # Varnish pods will be exposed on that port
+   EOF
     ```
 
 Step 2 and 3 can be executed in any order. If the backend doesn't exist on `VarnishCluster` creation, Varnish will still work but won't cache any backends responding with 504 response code. Once the backend pods are up and running, Varnish will automatically pick up the changes, reload the VCL and start serving and caching the requests.
@@ -85,7 +83,12 @@ You can check if all works by doing `kubectl port-forward` and checking the serv
 Port forward your service:
 
 ```bash
-$ kubectl port-forward -n varnish-cluster service/varnishcluster-example 8080:6081
+$ kubectl port-forward -n varnish-cluster services/varnishcluster-example 8080:80
+Forwarding from 127.0.0.1:8080 -> 6081
+Forwarding from [::1]:8080 -> 6081
+...
+$ # OR
+$ kubectl port-forward -n varnish-cluster services/varnishcluster-example-headless-service 8080:6081
 Forwarding from 127.0.0.1:8080 -> 6081
 Forwarding from [::1]:8080 -> 6081
 ...