Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOM-47678] Secret querying fixes #110

Merged
merged 1 commit into from
Jul 7, 2023

Conversation

ddl-ebrown
Copy link
Contributor

@ddl-ebrown ddl-ebrown commented Jul 7, 2023

  • Add secrets list perms

    The secret reading code had a late change to search for matching
    secrets with the correct hephaestus-accessible label instead of
    performing a simple get by namespace/name.

    While this prevents Hephaestus code from even accessing the secret,
    it also now requires the list permission to query secrets.

  • Correctly specify fields for secret name and namespace by specifying
    the metadata. prefix

  • Unit tests don't support the filter syntax and don't apply Kubernetes
    permissioning, and the repo functiona. tests which actually use
    Kubernetes are currently not operational, so this slipped past review

Manual Test Results

As in #106, this requires manual testing given the functional tests in this repo are not working.

Using the follow resource definitions, I created k8s resources and then tailed the logs of Hephaestus to observe the behavior.

Resource definitions

apiVersion: v1
kind: Secret
metadata:
  name: foo
  # TODO: must be the same namespace as Hephaestus for hephaestus-owned to work
  namespace: domino-compute
  labels:
    hephaestus-accessible: "true"
    # NOTE: for hephaestus-owned to work, secret must be in domino-compute
    hephaestus-owned: "true"
type: Opaque
stringData:
  bar: "test of foo.bar"
---
apiVersion: hephaestus.dominodatalab.com/v1
kind: ImageBuild
metadata:
  name: domino-ethan-64a58393bb2e9b737895a66f
  namespace: domino-compute
spec:
  amqpOverrides:
    queueName: hephaestus.imagebuilds.status
  buildArgs:
  - REPOCLONER_MAX_CONNECTIONS=20
  - REPOCLONER_JAVA_CLASSPATH_FILE=/app/domino/repocloner/repocloner_image.classpath
  - REPOCLONER_SUPERUSER_API_KEY=<redacted>
  - REPOCLONER_JAVA_OPTS=-Xss256m -Xmx1g
  - REPOCLONER_CONNECT_TIMEOUT_SEC=120
  - REPOCLONER_JAVA_MAIN_CLASS=domino.repocloner.RepoClonerApp
  - REPOCLONER_SUPERUSER_USERNAME=domino-superuser
  - REPOCLONER_BLOB_API_HOST=http://nucleus-frontend.domino-platform:80
  - REPOCLONER_BLOB_SYNCHRONIZATION_RETRY_COUNT=2
  - REPOCLONER_IMAGE_DEFINITION_URL=http://nucleus-frontend.domino-platform:80/v1/builds/64a58393bb2e9b737895a66f/imageDefinition
  - REPOCLONER_MAX_CONNECTIONS_PER_ROUTE=20
  - REPOCLONER_READ_TIMEOUT_SEC=120
  - REPOCLONER_TARGET_DIR=/home/domino/repocloner
  - REPOCLONER_INTERNAL_GIT_HOST=http://git.domino-platform:80
  - REPOCLONER_ALLOW_PROJECT_SYMLINKS=false
  - REPOCLONER_IMAGE_DEFINITION_API_KEY=<redacted>
  # expects a secret domino-compute/foo with data bar
  context: https://github.com/dominodatalab/hephaestus/raw/51585793939ab0de611bea74ab14e29d846b2070/test/functional/testdata/docker-context/secrets/archive.tgz
  disableBuildCache: true
  disableCacheExport: true
  images:
  - 172.20.88.105:5000/dominodatalab/environment:649f7bbcbb2e9b737895a60d-33
  logKey: f66a-build-64a58393bb2e9b737895a66f
  registryAuth:
  - cloudProvided: false
    secret:
      name: domino-quay-repos
      namespace: domino-compute
    server: quay.io
  - cloudProvided: false
    secret:
      name: domino-registry
      namespace: domino-compute
    server: 172.20.88.105:5000
  secrets:
    - name: foo
      namespace: domino-compute

This continues to use a Dockerfile context available at Github at https://github.com/dominodatalab/hephaestus/raw/51585793939ab0de611bea74ab14e29d846b2070/test/functional/testdata/docker-context/secrets/archive.tgz with a simple Dockerfile like:

FROM alpine:3.16
RUN --mount=type=secret,id=domino-compute/foo/bar cat /run/secrets/bar

Relevant logs

Secret retrieval

2023-07-07T18:11:45.310462133Z    INFO    controller.imagebuild.component.build-dispatcher    Processing references to build secrets    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f"}
2023-07-07T18:11:45.3106744Z    INFO    controller.imagebuild.component.build-dispatcher    Finding secret    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "path": "domino-compute/foo"}
2023-07-07T18:11:45.314168529Z    INFO    controller.imagebuild.component.build-dispatcher    Taking ownership of secret    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "owner": "domino-ethan-64a58393bb2e9b737895a66f", "secret": "domino-compute/foo"}
2023-07-07T18:11:45.329612719Z    INFO    controller.imagebuild.component.build-dispatcher    Read secret bytes    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "path": "domino-compute/foo/bar", "bytes": 15}
2023-07-07T18:11:45.329657617Z    INFO    controller.imagebuild.component.build-dispatcher    Processing and persisting registry credentials    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f"}

Note specifically:

Taking ownership of secret {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "owner": "domino-ethan-64a58393bb2e9b737895a66f", "secret": "domino-compute/foo"}
Read secret bytes {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "path": "domino-compute/foo/bar", "bytes": 15}

Build logs

2023-07-07T18:15:04.344376252Z    INFO    controller.imagebuildmessage    Reconciliation complete    {"imagebuildmessage": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.8438524Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #1 [internal] load .dockerignore
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843907312Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #1 transferring context: 2B done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843919847Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #1 DONE 0.0s
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843925217Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843930678Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #2 [internal] load build definition from Dockerfile
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843937794Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #2 transferring dockerfile: 125B done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843951602Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #2 DONE 0.0s
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843961235Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:04.843966119Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #3 [internal] load metadata for docker.io/library/alpine:3.16
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.596767163Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #3 DONE 0.8s
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.596812446Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.596820703Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #4 [stage-0 1/2] FROM docker.io/library/alpine:3.16@sha256:cbe5d5973103a2d03408d1689a6efde4ea4920bde9f4b51fe7872e60ce2d8e56
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.59682877Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #4 resolve docker.io/library/alpine:3.16@sha256:cbe5d5973103a2d03408d1689a6efde4ea4920bde9f4b51fe7872e60ce2d8e56 0.0s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.596836459Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #4 CACHED
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.596847654Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.59685308Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #5 [stage-0 2/2] RUN --mount=type=secret,id=domino-compute/foo/bar cat /run/secrets/bar
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.711150856Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #5 0.092 test of foo.bar    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.711199607Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.711213678Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #5 DONE 0.1s
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.711242498Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.71124917Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 exporting to image
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.711256056Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 exporting layers 0.1s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.862088963Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 exporting manifest sha256:a1526016f8b64dba49e2c93f3f0050884fcfe199fc13f1bb1fb9a3f47d8ba965 0.0s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.862131614Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 exporting config sha256:581c3650bd3b3a1c614694b9a3a378cdd1ab1c846079fdd2e4a4492e35e7ac08 0.0s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:05.862141246Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 pushing layers
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:07.515149912Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 pushing layers 1.6s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:07.515211836Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 pushing manifest for 172.20.88.105:5000/dominodatalab/environment:649f7bbcbb2e9b737895a60d-33@sha256:a1526016f8b64dba49e2c93f3f0050884fcfe199fc13f1bb1fb9a3f47d8ba965
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:08.24752041Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    Solve complete    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:08.247577865Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 pushing manifest for 172.20.88.105:5000/dominodatalab/environment:649f7bbcbb2e9b737895a60d-33@sha256:a1526016f8b64dba49e2c93f3f0050884fcfe199fc13f1bb1fb9a3f47d8ba965 0.8s done
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:08.247589079Z    INFO    controller.imagebuild.component.build-dispatcher.buildkit    #6 DONE 2.5s
    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "addr": "tcp://hephaestus-buildkit-0.hephaestus-buildkit.domino-compute:1234", "logKey": "f66a-build-64a58393bb2e9b737895a66f"}
2023-07-07T18:15:08.247807942Z    INFO    controller.imagebuild.component.build-dispatcher    Transitioning status    {"imagebuild": "domino-compute/domino-ethan-64a58393bb2e9b737895a66f", "phase": "Succeeded"}

Note specifically

#5 [stage-0 2/2] RUN --mount=type=secret,id=domino-compute/foo/bar cat /run/secrets/bar
#5 0.092 test of foo.bar

foo.bar is the value present in the secret above

ImageBuild verification

The resource reaches the success status

> kubectl get imagebuilds.hephaestus.dominodatalab.com --namespace domino-compute
NAME                                    STATUS      ALLOCATION TIME   BUILD TIME   AGE
domino-ethan-64a58393bb2e9b737895a66f   Succeeded   3m17.675s         4.084s       22m

The secret still exists

> kubectl get secret --all-namespaces --field-selector metadata.name=foo,metadata.namespace=domino-compute
NAMESPACE        NAME   TYPE     DATA   AGE
domino-compute   foo    Opaque   1      22m

Deleting the ImageBuild properly cleans up the secret

❯ kubectl delete  imagebuilds.hephaestus.dominodatalab.com/domino-ethan-64a58393bb2e9b737895a66f --namespace domino-compute
imagebuild.hephaestus.dominodatalab.com "domino-ethan-64a58393bb2e9b737895a66f" deleted

❯ kubectl get secret --all-namespaces --field-selector metadata.name=foo,metadata.namespace=domino-compute
No resources found

Copy link
Contributor

@ddl-yaniv-amar ddl-yaniv-amar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ddl-ebrown ddl-ebrown changed the title [DOM-47678] Add secrets list perms [DOM-47678] Secret querying fixes Jul 7, 2023
@ddl-ebrown ddl-ebrown force-pushed the DOM-47678-add-secret-list-perms branch from 3b30988 to 26604e9 Compare July 7, 2023 17:46
 - Add secrets list perms

   The secret reading code had a late change to search for matching
   secrets with the correct hephaestus-accessible label instead of
   performing a simple get by namespace/name.

   While this prevents Hephaestus code from even accessing the secret,
   it also now requires the list permission to query secrets.

 - Correctly specify fields for secret name and namespace by specifying
   the metadata. prefix

 - Unit tests don't support the filter syntax and don't apply Kubernetes
   permissioning, and the repo functiona. tests which actually use
   Kubernetes are currently not operational, so this slipped past review
@ddl-ebrown ddl-ebrown force-pushed the DOM-47678-add-secret-list-perms branch from 26604e9 to dcff29a Compare July 7, 2023 18:39
@ddl-ebrown ddl-ebrown merged commit 20fdefe into main Jul 7, 2023
7 checks passed
@ddl-ebrown ddl-ebrown deleted the DOM-47678-add-secret-list-perms branch July 7, 2023 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants