huge refactor for the new datahub build/deploy CI/CD github actions workflow #117

shaneknapp · 2024-06-27T19:38:14Z

removed ability and code for building images (for now) 2) add support for image tagging (defined in hubploy.yaml, defaults to 'latest' if not specified 3) added verbose and debuging output
added --dry-run for helm actions
fixed almost every line > 80 chars
consistent use of " vs a mix of " and '
did a ton of readability cleanup
added support for deploying more than one image to a hub
maybe more?

i've tested this extensively on my own device and everything work perfectly!

shaneknapp · 2024-06-27T20:56:36Z

some examples below...

help output:

(dh) ➜  datahub git:(staging) ✗ hubploy --help
usage: hubploy [-h] [-d] [-D] [-v] {deploy} ...

positional arguments:
  {deploy}
    deploy          Deploy a chart to the given environment.

options:
  -h, --help        show this help message and exit
  -d, --debug       Enable tool debug output (not including helm debug).
  -D, --helm-debug  Enable Helm debug output.
  -v, --verbose     Enable verbose output.

(dh) ➜  datahub git:(staging) ✗ hubploy deploy --help
usage: hubploy deploy [-h] [--namespace NAMESPACE] [--set SET] [--set-string SET_STRING] [--version VERSION] [--timeout TIMEOUT] [--force] [--atomic]
                      [--cleanup-on-fail] [--dry-run]
                      deployment chart {develop,staging,prod}

positional arguments:
  deployment
  chart
  {develop,staging,prod}

options:
  -h, --help            show this help message and exit
  --namespace NAMESPACE
  --set SET
  --set-string SET_STRING
  --version VERSION
  --timeout TIMEOUT
  --force
  --atomic
  --cleanup-on-fail
  --dry-run             Dry run the helm upgrade command. This also renders the chart to STDOUT.

deploy a hub with one image/profile to staging:

(dh) ➜  datahub git:(staging) ✗ hubploy deploy logodev hub staging
Activated service account credentials for: [hubploy@ucb-datahub-2018.iam.gserviceaccount.com]
Fetching cluster endpoint and auth data.
kubeconfig entry generated for spring-2024.
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://jupyterhub.github.io/helm-chart/" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading jupyterhub from repo https://jupyterhub.github.io/helm-chart/
Deleting outdated charts
Release "logodev-staging" has been upgraded. Happy Helming!
NAME: logodev-staging
LAST DEPLOYED: Thu Jun 27 13:25:59 2024
NAMESPACE: logodev-staging
STATUS: deployed
REVISION: 72
TEST SUITE: None

verbose output:

(dh) ➜  datahub git:(staging) ✗ hubploy -v deploy logodev hub staging
INFO:hubploy.__main__:Namespace(command='deploy', debug=False, helm_debug=False, verbose=True, deployment='logodev', chart='hub', environment='staging', namespace=None, set=None, set_string=None, version=None, timeout=None, force=False, atomic=False, cleanup_on_fail=False, dry_run=False)
INFO:hubploy.helm:Deploying logodev to staging
INFO:hubploy.helm:Getting image and deployment config for logodev
INFO:hubploy.helm:Using image us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image:latest for jupyterhub.singleuser.image
INFO:hubploy.auth:Getting auth config for logodev
INFO:hubploy.auth:Attempting to authenticate with gcloud...
Activated service account credentials for: [hubploy@ucb-datahub-2018.iam.gserviceaccount.com]
Fetching cluster endpoint and auth data.
kubeconfig entry generated for spring-2024.
INFO:hubploy.helm:Deploying logodev-staging in namespace logodev-staging
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://jupyterhub.github.io/helm-chart/" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading jupyterhub from repo https://jupyterhub.github.io/helm-chart/
Deleting outdated charts
Release "logodev-staging" has been upgraded. Happy Helming!
NAME: logodev-staging
LAST DEPLOYED: Thu Jun 27 13:28:58 2024
NAMESPACE: logodev-staging
STATUS: deployed
REVISION: 73
TEST SUITE: None

debug output:

(dh) ➜  datahub git:(staging) ✗ hubploy -d deploy logodev hub staging
INFO:hubploy.__main__:Namespace(command='deploy', debug=True, helm_debug=False, verbose=False, deployment='logodev', chart='hub', environment='staging', namespace=None, set=None, set_string=None, version=None, timeout=None, force=False, atomic=False, cleanup_on_fail=False, dry_run=False)
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'image_name': 'us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image', 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}, 'images': [<hubploy.config.RemoteImage object at 0x7fe2fc3f7730>]}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
INFO:hubploy.helm:Deploying logodev to staging
INFO:hubploy.helm:Getting image and deployment config for logodev
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'image_name': 'us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image', 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}, 'images': [<hubploy.config.RemoteImage object at 0x7fe2eb2d20e0>]}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
DEBUG:hubploy.helm:Using helm config files: ['deployments/logodev/config/common.yaml', 'deployments/logodev/config/staging.yaml']
DEBUG:hubploy.helm:Using helm secret files: ['deployments/logodev/secrets/staging.yaml']
INFO:hubploy.helm:Using image us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image:latest for jupyterhub.singleuser.image
DEBUG:hubploy.helm:Activating cluster credentials for deployment logodev
INFO:hubploy.auth:Getting auth config for logodev
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'image_name': 'us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image', 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}, 'images': [<hubploy.config.RemoteImage object at 0x7fe2fc550f40>]}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
INFO:hubploy.auth:Attempting to authenticate with gcloud...
Activated service account credentials for: [hubploy@ucb-datahub-2018.iam.gserviceaccount.com]
Fetching cluster endpoint and auth data.
kubeconfig entry generated for spring-2024.
INFO:hubploy.helm:Deploying logodev-staging in namespace logodev-staging
DEBUG:hubploy.helm:Running helm dep up in subdirectory 'hub'
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://jupyterhub.github.io/helm-chart/" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading jupyterhub from repo https://jupyterhub.github.io/helm-chart/
Deleting outdated charts
DEBUG:hubploy.helm:Loading kubeconfig for k8s access
DEBUG:hubploy.helm:Checking for namespace logodev-staging and creating if it doesn't exist
DEBUG:hubploy.helm:Running helm upgrade with command: ['helm', 'upgrade', '--wait', '--install', '--namespace', 'logodev-staging', 'logodev-staging', 'hub', '-f', 'deployments/logodev/config/common.yaml', '-f', 'deployments/logodev/config/staging.yaml', '-f', '/tmp/tmpz26wyhz1', '--set-string', 'jupyterhub.singleuser.image.tag=latest', '--set-string', 'jupyterhub.singleuser.image.name=us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/logodev-user-image']
Release "logodev-staging" has been upgraded. Happy Helming!
NAME: logodev-staging
LAST DEPLOYED: Thu Jun 27 13:29:43 2024
NAMESPACE: logodev-staging
STATUS: deployed
REVISION: 74
TEST SUITE: None

using the hashes from artifact repo, deploying multiple images to the dev hub (debug output to see what it does):

(dh) ➜  datahub git:(staging) ✗ hubploy -d deploy dev hub staging
INFO:hubploy.__main__:Namespace(command='deploy', debug=True, helm_debug=False, verbose=False, deployment='dev', chart='hub', environment='staging', namespace=None, set=None, set_string=None, version=None, timeout=None, force=False, atomic=False, cleanup_on_fail=False, dry_run=False)
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'images': [<hubploy.config.RemoteImage object at 0x7f6821a7ae60>, <hubploy.config.RemoteImage object at 0x7f6821a7ae90>], 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
INFO:hubploy.helm:Deploying dev to staging
INFO:hubploy.helm:Getting image and deployment config for dev
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'images': [<hubploy.config.RemoteImage object at 0x7f68216be2f0>, <hubploy.config.RemoteImage object at 0x7f68216be2c0>], 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
DEBUG:hubploy.helm:Using helm config files: ['deployments/dev/config/common.yaml', 'deployments/dev/config/staging.yaml']
DEBUG:hubploy.helm:Using helm secret files: ['deployments/dev/secrets/staging.yaml']
INFO:hubploy.helm:Using image us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-user-image:0e5f0ac for jupyterhub.singleuser.image
INFO:hubploy.helm:Using image us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-secondary:aa0abf0 for jupyterhub.singleuser.image
DEBUG:hubploy.helm:Activating cluster credentials for deployment dev
INFO:hubploy.auth:Getting auth config for dev
DEBUG:hubploy.config:Config loaded and parsed: {'images': {'images': [<hubploy.config.RemoteImage object at 0x7f68327ef8b0>, <hubploy.config.RemoteImage object at 0x7f68216be920>], 'registry': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gcr-key.json'}}}, 'cluster': {'provider': 'gcloud', 'gcloud': {'project': 'ucb-datahub-2018', 'service_key': 'gke-key.json', 'cluster': 'spring-2024', 'zone': 'us-central1'}}}
INFO:hubploy.auth:Attempting to authenticate with gcloud...
Activated service account credentials for: [hubploy@ucb-datahub-2018.iam.gserviceaccount.com]
Fetching cluster endpoint and auth data.
kubeconfig entry generated for spring-2024.
INFO:hubploy.helm:Deploying dev-staging in namespace dev-staging
DEBUG:hubploy.helm:Running helm dep up in subdirectory 'hub'
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://jupyterhub.github.io/helm-chart/" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading jupyterhub from repo https://jupyterhub.github.io/helm-chart/
Deleting outdated charts
DEBUG:hubploy.helm:Loading kubeconfig for k8s access
DEBUG:hubploy.helm:Checking for namespace dev-staging and creating if it doesn't exist
DEBUG:hubploy.helm:Running helm upgrade with command: ['helm', 'upgrade', '--wait', '--install', '--namespace', 'dev-staging', 'dev-staging', 'hub', '-f', 'deployments/dev/config/common.yaml', '-f', 'deployments/dev/config/staging.yaml', '-f', '/tmp/tmpvel19ulj', '--set-string', 'jupyterhub.singleuser.image.tag=0e5f0ac', '--set-string', 'jupyterhub.singleuser.image.name=us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-user-image', '--set-string', 'jupyterhub.singleuser.image.tag=aa0abf0', '--set-string', 'jupyterhub.singleuser.image.name=us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-secondary']
Release "dev-staging" has been upgraded. Happy Helming!
NAME: dev-staging
LAST DEPLOYED: Thu Jun 27 13:54:21 2024
NAMESPACE: dev-staging
STATUS: deployed
REVISION: 54
TEST SUITE: None

the hacked config for the dev hub:

images:
  images:
    - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-user-image:0e5f0ac
    - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/dev-secondary:aa0abf0
  registry:
    provider: gcloud
    gcloud:
      project: ucb-datahub-2018
      service_key: gcr-key.json

cluster:
  provider: gcloud
  gcloud:
    project: ucb-datahub-2018
    service_key: gke-key.json
    cluster: spring-2024
    zone: us-central1

1) removed ability and code for building images (for now) 2) add support for image tagging (defined in hubploy.yaml, defaults to 'latest' if not specified 3) added verbose and debuging output 4) added --dry-run for helm actions 5) fixed almost every line > 80 chars 6) consistent use of " vs a mix of " and ' 7) did a ton of readability cleanup 8) maybe more?

shaneknapp · 2024-06-27T21:33:32Z

i've also successfully built and pushed the logodev image to artifact repository and successfully deployed said new image (w/the latest tag) to logodev-staging!

hubploy/helm.py

shaneknapp · 2024-07-01T17:29:46Z

added CLI help text for all the things:

(dh) ➜  datahub git:(staging) ✗ hubploy deploy --help
usage: hubploy deploy [-h] [--namespace NAMESPACE] [--set SET] [--set-string SET_STRING] [--version VERSION] [--timeout TIMEOUT] [--force] [--atomic]
                      [--cleanup-on-fail] [--dry-run] [--image-overrides IMAGE_OVERRIDES [IMAGE_OVERRIDES ...]]
                      deployment chart {develop,staging,prod}

positional arguments:
  deployment            The name of the hub to deploy.
  chart                 The path to the main hub chart.
  {develop,staging,prod}
                        The environment to deploy to.

options:
  -h, --help            show this help message and exit
  --namespace NAMESPACE
                        Helm option: the namespace to deploy to. If not specified, the namespace will be derived from the environment
  --set SET             Helm option: set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
  --set-string SET_STRING
                        Helm option: set STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
  --version VERSION     Helm option: specify a version constraint for the chart version to use. This constraint can be a specific tag (e.g. 1.1.1) or it may reference a
                        valid range (e.g. ^2.0.0). If this is not specified, the latest version is used.
  --timeout TIMEOUT     Helm option: time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, etc). Defaults to 300 seconds.
  --force
  --atomic              Helm option: if set, upgrade process rolls back changes made in case of failed upgrade. The --wait flag will be set automatically if --atomic is
                        used.
  --cleanup-on-fail     Helm option: allow deletion of new resources created in this upgrade when upgrade fails.
  --dry-run             Dry run the helm upgrade command. This also renders the chart to STDOUT.
  --image-overrides IMAGE_OVERRIDES [IMAGE_OVERRIDES ...]
                        Override one or more images and tags to deploy. Format is: <path_to_image1/image_name>:<tag1> <path_to_image2/image_name>:<tag2> ... IMPORTANT:
                        The order of images passed in must match the order in which they appear in hubploy.yaml and separated by spaces without quotes. You must always
                        specify a tag when overriding images.

shaneknapp mentioned this pull request Jun 27, 2024

since we're not testing just make the tests pass #118

Merged

shaneknapp added 4 commits June 27, 2024 14:21

removing unused imports

e0efb87

disable some tests as we arent building or pushing anymore

8a92d85

update build python

88be478

shaneknapp force-pushed the major-refactor branch from 9623735 to 88be478 Compare June 27, 2024 21:24

add a little more messaging about tagging

7d5491e

shaneknapp added 2 commits June 27, 2024 14:48

wording etc

2ea0e83

add ability to override image and tag via the command line

758c6ed

shaneknapp commented Jun 28, 2024

View reviewed changes

hubploy/helm.py Outdated Show resolved Hide resolved

shaneknapp mentioned this pull request Jun 28, 2024

Unit Testing for Hubploy #95

Open

shaneknapp added 9 commits June 28, 2024 15:09

finish up image overrides

acf5209

more error checking

a665f21

belay that, i was already error checking XD

36f4db6

be clear about tags w/overrides

355ecfd

the old deploy method is not needed any more

1541cc1

utils.py is no longer needed

bf0c963

commitrange.py is no longer needed

4e72251

removing unnecessary imports

22cc0d7

every CLI arg now has help text!

8bdaea7

shaneknapp added 7 commits July 1, 2024 10:36

reorder imports

0cd4295

ran through ruff delinter and fixed everything

3b3b5fc

minor formatting of help strings

f0d2bc1

add help text for --force

1c2f459

bump version, and udpate package metadata

cc534cc

remove all tests and circleci workflows

fa7122f

better handling of no image tags

c3e5b43

shaneknapp added 16 commits August 16, 2024 15:15

remove container registry auth code

63b33f9

exit if CI environment is detected with helm debug or dry run specified

c05f093

explain CI limitations in help strings

fca6412

add env var requirement for helm debug and dry-run args

9352e22

add comprehensive debugging/verbose output for auth module

bd9902f

remove this line

3bed611

final bits of debugging/info output

683fc68

fixing typo in help text

19802d4

fixing typo in help text

beb6343

re-add aws auth contextmanager, coz who knows if we'll ever need it

3cdb8f7

fixing newlines at EOF

d267768

formatting from ruff

d510ecf

adding min python version req for hubploy

4489c5a

adding python linter

7c1398b

ruff format

ca5ec61

updated readme

cea3849

shaneknapp changed the title ~~huge refactor in preparation for new build/deploy workflow~~ huge refactor for the new datahub build/deploy CI/CD github actions workflow Oct 10, 2024

adding some whitespace to readme

f76a897

shaneknapp merged commit 2c408be into berkeley-dsep-infra:main Oct 10, 2024
1 check passed

shaneknapp deleted the major-refactor branch October 10, 2024 22:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

huge refactor for the new datahub build/deploy CI/CD github actions workflow #117

huge refactor for the new datahub build/deploy CI/CD github actions workflow #117

shaneknapp commented Jun 27, 2024 •

edited

Loading

shaneknapp commented Jun 27, 2024 •

edited

Loading

shaneknapp commented Jun 27, 2024

shaneknapp commented Jul 1, 2024 •

edited

Loading

huge refactor for the new datahub build/deploy CI/CD github actions workflow #117

huge refactor for the new datahub build/deploy CI/CD github actions workflow #117

Conversation

shaneknapp commented Jun 27, 2024 • edited Loading

shaneknapp commented Jun 27, 2024 • edited Loading

shaneknapp commented Jun 27, 2024

shaneknapp commented Jul 1, 2024 • edited Loading

shaneknapp commented Jun 27, 2024 •

edited

Loading

shaneknapp commented Jun 27, 2024 •

edited

Loading

shaneknapp commented Jul 1, 2024 •

edited

Loading