The intention here is to create a single Packer + cloud-init configuration set that can be used across cloud providers to configure taskcluster worker instances.
- Debugability: we should be able to run everything locally as well as in cloud providers
- Clarity: it should be clear which steps run on base images and which steps run on derived images
- Portability: the configuration should be generic enough to be run beyond Firefox CI's worker deployment
packer(go get github.com/hashicorp/packer)vagrant(if building a local image)
- If building AWS AMIs you should have:
- AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, environment variables, representing your AWS Access Key and AWS Secret Key, respectively. (see here)
- Run the following command to get temporary credentials:
eval "$(./aws-signin.sh)"
- You need a whole set of IAM privileges, see here
- If building Google Cloud Images you should have done one of:
- run
gcloud auth login --update-adcwhich creates$HOME/.config/gcloud/application_default_credentials.json(needed every 24 hours). - Configured Service Account credentials and have a JSON file whose location is specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable.
- In either case you need
Compute Instance Admin (v1)permissions, if using a service account you'll needService Account User. See here for more information.
- run
Install Poetry (https://python-poetry.org/) if you don't already have it.
# create and activate a poetry virtualenv for this repo
poetry shell
# install package
poetry installpacker plugins install github.com/hashicorp/googlecomputeSee monopacker --help for details.
You will need to know the builder or builders you want to build; builder1 builder2 are used in the example here.
gcloud auth login --update-adc # needed every 24 hours
monopacker build builder1 builder2Note that you can get more logging from packer by setting PACKER_LOG=1.
See TEMPLATING.md for information, another FAQ, and more.
When developing templates, you can run the validation without running packer with monopacker validate (which otherwise has the same arguments as monopacker build):
monopacker validate mynewbuilderTo see the generated packer template:
monopacker packer-template mynewbuildermonopacker build generic_translations_gcp --packer-args '-on-error=ask'
gcloud compute ssh --zone ...
# when done on host, in monopacker choose to 'c' cleanup-on-error=abort can also be handy.
Make sure that you are operating in a Python virtualenv and have installed the package.
# all the debug output
PACKER_LOG=1 VAGRANT_LOG=debug monopacker build my_builder# create a yaml file of the form:
cat << EOF > fake_secrets.yaml
- name: foo
path: /path/to/foo
value: what
- name: bar
path: /path/to/bar
value: yeah
EOF
# for a custom secrets file, pass --secrets_file to monopacker:
monopacker build --secrets_file="/path/to/secrets.yaml" mybuilder
# by default ./fake_secrets.yaml is usedAWS Metal instances take a long time to boot. See here.
Launching a bare metal instance boots the underlying server, which includes verifying all hardware and firmware components. This means that it can take 20 minutes from the time the instance enters the running state until it becomes available over the network.
You can technically do this, but only on an OS that runs Docker natively. macOS runs Docker in a Linux VM under the hood, which means you can't do this easily. Mostly, I just haven't tried to make this work.
To run the tests for this library, run poetry run pytest.
To update dependencies, run poetry cache clear pypi --all && poetry update.