Skip to content

Latest commit

 

History

History
242 lines (185 loc) · 7.64 KB

packer.adoc

File metadata and controls

242 lines (185 loc) · 7.64 KB
Raw

Custom images using Packer

Table of Contents

This document describes how we use Packer to create custom images for our deployments. It allows us to speed up deployment by a significant factor.

Currently this process is tested only against AWS cloud provider. But packer supporting a plethora of builders, there is no reason to stop there.

What happens during deployment ?

During deployment, before the infrastructure step, the AMI, if it exists, is detected and picked before the cloudformation stack is created.

There is 2 detection strategies:

  1. Common images using the image name

    Variable: custom_image_filter

    Example:

    custom_image_filter: RHEL 7.5 GOLD packer 1542702393

  2. Opinionated image using (cloud) tags and the tuple (env_type, version, stage)

    Variable: custom_image_stage

    (env_type and version are always provided when deploying)

    This allows us to enable or disable an AMI easily by adding or not the stage into its tag stages in ec2. env_type and version have to match the values in the tags of the image too.

    Example:

    custom_image_stage: PROD

If no AMI is found, then it defaults to what is used in the Cloudformation template.

Requirements to use custom images

When deploying a config:

  1. Make sure either:

    • the config is using the common CloudFormation template

    • the CF template in the config directory supports the variable custom_image

  2. Run the playbook with -e custom_image_filter=IMAGENAME. For example:

    -e "custom_image_filter='RHEL 7.5 GOLD packer 1542702393'"

    Or if you are using config-specific images, set the custom_image_stage variable:

    -e custom_image_stage=PROD

If the image is found, then it will be used, otherwise, playbook defaults to what is defined in the CloudFormation template.

Build common images

Build to your account

Create and copy the image to multiple regions:

packer build -var-file=${HOME}/secrets/gpte.json rhel_gold.json

This is pretty straightforward. See the file rhel_gold.json.

The packer command will:

  1. Create an instance using a source AMI.

  2. Shutdown and save the AMI

  3. Push the AMI to the regions specified in the rhel_gold.json

Build and use opinionated images

We described the process for creating a common image that can be used for any config. In this section we describe the process for creating a custom image specific to a config.

The directory to build packer opiniated images is private because it contains private information like the repository path. For the GPTE organisation, it lives in the private OPEN_Admin repo here: OPENTLC-Deployer/packer/. Inside it, there is a directory for each config.

Inside a config directory, you will find:

  • packer.json: It’s the file used by packer to build the image. It gathers all the information needed for the build.

  • variables.yml: If you have a look inside packer.json you will see that it’s calling ansible-playbook with --extra-vars @variables.yml. The file contains the variables needed for the deployment to work: cloud_provider, own_repo_path, etc.

  • readme.adoc: describe how to build an image for the config.

Usually to create the image for a specific version, you run:

# Change the variables for your need
packer build -var-file=/home/user/secrets/gpte.json
                 -var "variables=variables.yml"
                 -var "version=3.10.14"
                 -var "stages=TEST"
                 packer.json

The packer command will:

  1. Create an instance using a source AMI.

  2. Run the main.yml playbook there using the variables you passed in variables.yml Only the tasks tagged packer will be executed.

  3. Shutdown and save the AMI

  4. Push the AMI to the regions specified in the packer.json or using -var regions=…​

Let’s describe the 3 files used in this command:

  • packer.json

  • gpte.json secret file

  • variables.yml config variables

packer.json

packer.json is the main piece. Here is an example:

{
    "variables": {
        "ami_regions": "us-east-1,eu-central-1,ap-southeast-1,sa-east-1",
        "env_type": "ocp-clientvm"
    },
    "builders": [
        {
            "type": "amazon-ebs",
            "region": "us-east-1",
            "source_ami": "ami-0456c465f72bd0c95",
            "subnet_id": "subnet-0110de3d886fb926e",
            "associate_public_ip_address": "true",
            "instance_type": "t2.large",
            "ssh_username": "ec2-user",
            "access_key": "{{user `aws_access_key_id`}}",
            "secret_key": "{{user `aws_secret_access_key`}}",
            "ami_regions": "{{user `ami_regions`}}",
            "ami_name": "RHEL 7.5 {{user `env_type`}} {{user `osrelease`}} packer {{timestamp}}",
            "tags": {
                "env_type": "{{user `env_type`}}",
                "version": "{{user `version`}}",
                "stages": "{{user `stages`}}",
                "skip_packer_tasks": "yes",
                "hosts": "all"
            }
        }
    ],

    "provisioners": [
        {
            "type": "ansible",
            "playbook_file": "main.yml",
            "groups": ["bastions"],
            "user": "ec2-user",
            "extra_arguments": [
                "--extra-vars", "osrelease={{user `version`}}",
                "--extra-vars", "env_type={{user `env_type`}}",
                "--extra-vars", "@{{user `variables`}}",
                "--tags", "step0000,packer"
            ],
            "ansible_env_vars": ["ANSIBLE_HOST_KEY_CHECKING=False"]
        }
    ]
}

Secret file

In the previous command, gpte.json is the secret file. It contains variables like:

{
        "aws_access_key_id": "...",
        "aws_secret_access_key": "..."
}

Config variables

This file (variables.yml in this example) contains the variables needed to run the config and create the image. Here is an example:

---
cloud_provider: ec2
software_to_deploy: none
own_repo_path: http://example.com/repos/ocp/{{ osrelease }}
install_ipa_client: true

During deployment

If you want the image to be used, you need to provide the variable custom_image_stage. For example:

custom_image_stage: TEST

During deployment, the image, if it exists, will automatically be picked by AgnosticD, unless you set allow_custom_images to false.

AgnosticD will pick an image if those tags on AMI tags match some variables:

  • env_type variable ⇐⇒ env_type tag value

  • osrelease variable ⇐⇒ version tag value

  • custom_image_stage variable included in stages tag value

If a custom image is detected and used during deployment, then the tasks tagged packer will be skipped. This is how we save time!

Skip packer tasks, or not.

By default, when building the image, all tasks tagged packer will be executed.

Then, when you deploy, if a custom image is detected for you, all the tasks tagged packer will be skipped. If you want to change this and still want to run those tasks again (even if they were already done in the image), you can update the cloud Tag skip_packer_tasks on the AMI.

How to tag tasks to be part of the image

When building an image, if you want to add more tasks to it, just tag the task with packer and add a condition so it’s not run during the deployment:

- name: My heavy task that is done in the image and not during deployment
  tags: packer
  when: not hostvars.localhost.skip_packer_tasks | d(false)
  [...]