Automated builds

Nightly Builds via AWS EC2

We combine a few services from AWS to spin up a "build server" once a day; EC2, Auto Scaling, and Simple Notification Service. This setup was heavily inspired by this and this.

First, we start with a basic EC2 instance and install and configure a few things like Git and the Github credentials for the "Blue Button Builder Bot" user. We then create an AMI from that instance.

Next we have the script that we load into the Autoscaling configuration, which the EC2 instance will run every time it starts up. It looks like this:

#!/bin/bash -x
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

EMAIL=bluebutton@example.com

su ubuntu -l
export HOME=/home/ubuntu/
cd $HOME/connector
npm_install=$(npm install)
build_result=$(source build-and-deploy.sh) # Run the script

# Get some information about the running instance
instance_id=$(wget -qO- instance-data/latest/meta-data/instance-id)
public_ip=$(wget -qO- instance-data/latest/meta-data/public-ipv4)
zone=$(wget -qO- instance-data/latest/meta-data/placement/availability-zone)
region=$(expr match $zone '\(.*\).')
uptime=$(uptime)

# Send status email
/usr/sbin/sendmail -oi -t -f $EMAIL <<EOM
From: $EMAIL
To: $EMAIL
Subject: Results of scheduled Blue Button Build  script

This email message was generated on the following EC2 instance:

  instance id: $instance_id
  region:      $region
  public ip:   $public_ip
  uptime:      $uptime
  git config:  $gitconfig


  ec2-describe-instances --region $region $instance_id

<p><pre><code>$build_result</code></pre></p>

EOM

# Give the script and email some time to do their thing
sleep 600 # 10 minutes

# This will stop the EBS boot instance, stopping the hourly charges.
# Have Auto Scaling terminate it, stopping the storage charges.
shutdown -h now

exit 0

Save that file (e.g. ec2-scheduled-build.sh) somewhere it can be referenced in the next step, which is to set up the Autoscaling with these five commands:

as-create-launch-config --instance-type t2.micro --user-data-file ec2-scheduled-build.sh --image-id ami-******** --group sg-******** --launch-config "blue-button-builder-bot-config"

as-create-auto-scaling-group --auto-scaling-group "blue-button-builder-bot-scale-group" --launch-configuration "blue-button-builder-bot-config" --availability-zones "$EC2_ZONE" --min-size 0 --max-size 0

as-suspend-processes "blue-button-builder-bot-scale-group" --processes ReplaceUnhealthy

as-put-scheduled-update-group-action --name "blue-button-builder-bot-start" --auto-scaling-group "blue-button-builder-bot-scale-group" --min-size 1 --max-size 1 --recurrence "30 11 * * *"

as-put-scheduled-update-group-action --name "blue-button-builder-bot-stop" --auto-scaling-group "blue-button-builder-bot-scale-group" --min-size 0 --max-size 0 --recurrence "45 11 * * *"

as-put-notification-configuration --auto-scaling-group "blue-button-builder-bot-scale-group" --topic-arn "arn:aws:sns:us-east-1:*********:blue-button-build" --notification-types "autoscaling:EC2_INSTANCE_LAUNCH_ERROR" "autoscaling:EC2_INSTANCE_TERMINATE_ERROR"

If things go awry and you need to remove all those, here are the commands:

as-delete-scheduled-action --force --name "blue-button-builder-bot-start" --auto-scaling-group "blue-button-builder-bot-scale-group"

as-delete-scheduled-action --force --name "blue-button-builder-bot-stop" --auto-scaling-group "blue-button-builder-bot-scale-group"

as-update-auto-scaling-group "blue-button-builder-bot-scale-group" --min-size 0 --max-size 0

as-delete-auto-scaling-group --force-delete --auto-scaling-group "blue-button-builder-bot-scale-group"

as-delete-launch-config --force --launch-config "blue-button-builder-bot-config"