In this test we went from application version ABC
to XYZ
in the stack:
ScalingAsgRollingUpdate
(CFN stackplayground-CODE-scaling-asg-rolling-update
).
In response to a previous version of this test, we also injected the current
desired capacity as the value for the
MinInstancesInService
property before starting the rolling update.
The main aim of this test was to establish whether setting MinInstancesInService
dynamically would allow us to avoid
the undesirable mid-deployment under-provisioning that we noticed when this value was hardcoded.
Setting the MinInstancesInService
property dynamically allows us to deploy relatively quickly, without increasing the
risk of mid-deployment performance problems.
-
Build number 116 was deployed to start from a known state, running artifact
ABC
The ASG starts with a capacity of:
Capacity Value Min 3 Desired 3 Max 9 -
The service was manually scaled up by invoking our
scale-out
script 3 times.The ASG capacities are now:
Capacity Value Min 3 Desired 6 Max 9 There are 6 instances capable of serving requests (all running
ABC
). -
Build number 117 was deployed. This updates to use artifact
XYZ
and setsMinInstancesInService
to 6. -
The CFN stack
playground-CODE-scaling-asg-rolling-update
begins to update the ASG:Temporarily setting autoscaling group MinSize and DesiredCapacity to 9.
Consequently, the ASG now has a capacity of:
Capacity Value Min 9 Desired 9 Max 9 Due to the dynamically set
MinInstancesInService
property, AWS now avoids under-provisioning the service during the deployment:Rolling update initiated. Terminating 6 obsolete instance(s) in batches of 3, while keeping at least 6 instance(s) in service.
-
Once three
SUCCESS
signals are received, 3 instances are terminated and 3 new ones are launched.Terminating instance(s) [i-023fbdd4382a9dcd1,i-0005ea168f117bd60,i-08f8bf3b8d8dacdbb]; replacing with 3 new instance(s).
The ASG still has a capacity of:
Capacity Value Min 9 Desired 9 Max 9 There are 6 instances capable of serving requests (3 running
XYZ
and 3 runningABC
). -
The 3 new instances (launched in previous step) send a
SUCCESS
signal and start serving traffic.The ASG still has a capacity of:
Capacity Value Min 9 Desired 9 Max 9 There are (briefly) 9 instances capable of serving requests (6 running
XYZ
and 3 runningABC
). -
It is now safe for AWS to terminate the remaining instances running ABC, so it does so:
Terminating instance(s) [i-0b455277568a554b5,i-0e92472e453425cb1,i-0dfe47c2a8e53b0b7]; replacing with 0 new instance(s).
The update completes once these instances are terminated.
-
The ASG capacities are updated, so we end with the capacities that we started with:
Capacity Value Min 3 Desired 6 Max 9 All 6 instances serving traffic are now running
XYZ
.
Full details can be seen via the dashboard.