Deployment Patterns

Learning goal

Basic use of match criteria in a virtual service
Use header based routing for blue/green deployment
Use weighted traffic distribution for canary deployment
Mirror traffic for shadow deployment

Introduction

These exercises will introduce you to the match, weight and mirror fields of the HTTPRoute. These fields and their combinations can be used to enable several useful deployment patterns like blue/green deployments, canary deployments and shadow deployments.

These exercises build on the Basic traffic routing exercises.

💡 If you have not completed exercise 00-setup-introduction you need to label your namespace with istio-injection=enabled.

Exercise: Blue/Green Deployment

This exercise is going to introduce you to the HTTPRoute match field in a virtual service. We want to implement a blue/green deployment pattern which allows the client to actively select a version of the name service it will hit.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service-route
spec:
  hosts:
  - my-service
  http:
  - match:
    - headers:
        my-header:
          exact: use-v2
    - route:
        - destination:
            host: my-service
            subset: v2

In the example above we define a HTTPMatchRequest with the field match. The headers field declares that we want to match on request headers. The exact field declares that the header must match exactly use-v2.

Istio allows to use other parts of incoming requests and match them to values you define. To see what these are expand More Info below.

More Info

Match Fields

uri: Matches the request URI to the specified value
schema: Matches the request schema. (HTTP, HTTPS, ...)
method: Matches the request method. (GET, POST, ...)
authority: Matches the request authority header
header: Matches the request headers

NOTE: Headers need to be lower cased and may use hyphens. If headers is used uri, schema, method and authority are ignored.

Match Types

exact: Exactly matches the provided value
prefix: Only the prefix part of the provided value will get matched
regex: The provided value will be matched based on the regex

NOTE: Istio regex's use the re2 regular expression syntax

You can have multiple conditions in a match block and multiple match blocks.

💡 All conditions inside a single match block have AND semantics, while the list of match blocks have OR semantics. The rule is matched if any one of the match blocks succeed.

NOTE: Matches are evaluated prior to any destination's being applied.

Overview

A general overview of what you will be doing in the Step By Step section.

Deploy the sentences application services with kubectl

You will deploy two versions of the name service.

Observe the traffic flow with Kiali
Route traffic to a specific version v2 matching a header specified by the client

Step by Step

Expand the Tasks section below to do the exercise.

Tasks

Task: Deploy the sentences app

kubectl apply -f 02-deployment-patterns/start/
kubectl apply -f 02-deployment-patterns/start/name-v1/
kubectl apply -f 02-deployment-patterns/start/name-v2/

This will deploy two versions of the name service along with a destination rule and virtual service as defined in a previous exercise.

Task: Run the `scripts/loop-query.sh` script

./scripts/loop-query.sh

Task: Observe the traffic flow with Kiali

In Kiali go to graphs, select your namespace and the versioned app graph.

You should see the traffic being routed to the name-v1 workload because of the precedence of the routes in the name virtual service.

Task: Stop the `loop-query.sh`

Use ctrl+ c to stop the loop-query.sh script.

Task: Add a match block with header condition

Update the name-vs.yaml file with an HTTPMatchRequest and apply it.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:
    - headers:
        x-test:
          exact: use-v2
    route:
    - destination:
        host: name
        subset: name-v2
  - route:
    - destination:
        host: name
        subset: name-v1

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Task: Run loop-query.sh with the `x-test` header

./scripts/loop-query.sh -h 'x-test: use-v2'

Task: Observe the traffic flow with Kiali

You should see all traffic being directed to v2 of the name workload. That is because the match evaluated to true and the route under the match block is used.

Task: Run the `scripts/loop-query.sh` without header

Use ctrl+ c to stop the loop-query.sh script and run it without passing the header.

./scripts/loop-query.sh

Task: Observe the traffic flow with Kiali

You should now see all traffic being routed to v1 of the name workload. This is because when the match did not evaluate to true the default route was used.

Whenever you use matches for traffic routing. You should always ensure you have a default route.

💡 In the file name-vs.yaml you created. Notice the indentation for the route to name-v1, which is our default route. E.g the route to name-v2 is nested under the match block. That is a different route...

Task: Remove the default route

In the name-vs.yaml file remove the default route and apply the changes.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:
    - headers:
        x-test:
          exact: use-v2
    route:
    - destination:
        host: name
        subset: name-v2

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Task: Observe the traffic flow with Kiali

Go to Graph menu item and select the Versioned app graph from the drop down menu.

The problem we have here is that the match is evaluated first before any destination's are applied. Since the match was not true the route defined under it was not applied. Nor have we provided another route to fall back on when the match does not evaluate to true.

Task: Add the default route and use a different header

First add the default route in the name-vs.yaml file again and apply it.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:
    - headers:
        x-test:
          exact: use-v2
    route:
    - destination:
        host: name
        subset: name-v2
  - route:
    - destination:
        host: name
        subset: name-v1

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Now try using a different header with the value use-v2. Modify the virtual service to reflect the header you chose and pass the header to the loop-query.sh script.

You can call it anything you want. For example x-version is used below but it really doesn't matter.

./scripts/loop-query.sh -h 'x-version: use-v2'

Task: Observe the traffic flow with Kiali

Go to Graph menu item and select the Versioned app graph from the drop down menu.

In order for a match on a header to work it must be propagated to all the services in the application tree.

As the header is not propagated to the name service the default route is hit.

You would need to modify the sentence service to propagate the header onwards so the virtual service could match on it. If you look in the code for the application, you can see which headers are currently forwarded.

Exercise: Canary Deployment

This exercise is going to introduce you to the HTTPRoute weight field in a virtual service. We want to implement a canary deployment pattern to the name service's v1 and v2 workloads and header based blue/green deployment to a v3 workload.

More Info

The canary deployment pattern is often employed after a blue/green deployment pattern. Blue/green deployments are characterized by an explicit choice by the client/user of which version to use.

A canary deployment removes the need for this explicit choice by weighting the traffic between releases. But it is not an uncommon scenario to have a canary deployment alongside of a blue/green deployment for the next unreleased version.

Canary deployment is a pattern for rolling out releases to a subset of users/clients. The idea is to test and gather feedback from this subset and reduce risk by gradually introducing a new release.

😆 Fun fact. The term canary comes from the coal mining industry where canaries were used to alert miners when toxic gases reached dangerous levels. In the same way canary deployments can alert you to issues, bad design or whether features actually give the intended value.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service-route
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service
        subset: v1
      weight: 90
    - destination:
        host: my-service
        subset: v2
      weight: 10

In the above example we define a traffic distribution percentage with the weight fields on the destinations of the HTTPRoute. The v1 workload of my-service destination will receive 90% of all traffic while the v2 workload of my-service will receive 10% of all traffic.

NB: Before v1.13 of Istio, weight were thought to be percentages, and the sum of the weight fields across the destinations of a single route SHOULD BE == 100.

With Istio v1.13 weights specify relative proportions, and distribution is relative to the sum of weights, which may be different from 100. A destination will receive weight/(sum of all weights) requests.

Overview

A general overview of what you will be doing in the Step By Step section.

Deploy the sentences application services with kubectl

You will deploy three versions of the name service

Observe the traffic flow with Kiali
incorporate version v3 of the name service in the destination rule and the virtual service for header based routing

The use case is that new version v3 will be the new blue/green deployment.

Create a canary deployment for version v1and v2 of the name service with the weight field

The use case is that v2 has been validated and you are doing a canary deployment to reduce risk.

Promote v2 of the name service to receive all traffic.

Step By Step

Expand the Tasks section below to do the exercise.

Tasks

Task: Deploy the sentences app with three version of the `name` service.

kubectl apply -f 02-deployment-patterns/start/
kubectl apply -f 02-deployment-patterns/start/name-v1/
kubectl apply -f 02-deployment-patterns/start/name-v2/
kubectl apply -f 02-deployment-patterns/start/name-v3/

Task: Open file `02-deployment-patterns/start/name-dr.yaml` and add `name-v3`:

  - name: name-v3
    labels:
      version: v3

kubectl apply -f 02-deployment-patterns/start/name-dr.yaml

Task: Adjust the `match` field in `name-vs.yaml` to `use-v3`

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:                # Add from here...
    - headers:            #
        x-test:           #
          exact: use-v3   #
    route:                #
    - destination:        #
        host: name        #
        subset: name-v3   # ...to here
  - route:
    - destination:
        host: name
        subset: name-v1
  - route:
    - destination:
        host: name
        subset: name-v2

Again, notice the indentation. name-v3 will only be hit if the header match is true. The other two routes are evaluated top down.

❗ If you just did the blue/green exercise, make sure your routes to name-v1 and name-v2 are like above.

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Task: Run `scripts/loop-query.sh`

If not already running, execute the loop-query.sh script.

./scripts/loop-query.sh

Task: Observe the traffic flow with Kiali

Go to Graph menu item and select the Versioned app graph from the drop down menu.

The traffic should still be routed to the v1 workload as the match condition did not evaluate to true and order of precedence dictates the first destination which will direct traffic to v1 workload.

Task: Add the `weight` fields in `name-vs.yaml`

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:
    - headers:
        x-test:
          exact: use-v3
    route:
    - destination:
        host: name
        subset: name-v3
  - route:
    - destination:
        host: name
        subset: name-v1
      weight: 90                # Add this, NOTE: 'route' removed
    - destination:
        host: name
        subset: name-v2
      weight: 10                # and this

💡 The weight is distributed by route so the destination for v2 must be under the same route block.

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Task: Observe the traffic flow with Kiali

Go to Graph menu item and select the Versioned app graph from the drop down menu.

You should see that the traffic is distributed approximately 90% to v1 and 10% to v2.

Task: In a new terminal pass the header `use-v3` to `scripts/loop-query.sh`

./scripts/loop-query.sh -h 'x-test: use-v3'

Task: Observe the traffic flow with Kiali

You can see that the traffic distribution is no longer 90/10 between v1 and v2.

We have two clients. One has all traffic routed v3. The others traffic is distributed 90% to v1 and 10% v2.

The Kiali graph is showing you the percentage of the overall traffic. The weight is applied to the traffic that hits the second route.

Run scripts/loop-query.sh without the header in another terminal, or several, and observe how it affects the traffic distribution. You will see the percentage of the overall traffic change to look more like the 90/10 weight.

Task: Promote `v2` to receive all traffic

Stop sending traffic to v3 with ./scripts/loop-query.sh -h 'x-test: use-v3'.

Remove the name-v1 subset from name-dr.yaml file.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: name-destination-rule
spec:
  host: name
  exportTo:
  - "."
  subsets:
  - name: name-v2
    labels:
      version: v2
  - name: name-v3
    labels:
      version: v3

Remove the name-v1 destination from name-vs.yaml file, adjust the weight field to 100% on name-v2 destination and apply changes.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - match:
    - headers:
        x-test:
          exact: use-v3
    route:
    - destination:
        host: name
        subset: name-v3
  - route:
    - destination:
        host: name
        subset: name-v2
      weight: 100

kubectl apply -f 02-deployment-patterns/start/name-dr.yaml
kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

💡 You could simply adjust the weight values in the name-vs.yaml file. But for the next exercise it will cause less confusion if the subsets and destinations are removed.

NB: do note that in the above we're removing a subset from the DestinationRule, and changing the VirtualService, but we're applying the DestinationRule first. This means that theoretically the old VirtualService could be routing to a subset that doesn't exist, and drop traffic. This isn't an issue during our exercises.

But in "real life" when you're removing subsets, you want to update your VirtualServices first, since you don't want to use a subset that's going to disappear. And when you're adding subsets, you want to update the DestinationRules first, because we want the subsets to be there, before we're updating our VirtualServices.

Finally, delete the name-v1 workload.

kubectl delete -f 02-deployment-patterns/start/name-v1/

Task: Observe the traffic flow with Kiali

All traffic will now be routed to the v2 workload. Normally you would adjust the weights gradually to expose v2 to more and more users.

Exercise: Shadow Deployment

This exercise will introduce you to the HTTPRoute mirror field in a virtual service. We want to route traffic to the name service v3 workload while still forwarding traffic to the original destination. This is often called a shadow deployment.

In Istio mirrored traffic is on a best effort basis. This means that the sidecar/gateway will not wait for mirrored traffic before sending a response from the original destination.

The use case here is the following. You have completed a canary deployment for v1 and v2 of the name service. You have also done a blue/green deployment for v3 of the name service. You have tested it's functionality and everything looks good. Now you would like to load test v3 as it has some changes that could affect performance.

More Info

The idea behind shadow deployments is that the clients/users have no idea about the deployed version and the deployed version has zero impact on the clients/users as the mirrored traffic happens out of band for the critical request path for the primary service.

Shadow deployments are particularly useful for load testing and refactoring of monolithic applications. But they can be quite complex and care has to be taken if there are interactions with other systems. Imagine shadow testing a payment service for a shopping cart. You wouldn't want users paying twice for an order.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service-route
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service
        subset: v2
      weight: 100
    - destination:
        host: my-service
        subset: v3
    mirror:
      host: my-service
      subset: v3
    mirrorPercentage:
      value: 100.0

In the example above you can see that the weight field routes 100% of traffic to the v2 subset of my-service. The mirror field will route 100% of the traffic v2 receives to the v3 subset.

Adjusting the mirrorPercentage value will adjust how much of the traffic routed to v2 will be mirrored to the v3 subset. E.g, if set to 50.0 then 50% percent of traffic routed to the v2 subset will be mirrored to the v3 subset. If the field is not defined then the v3 subset will get 100% of the traffic routed to v2.

Overview

A general overview of what you will be doing in the Step By Step section.

Observe the traffic in Kiali
Remove the header based routing to v3 of the name service

The use case it that has been validated and you want to load test it now

Use the mirror field to shadow all traffic to v2 to v3 also
Inspect the v3 workload to see if it is receiving traffic

Step By Step

Expand the Tasks section below to do the exercise.

Tasks

Task: Remove the `match` block from `name-vs.yaml`

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - route:
    - destination:
        host: name
        subset: name-v2
      weight: 100

Task: Add the `mirror` blocks in `name-vs.yaml`

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: name-route
spec:
  hosts:
  - name
  exportTo:
  - "."
  gateways:
  - mesh
  http:
  - route:
    - destination:
        host: name
        subset: name-v2
      weight: 100
    mirror:
      host: name
      subset: name-v3
    mirrorPercentage:
      value: 100.0

Task: Apply the changes to `name-vs.yaml`

kubectl apply -f 02-deployment-patterns/start/name-vs.yaml

Task: Run `scripts/loop-query.sh`

./scripts/loop-query.sh

You should only see responses from the name-v2 workload. It should look something like the following.

Porthos (v2) is 82 years
Athos (v2) is 42 years
Athos (v2) is 85 years
Athos (v2) is 16 years
d'Artagnan (v2) is 57 years
Porthos (v2) is 99 years
Porthos (v2) is 34 years
Athos (v2) is 92 years
Porthos (v2) is 11 years
Athos (v2) is 75 years

Requests mirrored to the name-v3 workload are done as fire and forget requests. All responses are discarded.

Task: Inspect the `name-v3` workload to see if it is receiving traffic

Export the pod name to an environment variable.

export NAME_V3_POD=$(kubectl get pod -l app=sentences,version=v3 -o jsonpath={.items..metadata.name})

Get the logs with kubectl.

kubectl logs "$NAME_V3_POD" --tail=20 --follow

You should see GET requests hitting the name-v3 workload. It should look something like this.

WARNING:root:Using name 'd'Artagnan (v3)'
WARNING:root:Operation 'name' took 0.171ms
127.0.0.1 - - [01/Jul/2021 14:18:13] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [01/Jul/2021 14:18:13] "GET / HTTP/1.1" 200 -
WARNING:root:Using name 'Porthos (v3)'
WARNING:root:Operation 'name' took 0.160ms
127.0.0.1 - - [01/Jul/2021 14:18:13] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [01/Jul/2021 14:18:13] "GET / HTTP/1.1" 200 -
WARNING:root:Using name 'Athos (v3)'
WARNING:root:Operation 'name' took 0.238ms
127.0.0.1 - - [01/Jul/2021 14:18:14] "GET / HTTP/1.1" 200 -

Task: Observe the traffic in Kiali

Go to Graph menu item and select the Versioned app graph from the drop down menu.

If you look at the version app graph you may or may not see the mirrored traffic being shown. It depends on the version of Kiali you are using.

But you know that the client is not getting responses from v3 from the terminals output.

Porthos (v2) is 82 years
Athos (v2) is 42 years
Athos (v2) is 85 years
Athos (v2) is 16 years
d'Artagnan (v2) is 57 years
Porthos (v2) is 99 years
Porthos (v2) is 34 years
Athos (v2) is 92 years
Porthos (v2) is 11 years
Athos (v2) is 75 years

Go to the menu item Workloads and select name-v3 and the inbound metrics tab.

Metrics are also collected for the destination traffic of v3. If you select Source from the Reported from drop down you will not see any metrics as responses are discarded.

Summary

In these exercises you learned some of functionality provided by Istio's HTTPRoute. Implementing blue/green, canary and shadow deployment patterns. But there is a lot more you can do. You can also manipulate headers, do HTTP redirects, HTTP rewrites, etc.

See the documentation for a more complete overview of what you can do.

The main takeaways are:

Istio enables you to apply various deployment patterns inside the mesh
The match field is quite powerful
The order in which you update DestinationRules and VirtualServices is important
You should always provide a default route when using a match
The application MUST propagate the headers in order for the virtual service to work with it

Cleanup

kubectl delete -f 02-deployment-patterns/start/name-v3
kubectl delete -f 02-deployment-patterns/start/name-v2
kubectl delete -f 02-deployment-patterns/start/name-v1
kubectl delete -f 02-deployment-patterns/start/

Files

02-deployment-patterns.md

Latest commit

History

02-deployment-patterns.md

File metadata and controls

Deployment Patterns

Learning goal

Introduction

Exercise: Blue/Green Deployment

Overview

Step by Step

Task: Deploy the sentences app

Task: Run the scripts/loop-query.sh script

Task: Observe the traffic flow with Kiali

Task: Stop the loop-query.sh

Task: Add a match block with header condition

Task: Run loop-query.sh with the x-test header

Task: Observe the traffic flow with Kiali

Task: Run the scripts/loop-query.sh without header

Task: Observe the traffic flow with Kiali

Task: Remove the default route

Task: Observe the traffic flow with Kiali

Task: Add the default route and use a different header

Task: Observe the traffic flow with Kiali

Exercise: Canary Deployment

Overview

Step By Step

Task: Deploy the sentences app with three version of the name service.

Task: Open file 02-deployment-patterns/start/name-dr.yaml and add name-v3:

Task: Adjust the match field in name-vs.yaml to use-v3

Task: Run scripts/loop-query.sh

Task: Observe the traffic flow with Kiali

Task: Add the weight fields in name-vs.yaml

Task: Observe the traffic flow with Kiali

Task: In a new terminal pass the header use-v3 to scripts/loop-query.sh

Task: Observe the traffic flow with Kiali

Task: Promote v2 to receive all traffic

Task: Observe the traffic flow with Kiali

Exercise: Shadow Deployment

Overview

Step By Step

Task: Remove the match block from name-vs.yaml

Task: Add the mirror blocks in name-vs.yaml

Task: Apply the changes to name-vs.yaml

Task: Run scripts/loop-query.sh

Task: Inspect the name-v3 workload to see if it is receiving traffic

Task: Observe the traffic in Kiali

Summary

Cleanup

Task: Run the `scripts/loop-query.sh` script

Task: Stop the `loop-query.sh`

Task: Run loop-query.sh with the `x-test` header

Task: Run the `scripts/loop-query.sh` without header

Task: Deploy the sentences app with three version of the `name` service.

Task: Open file `02-deployment-patterns/start/name-dr.yaml` and add `name-v3`:

Task: Adjust the `match` field in `name-vs.yaml` to `use-v3`

Task: Run `scripts/loop-query.sh`

Task: Add the `weight` fields in `name-vs.yaml`

Task: In a new terminal pass the header `use-v3` to `scripts/loop-query.sh`

Task: Promote `v2` to receive all traffic

Task: Remove the `match` block from `name-vs.yaml`

Task: Add the `mirror` blocks in `name-vs.yaml`

Task: Apply the changes to `name-vs.yaml`

Task: Run `scripts/loop-query.sh`

Task: Inspect the `name-v3` workload to see if it is receiving traffic