Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add control to use timestamp to trigger the package creation or not (useful for CI/CD) #521

Merged
merged 5 commits into from
Jan 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -621,7 +621,7 @@ Q2: How to force recreate deployment package?

Q3: `null_resource.archive[0] must be replaced`

> Answer: This probably mean that zip-archive has been deployed, but is currently absent locally, and it has to be recreated locally. When you run into this issue during CI/CD process (where workspace is clean) or from multiple workspaces, you can set environment variable `TF_RECREATE_MISSING_LAMBDA_PACKAGE=false` or pass `recreate_missing_package = false` as a parameter to the module and run `terraform apply`.
> Answer: This probably mean that zip-archive has been deployed, but is currently absent locally, and it has to be recreated locally. When you run into this issue during CI/CD process (where workspace is clean) or from multiple workspaces, you can set environment variable `TF_RECREATE_MISSING_LAMBDA_PACKAGE=false` or pass `recreate_missing_package = false` as a parameter to the module and run `terraform apply`. Alternatively, you can pass `trigger_on_package_timestamp = false` as a parameter to ignore the file timestamp when deciding to create the archive or not.

Q4: What does this error mean - `"We currently do not support adding policies for $LATEST."` ?

Expand Down Expand Up @@ -651,6 +651,7 @@ Q4: What does this error mean - `"We currently do not support adding policies fo
- [Event Source Mapping](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/event-source-mapping) - Create Lambda Function with event source mapping configuration (SQS, DynamoDB, Amazon MQ, and Kinesis).
- [Triggers](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/triggers) - Create Lambda Function with some triggers (eg, Cloudwatch Events, EventBridge).
- [Code Signing](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/code-signing) - Create Lambda Function with code signing configuration.
- [Simple CI/CD](https://github.com/terraform-aws-modules/terraform-aws-lambda/tree/master/examples/simple-cicd) - Create Lambda Function as if it runs on CI/CD platform where `builds` directory is often absent.

# Examples by the users of this module

Expand Down Expand Up @@ -852,6 +853,7 @@ No modules.
| <a name="input_timeout"></a> [timeout](#input\_timeout) | The amount of time your Lambda Function has to run in seconds. | `number` | `3` | no |
| <a name="input_timeouts"></a> [timeouts](#input\_timeouts) | Define maximum timeout for creating, updating, and deleting Lambda Function resources | `map(string)` | `{}` | no |
| <a name="input_tracing_mode"></a> [tracing\_mode](#input\_tracing\_mode) | Tracing mode of the Lambda Function. Valid value can be either PassThrough or Active. | `string` | `null` | no |
| <a name="input_trigger_on_package_timestamp"></a> [trigger\_on\_package\_timestamp](#input\_trigger\_on\_package\_timestamp) | Whether to recreate the Lambda package if the timestamp changes | `bool` | `true` | no |
| <a name="input_trusted_entities"></a> [trusted\_entities](#input\_trusted\_entities) | List of additional trusted entities for assuming Lambda Function role (trust relationship) | `any` | `[]` | no |
| <a name="input_use_existing_cloudwatch_log_group"></a> [use\_existing\_cloudwatch\_log\_group](#input\_use\_existing\_cloudwatch\_log\_group) | Whether to use an existing CloudWatch log group or create new | `bool` | `false` | no |
| <a name="input_vpc_security_group_ids"></a> [vpc\_security\_group\_ids](#input\_vpc\_security\_group\_ids) | List of security group ids when Lambda Function should run in the VPC. | `list(string)` | `null` | no |
Expand Down
4 changes: 4 additions & 0 deletions examples/fixtures/python3.10-app1/index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
def lambda_handler(event, context):
print("Hello from app1!")

return event
1 change: 1 addition & 0 deletions examples/simple-cicd/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/src
53 changes: 53 additions & 0 deletions examples/simple-cicd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Simple CI/CD example

Configuration in this directory creates AWS Lambda Function as it would run in a context of CICD executions, where the Terraform working directory is empty and there is no `builds` directory, that:

- `terraform plan` doesn't trigger a diff if the source code of the lambda function didn't change.
- `terraform plan` does trigger a diff if the source code of the lambda function has changed.
- `terraform apply` works if the code has changed.

## Usage

To run this example you need to execute:

```bash
./test.sh
```

Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.63 |
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_lambda_function"></a> [lambda\_function](#module\_lambda\_function) | ../../ | n/a |

## Resources

| Name | Type |
|------|------|
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |

## Inputs

No inputs.

## Outputs

No outputs.
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
26 changes: 26 additions & 0 deletions examples/simple-cicd/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
provider "aws" {
region = "eu-west-1"
# region = "us-east-1"

# Make it faster by skipping something
skip_metadata_api_check = true
skip_region_validation = true
skip_credentials_validation = true
}

resource "random_pet" "this" {
length = 2
}

module "lambda_function" {
source = "../../"

function_name = "${random_pet.this.id}-lambda-simple"
handler = "index.lambda_handler"
runtime = "python3.10"

source_path = [
"${path.module}/src/python3.10-app1",
]
trigger_on_package_timestamp = false
}
Empty file added examples/simple-cicd/outputs.tf
Empty file.
126 changes: 126 additions & 0 deletions examples/simple-cicd/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# vim:ts=4:sw=4:noet

set -eo pipefail

trap ctrl_c INT

ctrl_c() {
echo "** Trapped CTRL-C"
exit 1
}

failed=0

:echo() {
local color=${2:-"33;1"}
echo -e "\e[${color}m$1\e[0m"
}

:note() {
:echo "$1" "35;1"
}

:case() {
if [ $? -ne 0 ]
then failed=1
fi

if [ "$failed" -eq 1 ]
then :echo "SKIPPED: $1"; return 1
else echo; :echo "CASE: $1"
fi
}

:check_diff() {
expected="$1"

set +e
terraform plan -detailed-exitcode
status=$?
set -e
# ${status} possible values:
# 0 - Succeeded, diff is empty (no changes)
# 1 - Errored
# 2 - Succeeded, there is a diff
if [ "${status}" -ne "${expected}" ]; then
case "${expected}" in
0)
:echo "Error: we don't expect any diff here!"
return 1
;;
2)
echo "Error: we DO expect some diff here!"
return 1
;;
esac
fi
}

terraform=$(which terraform)
terraform() {
$terraform "$@" < <(yes yes)
}

:note "Preparing ..."
rm -rf src
mkdir -p src
cp -r "../fixtures/python3.10-app1" src
terraform init
:echo "Destroy / Remove ZIP files"
terraform destroy
rm -rf builds 2>/dev/null || true

#############################################################
# Part 1: Check that CICD environment won't detect any diff #
#############################################################

:echo
:note "Starting Part 1: Check that CICD environment won't detect any diff"

:case "Apply / No diff" && {
terraform apply
:check_diff 0
}

:case "Remove 'builds' dir / No diff" && {
rm -rf builds
:check_diff 0
}

###############################################################################
# Part 2: Check that CICD environment will detect diff if lambda code changes #
###############################################################################

:echo
:note "Starting Part 2: Check that CICD environment will detect diff if lambda code changes"

:note "Change the source code / Remove 'builds' dir"
echo "" >> src/python3.10-app1/index.py
rm -rf builds

:case "Plan / Expect diff" && {
terraform plan
:check_diff 2
}

:case "Apply / No diff" && {
terraform apply
:check_diff 0
}

:note "Remove 'builds' dir"
rm -rf builds

:case "Plan / No diff" && {
terraform plan
:check_diff 0
}

#:case "Destroy / Remove ZIP files" && {
# terraform plan -destroy
# terraform destroy -auto-approve
# rm builds/*.zip
#}

:note "All tests have passed successfully."
Empty file.
14 changes: 14 additions & 0 deletions examples/simple-cicd/versions.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
terraform {
required_version = ">= 1.0"

required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.63"
}
random = {
source = "hashicorp/random"
version = ">= 2.0"
}
}
}
2 changes: 1 addition & 1 deletion package.tf
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ resource "null_resource" "archive" {

triggers = {
filename = data.external.archive_prepare[0].result.filename
timestamp = data.external.archive_prepare[0].result.timestamp
timestamp = var.trigger_on_package_timestamp ? data.external.archive_prepare[0].result.timestamp : null
}

provisioner "local-exec" {
Expand Down
6 changes: 6 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -761,3 +761,9 @@ variable "recreate_missing_package" {
type = bool
default = true
}

variable "trigger_on_package_timestamp" {
description = "Whether to recreate the Lambda package if the timestamp changes"
type = bool
default = true
}
1 change: 1 addition & 0 deletions wrappers/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ module "wrapper" {
timeout = try(each.value.timeout, var.defaults.timeout, 3)
timeouts = try(each.value.timeouts, var.defaults.timeouts, {})
tracing_mode = try(each.value.tracing_mode, var.defaults.tracing_mode, null)
trigger_on_package_timestamp = try(each.value.trigger_on_package_timestamp, var.defaults.trigger_on_package_timestamp, true)
trusted_entities = try(each.value.trusted_entities, var.defaults.trusted_entities, [])
use_existing_cloudwatch_log_group = try(each.value.use_existing_cloudwatch_log_group, var.defaults.use_existing_cloudwatch_log_group, false)
vpc_security_group_ids = try(each.value.vpc_security_group_ids, var.defaults.vpc_security_group_ids, null)
Expand Down