Artifact Storage #3449
Labels
pending-decision
Pending decision from maintainers
preserved
Preserved issues never go stale
rfc
Request For Comments
Summary
Add new functionality to Terragrunt so that module outputs, local variables and arbitrary string data (I'll collectively refer to these as artifacts until a better name comes around) can be stored in an IaC-agnostic location. This will allow other tools and services to consume these values directly without requiring the user to bake this into their module.
This location would start off simple, storing these artifacts inside AWS Systems Manager Parameter Store (SSM), and could evolve into DNS, S3 JSON objects etc.
Motivation
If Terraform/OpenTofu/Terragrunt is your only IaC tool, passing data between different pieces of infrastructure is fairly straight forward - either use the Terraform remote_state datasource or have it all abstracted away from you through Terragrunt dependencies.
When you inevitably want to start integrating Terraform resources with other IaC frameworks, such as Serverless Framework or AWS CDK, sharing resource IDs or URLs become tricky.
In the past, I've opted for one of two options - either wrap the third-party Terraform module inside my own to store everything I need inside aws_ssm_parameter resources, or manually create these SSM parameters and remember to update them each time a resource is updated.
The goal of this proposal is to allow a seamless integration and data sharing between Terragrunt-managed resources and other Infrastructure as Code frameworks, initially through the use of SSM parameters.
Proposal
Introduce a new top-level configuration named "artifacts". This "artifacts" config contains options to choose the storage backend and define the data to be stored in those artifacts. These artifact values can be fetched from a local Terragrunt variable, module outputs or arbitrary strings defined inline.
This proposal addresses the motivation outlined above by acting as a native way to store data in a location where most other IaC frameworks are able to read from. CloudFormation, Serverless Framework, Pulumni and CDK all directly support consuming data from SSM parameters.
The following is an example of the interface that may be exposed to a user:
Although I don't believe this should be a requirement, other Terragrunt units would be able to reference this data through the typical dependency blocks, under an object named "artifacts". I.E.
dependency.vpc.artifacts.data["/SHARED/ACCOUNT/ID"]
.Technical Details
Introduce a new top-level block -
artifacts
.This
artifacts
block will start off containing the following configuration items:backend
- for selecting between SSM,DNS, S3 etcdata
- to define each artifact's location and data contained withinssm_XYZ
- allows the user to customize the SSM location, such as account, region etc.Over time, more backends can be supported, however SSM seems to be the most popular currently.
This configuration block would be evaluated after Terragrunt has finished a plan or apply step, and will make the necessary SDK calls or generate the Terraform resources required to create the artifacts.
Press Release
Terragrunt now supports sharing data with other frameworks!
Using the new
artifacts
block, users can define framework-agnostic locations to store data from your Terragrunt units. A VPC module deployed through Terragrunt could store the VPC and subnet IDs as artifacts to allow alternative IaC frameworks to consume this information.Drawbacks
No response
Alternatives
No response
Migration Strategy
No response
Unresolved Questions
No response
References
No response
Proof of Concept Pull Request
No response
Support Level
Customer Name
No response
The text was updated successfully, but these errors were encountered: