Skip to content

Latest commit

 

History

History
 
 

terraform

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
modules
modules
 
 
README.md
README.md
 
 
iam.tf
iam.tf
 
 
main.tf
main.tf
 
 
variables.auto.tfvars
variables.auto.tfvars
 
 
variables.tf
variables.tf
 
 
versions.tf
versions.tf
 
 

README.md

Vertex Pipelines

This Terraform module contains Infrastructure-as-Code (IaC) for an example deployment.

Infrastructure Overview

Below is a list of the infrastructure that is created as part of this Terraform module.

  • Enabling of relevant Google Cloud APIs
    • NOTE: make sure the 'Service Usage' API is enabled (in the console)
  • GCS bucket where the compiled pipelines will be published
  • GCS bucket for the Cloud Function build
  • GCS bucket for the "assets" folder
  • Cloud Function for triggering the pipelines
  • Pub/Sub topic that is used to trigger the Cloud Function
  • Cloud Scheduler jobs for scheduling the pipeline runs
  • Service accounts (and suitable IAM roles) for:
    • Vertex Pipelines to execute the pipelines
    • Cloud Function to trigger the pipeline execution
    • Cloud Scheduler to publish messages to the pub/sub topic

Requirements

Name Version
terraform >= 0.13
google ~> 4.9.0

Providers

Name Version
google 3.85.0

Modules

Name Source Version
api_services ./modules/apis n/a
assets_bucket_iam ./modules/bucket_iam n/a
compiled_pipelines_bucket_iam ./modules/bucket_iam n/a
function ./modules/function n/a
gcs_buckets terraform-google-modules/cloud-storage/google ~> 2.1
pipeline_root_bucket_iam ./modules/bucket_iam n/a
pipelines_sa_iam ./modules/sa_iam n/a
pubsub terraform-google-modules/pubsub/google ~> 2.0
scheduler ./modules/pubsub_scheduler n/a
service_accounts terraform-google-modules/service-accounts/google ~> 3.0

Resources

Name Type
google_app_engine_application.app resource

Inputs

Name Description Type Default Required
api_list List of Google Cloud APIs to enable on the project. list(string) [] no
app_engine_region Region for the App Engine application. string n/a yes
cloud_function_config Config for the Cloud Function for triggering pipelines, 
object({
    name          = string
    region        = string
    description   = string
    vpc_connector = string
  })
 n/a yes
cloud_schedulers_config Map of configurations for cloud scheduler jobs (each a different pipeline schedule). 
map(object({
    name         = string
    region       = string
    description  = string
    schedule     = string
    time_zone    = string
    payload_file = string
  }))
 {} no
gcs_buckets_names Map of names of GCS buckets to create. map(string) {} no
project_id The ID of the project in which to provision resources. string n/a yes
pubsub_topic_name Name of the Pub/Sub topic to create for triggering pipelines. string n/a yes
service_accounts Map of service accounts to create. 
map(object({
    name          = string
    display_name  = string
    project_roles = list(string)
  }))
 {} no
vertex_region Region for Vertex Pipelines execution. string n/a yes

Outputs

No outputs.