Skip to content

Commit

Permalink
feat: adding extension notebooks (terraform-google-modules#303)
Browse files Browse the repository at this point in the history
  • Loading branch information
shanecglass authored Feb 12, 2024
1 parent d676b28 commit b2816c4
Show file tree
Hide file tree
Showing 24 changed files with 2,032 additions and 2,855 deletions.
2 changes: 1 addition & 1 deletion metadata.display.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023 Google LLC
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
26 changes: 10 additions & 16 deletions metadata.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023 Google LLC
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -27,7 +27,7 @@ spec:
version: 7.0.0
actuationTool:
flavor: Terraform
version: ">= 0.13"
version: ">= 1.3"
description: {}
content:
subBlueprints:
Expand Down Expand Up @@ -61,7 +61,6 @@ spec:
- name: dataset_id
description: Unique ID for the dataset being provisioned.
varType: string
defaultValue: null
required: true
- name: dataset_labels
description: Key value pairs in a map for dataset labels
Expand All @@ -70,27 +69,22 @@ spec:
- name: dataset_name
description: Friendly name for the dataset being provisioned.
varType: string
defaultValue: null
- name: default_table_expiration_ms
description: TTL of tables using the dataset in MS
varType: number
defaultValue: null
- name: delete_contents_on_destroy
description: (Optional) If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present.
varType: bool
defaultValue: null
- name: deletion_protection
description: Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail
description: Whether or not to allow deletion of tables and external tables defined by this module. Can be overriden by table-level deletion_protection configuration.
varType: bool
defaultValue: false
- name: description
description: Dataset description.
varType: string
defaultValue: null
- name: encryption_key
description: Default encryption key to apply to the dataset. Defaults to null (Google-managed).
varType: string
defaultValue: null
- name: external_tables
description: A list of objects which include table_id, expiration_time, external_data_configuration, and labels.
varType: |-
Expand Down Expand Up @@ -120,9 +114,10 @@ spec:
mode = string,
source_uri_prefix = string,
}),
expiration_time = string,
max_staleness = optional(string),
labels = map(string),
expiration_time = string,
max_staleness = optional(string),
deletion_protection = optional(bool),
labels = map(string),
}))
defaultValue: []
- name: location
Expand Down Expand Up @@ -161,11 +156,9 @@ spec:
- name: max_time_travel_hours
description: Defines the time travel window in hours
varType: number
defaultValue: null
- name: project_id
description: Project where the dataset and table are created
varType: string
defaultValue: null
required: true
- name: routines
description: A list of objects which include routine_id, routine_type, routine_language, definition_body, return_type, routine_description and arguments.
Expand Down Expand Up @@ -208,8 +201,9 @@ spec:
interval = string,
}),
}),
expiration_time = string,
labels = map(string),
expiration_time = string,
deletion_protection = optional(bool),
labels = map(string),
}))
defaultValue: []
- name: views
Expand Down
2 changes: 1 addition & 1 deletion modules/authorization/metadata.display.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023 Google LLC
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
6 changes: 2 additions & 4 deletions modules/authorization/metadata.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2023 Google LLC
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -28,7 +28,7 @@ spec:
version: 7.0.0
actuationTool:
flavor: Terraform
version: ">= 0.13"
version: ">= 1.3"
description: {}
content:
examples:
Expand Down Expand Up @@ -73,12 +73,10 @@ spec:
- name: dataset_id
description: Unique ID for the dataset being provisioned.
varType: string
defaultValue: null
required: true
- name: project_id
description: Project where the dataset and table are created
varType: string
defaultValue: null
required: true
- name: roles
description: An array of objects that define dataset access for one or more entities.
Expand Down
12 changes: 10 additions & 2 deletions modules/data_warehouse/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The resources/services/activations/deletions that this module will create/trigge
- Creates and inferences with a BigQuery ML model
- Creates a remote model and uses Generative AI to generate text through a BigQuery ML remote model
- Creates a Looker Studio report
- Deploys follow-on learning notebooks

### preDeploy
To deploy this blueprint you must have an active billing account and billing permissions.
Expand All @@ -35,9 +36,11 @@ Functional examples are included in the

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| deletion\_protection | Whether or not to protect GCS resources from deletion when solution is modified or changed. | `string` | `true` | no |
| create\_ignore\_service\_accounts | Whether or not to ignore creation of a service account if an account of the same name already exists | `string` | `true` | no |
| dataform\_region | Region that is used to deploy Dataform resources. This does not limit where resources can be run or what region data must be located in. | `string` | `null` | no |
| deletion\_protection | Whether or not to protect GCS resources from deletion when solution is modified or changed. | `string` | `false` | no |
| enable\_apis | Whether or not to enable underlying apis in this solution. | `string` | `true` | no |
| force\_destroy | Whether or not to protect BigQuery resources from deletion when solution is modified or changed. | `string` | `false` | no |
| force\_destroy | Whether or not to protect BigQuery resources from deletion when solution is modified or changed. | `string` | `true` | no |
| labels | A map of labels to apply to contained resources. | `map(string)` | <pre>{<br> "data-warehouse": true<br>}</pre> | no |
| project\_id | Google Cloud Project ID | `string` | n/a | yes |
| region | Google Cloud Region | `string` | n/a | yes |
Expand Down Expand Up @@ -86,6 +89,7 @@ A project with the following APIs enabled must be used to host the
resources of this module:

- Vertex AI API: `aiplatform.googleapis.com`
- Artifact Registry API: `artifactregistry.googleapis.com`
- BigQuery API: `bigquery.googleapis.com`
- BigQuery Connection API: `bigqueryconnection.googleapis.com`
- BigQuery Data Policy API: `bigquerydatapolicy.googleapis.com`
Expand All @@ -95,10 +99,14 @@ resources of this module:
- BigQuery Storage API: `bigquerystorage.googleapis.com`
- Google Cloud APIs: `cloudapis.googleapis.com`
- Cloud Build API: `cloudbuild.googleapis.com`
- Cloud Functions API: `cloudfunctions.googleapis.com`
- Compute Engine API: `compute.googleapis.com`
- Infrastructure Manager API: `config.googleapis.com`
- Data Catalog API: `datacatalog.googleapis.com`
- Dataform API: `dataform.googleapis.com`
- Data Lineage API: `datalineage.googleapis.com`
- Notebooks API: `notebooks.googleapis.com`
- Cloud Run API: `run.googleapis.com`
- Service Usage API: `serviceusage.googleapis.com`
- Google Cloud Storage API: `storage.googleapis.com`
- Google Cloud Storage JSON API: `storage-api.googleapis.com`
Expand Down

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3,342 changes: 604 additions & 2,738 deletions modules/data_warehouse/assets/data-warehouse-architecture.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit b2816c4

Please sign in to comment.