terraform-google-logging-sink

A Terraform module to create and manage Google Project Logging Sinks.

This module supports Terraform version 1 and is compatible with the Terraform AWS Provider version 4.1. and 5._**

This module is part of our Infrastructure as Code (IaC) framework that enables our users and customers to easily deploy and manage reusable, secure, and production-grade cloud infrastructure.

• Module Features
• Getting Started
• Module Argument Reference
  • Main Resource Configuration
  • Module Configuration
• Module Outputs
• External Documentation
  • AWS Documentation IAM
  • Terraform AWS Provider Documentation
• Module Versioning
  • Backwards compatibility in 0.0.z and 0.y.z version
• About Mineiros
• Reporting Issues
• Contributing
• Makefile Targets
• License

Module Features

This module implements the following Terraform resources:

• google_logging_project_sink

Getting Started

Most common usage of the module:

module "terraform-google-logging-sink" {
  source      = "git@github.com:mineiros-io/terraform-google-logging-sink.git?ref=v0.0.1"

  name        = "my-pubsub-instance-sink"
  destination = "pubsub.googleapis.com/projects/my-project/topics/instance-activity"
}

Module Argument Reference

See variables.tf and examples/ for details and use-cases.

Main Resource Configuration

• name: (Required string)

  The name of the logging sink.

• destination: (Required string)

  The destination of the sink (or, in other words, where logs are written to).

  Can be a Cloud Storage bucket, a PubSub topic, a BigQuery dataset or a Cloud Logging bucket.

  Examples:

  • "storage.googleapis.com/[GCS_BUCKET]"
  • "bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET]"
  • "pubsub.googleapis.com/projects/[PROJECT_ID]/topics/[TOPIC_ID]"
  • "logging.googleapis.com/projects/[PROJECT_ID]]/locations/global/buckets/[BUCKET_ID]"

  The writer associated with the sink must have access to write to the above resource.

• filter: (Optional string)

  The filter to apply when exporting logs. Only log entries that match the filter are exported.

  See Advanced Log Filters for information on how to write a filter.

• description: (Optional string)

  A description of this sink. The maximum length of the description is 8000 characters.

• disabled: (Optional bool)

  If set to True, then this sink is disabled and it does not export any log entries.

• project: (Optional string)

  The ID of the project to create the sink in.

  If omitted and either var.organization or var.folder are present, no project logging sink is created.

  If omitted and both var.organization and var.folder are omitted, the project associated with the provider is used.

• unique_writer_identity: (Optional bool)

  Whether or not to create a unique identity associated with this sink.

  If false (the default), then the writer_identity used is serviceAccount:cloud-logs@system.gserviceaccount.com.

  If true, then a unique service account is created and used for this sink. If you wish to publish logs across projects or utilize bigquery_options, you must set unique_writer_identity to true.

  Default is null.

• use_partitioned_tables: (Optional bool)

  Whether to use BigQuery's partition tables.

  By default, Logging creates dated tables based on the log entries' timestamps, e.g. syslog_20170523. With partitioned tables the date suffix is no longer present and special query syntax has to be used instead. In both cases, tables are sharded based on UTC timezone.

  Default is null.

• exclusions: (Optional list(exclusion))

  Log entries that match any of the exclusion filters will not be exported.

  If a log entry is matched by both filter and one of exclusion_filters it will not be exported. Can be repeated multiple times for multiple exclusions.

  Default is [].

  Each exclusion object in the list accepts the following attributes:

  • name: (Required string)

    A client-assigned identifier, such as load-balancer-exclusion.

    Identifiers are limited to 100 characters and can include only letters, digits, underscores, hyphens, and periods. First character has to be alphanumeric.

    The object accepts the following attributes:

    • description: (Optional string)

      A description of this exclusion.

    • filter: (Required string)

      An advanced logs filter that matches the log entries to be excluded. By using the sample function, you can exclude less than 100% of the matching log entries.

      See Advanced Log Filters for information on how to write a filter.

    • disabled: (Optional bool)

      If set to true, then this exclusion is disabled and it does not exclude any log entries.

• folder: (Optional string)

  The ID of the folder to create the sink in.

  If omitted, no folder logging sink is created.

  If provided along with var.project, only the project logging sink is created.

  Default is null.

• organization: (Optional string)

  The ID of the organization to create the sink in.

  If omitted, no organization logging sink is created.

  If provided along with var.project, only the project logging sink is created.

  If provided along with var.folder, the folder logging sink is created instead.

  Default is null.

• include_children: (Optional bool)

  Whether or not to include child projects in the sink export.

  If true, logs associated with child projects are also exported; otherwise only logs relating to the provided folder or organization are included.

  Default is null.

Module Configuration

• module_enabled: (Optional bool)

  Specifies whether resources in the module will be created.

  Default is true.

• module_depends_on: (Optional list(dependency))

  A list of dependencies. Any object can be assigned to this list to define a hidden external dependency.

  Default is [].

  Example:

  module_depends_on = [
    null_resource.name
  ]

Module Outputs

The following attributes are exported in the outputs of the module:

• project_sink: (list(object))

  All attributes of the created google_logging_project_sink resource.

• folder_sink: (list(object))

  All attributes of the created google_logging_folder_sink resource.

• organization_sink: (list(object))

  All attributes of the created google_logging_organization_sink resource.

• module_enabled: (bool)

  Whether this module is enabled.

External Documentation

AWS Documentation IAM

• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html
• https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies.html
• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

Terraform AWS Provider Documentation

• https://www.terraform.io/docs/providers/aws/r/iam_role.html
• https://www.terraform.io/docs/providers/aws/r/iam_role_policy.html
• https://www.terraform.io/docs/providers/aws/r/iam_role_policy_attachment.html
• https://www.terraform.io/docs/providers/aws/r/iam_instance_profile.html

Module Versioning

This Module follows the principles of Semantic Versioning (SemVer).

Given a version number MAJOR.MINOR.PATCH, we increment the:

1. MAJOR version when we make incompatible changes,
2. MINOR version when we add functionality in a backwards compatible manner, and
3. PATCH version when we make backwards compatible bug fixes.

Backwards compatibility in 0.0.z and 0.y.z version

• Backwards compatibility in versions 0.0.z is not guaranteed when z is increased. (Initial development)
• Backwards compatibility in versions 0.y.z is not guaranteed when y is increased. (Pre-release)

About Mineiros

Mineiros is a remote-first company headquartered in Berlin, Germany that solves development, automation and security challenges in cloud infrastructure.

Our vision is to massively reduce time and overhead for teams to manage and deploy production-grade and secure cloud infrastructure.

We offer commercial support for all of our modules and encourage you to reach out if you have any questions or need help. Feel free to email us at hello@mineiros.io or join our Community Slack channel.

Reporting Issues

We use GitHub Issues to track community reported issues and missing features.

Contributing

Contributions are always encouraged and welcome! For the process of accepting changes, we use Pull Requests. If you'd like more information, please see our Contribution Guidelines.

Makefile Targets

This repository comes with a handy Makefile. Run make help to see details on each available target.

License

This module is licensed under the Apache License Version 2.0, January 2004. Please see LICENSE for full details.

Copyright © 2020-2022 Mineiros GmbH