Add BigQuery dataset for cloud cost management #13

brettcurtis · 2024-02-24T18:09:08Z

This is a section of the work required for #9. We are waiting for an upstream issue before we can manage it all.

Summary by CodeRabbit

New Features
- Introduced Cloud Cost Management feature, which can be enabled or disabled based on user preference.
- Added capability to export Cloud Billing data to BigQuery for enhanced cost analysis.
Documentation
- Updated documentation to reflect new features and updated provider versions.
Tests
- Added test configurations to validate the Cloud Cost Management feature and BigQuery data export functionality.

coderabbitai · 2024-02-24T18:09:12Z

Walkthrough

The update introduces a new feature for managing cloud costs by enabling specific Google Cloud APIs and adding a BigQuery dataset for billing export. It includes updates to provider versions, a new variable for toggling cloud cost management, and adjustments in test fixtures to support the new functionality. This change allows for more efficient cloud cost analysis and management.

Changes

File Path	Change Summary
`README.md`, `global/README.md`	Added conditions for enabling APIs based on `enable_cloud_cost_management`, updated provider versions, and added `google_bigquery_dataset.billing_export`.
`global/main.tf`	Added a new BigQuery dataset resource for exporting billing data.
`global/variables.tf`	Introduced `enable_cloud_cost_management` variable to control cloud cost data collection.
`test/fixtures/.../infracost-usage.yml`	Specified usage values for BigQuery dataset resources related to billing export.
`test/fixtures/.../main.tf`	Enabled cloud cost management in the test module.
`test/fixtures/.../variables.tf`	Minor modification related to the `project` variable declaration.

Related issues

Support for enabling Cloud Cost Management #9: The changes in this PR support enabling Cloud Cost Management by allowing the export of cost data from BigQuery, which aligns with the issue's objective of analyzing cloud costs and usage data across Datadog.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit-tests for this file.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit tests for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository from git and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit tests.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
The JSON schema for the configuration file is available here.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

infracost · 2024-02-24T18:09:16Z

Infracost report

💰 Monthly cost will not change

Project	Cost change	New monthly cost
default_integration	$0	$9

Cost details

──────────────────────────────────
Project: default_integration
Module path: test/fixtures/default_integration

+ module.test.google_bigquery_dataset.billing_export[0]
  Monthly cost depends on usage

    + Queries (on-demand)
      Monthly cost depends on usage
        +$6.25 per TB

Monthly cost change for default_integration (Module path: test/fixtures/default_integration)
Amount:  $0.00 ($9 → $9)
Percent: 0%

──────────────────────────────────
Key: ~ changed, + added, - removed

11 cloud resources were detected:
∙ 4 were estimated, all of which include usage-based costs, see https://infracost.io/usage-file
∙ 7 were free, rerun with --show-skipped to see details

Governance checks

🟢 49 passed

48 FinOps policies and 1 Tagging policy passed.

_{View in Infracost Cloud. This comment will be updated when code changes.}

coderabbitai

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yaml

Commits

Files that changed from the base of the PR and between 5018923 and 06b8db9.

Files selected for processing (7)

README.md (1 hunks)
global/README.md (3 hunks)
global/main.tf (1 hunks)
global/variables.tf (1 hunks)
test/fixtures/default_integration/infracost-usage.yml (1 hunks)
test/fixtures/default_integration/main.tf (1 hunks)
test/fixtures/default_integration/variables.tf (1 hunks)

Files skipped from review due to trivial changes (1)

test/fixtures/default_integration/variables.tf

Additional comments: 8

test/fixtures/default_integration/main.tf (1)

17-20: The addition of the enable_cloud_cost_management = true attribute within the module "test" block is consistent with the PR objectives to enable cloud cost management features. Ensure that the usage of this attribute is consistent across the codebase and properly documented.

global/variables.tf (1)

12-16: The addition of the enable_cloud_cost_management variable is crucial for controlling the cloud cost management features. Ensure that the description provided is clear and comprehensive for users to understand its purpose and impact.

test/fixtures/default_integration/infracost-usage.yml (1)

15-27: The addition of usage values for google_bigquery_dataset and module.test.google_bigquery_dataset.billing_export[0] resources is important for accurate cost estimation and management. Ensure that these values are reviewed and adjusted based on realistic usage patterns.

global/README.md (3)

14-15: Ensure that the provider versions mentioned in the documentation are accurate and up-to-date with the versions used in the codebase.

26-26: The addition of the google_bigquery_dataset.billing_export resource in the documentation is important for users to understand the new cloud cost management capabilities. Ensure that the resource's purpose and configuration are clearly explained.

41-41: The introduction of the enable_cloud_cost_management input variable in the documentation is crucial for controlling the new cloud cost management features. Ensure that the documentation clearly explains its purpose and usage.

global/main.tf (1)

24-32: The addition of the google_bigquery_dataset "billing_export" resource is a key part of enabling cloud cost management features. Ensure that the resource's configuration aligns with the intended purpose and that it is correctly set up for exporting Cloud Billing data to BigQuery.

README.md (1)

27-28: The conditional enabling of the bigquerydatatransfer.googleapis.com and bigquery.googleapis.com APIs based on the enable_cloud_cost_management flag is a thoughtful approach to managing service dependencies. Ensure that the documentation clearly explains these conditions and their impact on the configuration.

Add BigQuery dataset for cloud cost management

06b8db9

brettcurtis self-assigned this Feb 24, 2024

brettcurtis linked an issue Feb 24, 2024 that may be closed by this pull request

Support for enabling Cloud Cost Management #9

Open

brettcurtis marked this pull request as ready for review February 24, 2024 18:10

coderabbitai bot approved these changes Feb 24, 2024

View reviewed changes

brettcurtis merged commit db412d0 into main Feb 24, 2024
6 checks passed

brettcurtis deleted the brettcurtis/issue9 branch February 24, 2024 18:14

brettcurtis mentioned this pull request Feb 27, 2024

Support for enabling Cloud Cost Management #9

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add BigQuery dataset for cloud cost management #13

Add BigQuery dataset for cloud cost management #13

brettcurtis commented Feb 24, 2024 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Feb 24, 2024 •

edited

Loading

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

CodeRabbit Discord Community

infracost bot commented Feb 24, 2024

coderabbitai bot left a comment

Add BigQuery dataset for cloud cost management #13

Add BigQuery dataset for cloud cost management #13

Conversation

brettcurtis commented Feb 24, 2024 • edited by coderabbitai bot Loading

Summary by CodeRabbit

coderabbitai bot commented Feb 24, 2024 • edited Loading

Walkthrough

Changes

Related issues

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

CodeRabbit Discord Community

infracost bot commented Feb 24, 2024

Infracost report

💰 Monthly cost will not change

Governance checks

coderabbitai bot left a comment

Choose a reason for hiding this comment

brettcurtis commented Feb 24, 2024 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Feb 24, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)