Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Update collect_freshness config in src_apple_store.yml to account for dbt macro logic #12

Closed
2 of 4 tasks
fivetran-avinash opened this issue Oct 11, 2023 · 0 comments · Fixed by #14
Closed
2 of 4 tasks
Assignees
Labels
priority:p4 Affects few users; pick up when available type:enhancement New functionality or enhancement update_type:testing Primary focus requires testing updates

Comments

@fivetran-avinash
Copy link
Contributor

fivetran-avinash commented Oct 11, 2023

Is there an existing feature request for this?

  • I have searched the existing issues

Describe the Feature

Our fivetran_utils collect_freshness macro was designed to override dbt's equivalent macro to allow for the enabling/disabling of variables when running dbt source snapshot-freshness. Thus if a source table does not exist, dbt will not run (and error on) a freshness test on the table.

dbt has since updated its macro in recent versions to basically provide the same functionality. So warnings are being thrown to upgrade the package logic.

We will want to modify all versions of the meta: is_enabled config in our src_apple_store.yml package ([example here])(https://github.com/fivetran/dbt_apple_store_source/blob/main/models/src_apple_store.yml#L188-L189) to match the config: enabled dbt logic. Once those dependencies are either updated or removed, we can further explore updates to our own internal collect_freshness macro.

Describe alternatives you've considered

At the moment, we will be proceeding forward with updating dbt_fivetran_utils to match the logic of dbt's collect_freshness macro. That way customers will no longer experience that warning.

More details can be found in this ticket.

Are you interested in contributing this feature?

  • Yes.
  • Yes, but I will need assistance and will schedule time during your office hours for guidance.
  • No.

Anything else?

Customers raising this issue can be found in this dbt slack thread https://getdbt.slack.com/archives/C03SAHKKG2Z/p1683127164993329

@fivetran-avinash fivetran-avinash added type:enhancement New functionality or enhancement priority:p4 Affects few users; pick up when available update_type:testing Primary focus requires testing updates labels Oct 11, 2023
@fivetran-catfritz fivetran-catfritz self-assigned this Jul 17, 2024
@fivetran-catfritz fivetran-catfritz linked a pull request Jul 17, 2024 that will close this issue
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:p4 Affects few users; pick up when available type:enhancement New functionality or enhancement update_type:testing Primary focus requires testing updates
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants