Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-384] [Bug] Negative part_number for dbt.split_part() is not working #615

Closed
3 tasks done
Tracked by #7916
dave-connors-3 opened this issue Mar 17, 2023 · 1 comment · Fixed by #618
Closed
3 tasks done
Tracked by #7916

[ADAP-384] [Bug] Negative part_number for dbt.split_part() is not working #615

dave-connors-3 opened this issue Mar 17, 2023 · 1 comment · Fixed by #618
Assignees
Labels
bug Something isn't working

Comments

@dave-connors-3
Copy link
Contributor

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature

per the docs, the split part macro supports negative part_number arguments to select elements from the end of the resulting array.

the current implementation in dbt-bigquery checks for negative values for this argument, and if the arg is negative, just disregards it wholesale!

{% macro bigquery__split_part(string_text, delimiter_text, part_number) %}

  {% if part_number >= 0 %}
    split(
        {{ string_text }},
        {{ delimiter_text }}
        )[safe_offset({{ part_number - 1 }})]
  {% else %}
    split(
        {{ string_text }},
        {{ delimiter_text }}
        )[safe_offset(
          length({{ string_text }})
          - length(
              replace({{ string_text }},  {{ delimiter_text }}, '')
          ) + 1
        )]
  {% endif %}

{% endmacro %}

We should update the logic to accept negative args and handle them appropriately.

Describe alternatives you've considered

keep on keepin on

Who will this benefit?

No response

Are you interested in contributing this feature?

yes!

Anything else?

No response

@dave-connors-3 dave-connors-3 added enhancement New feature or request triage labels Mar 17, 2023
@github-actions github-actions bot changed the title [Feature] support negative part_number for dbt.split_part() [ADAP-384] [Feature] support negative part_number for dbt.split_part() Mar 17, 2023
@dbeatty10
Copy link
Contributor

Good call @dave-connors-3

Looking at the postgres implementation, it is explicitly supporting the case of a negative part_number, so it makes sense for dbt-bigquery to match it:
postgres/macros/utils/split_part.sql

The first action to implement this will be adding a negative test case (e.g., -1 or -2) to this file in dbt-core:
tests/adapter/dbt/tests/adapter/utils/fixture_split_part.py

After opening a PR for dbt-core, then each adapter should be tested against that branch. We'd expect dbt-bigquery to fail (until you implement it). If other adapters fail, we'll want to open PRs for those as well.

Overall, one workflow you could use looks something like this:

  1. Open PR for dbt-core
  2. Open PRs for each of the adapters (or merely local branches)
  3. Temporarily update the dev-requirements.txt file within each adapter to point to your PR in Step 1, like this.
  4. Once all PRs are passing CI, reviewed, and approved, then merge the dbt-core PR
  5. Undo the temporary update to dev-requirements.txt in each of the adapter PRs (so that they point at the main branch of dbt-core again) like this
  6. Once CI passes, then all the adapter PRs can be merged as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants