Patch for `dbt show` for a model with a `struct` column containing a `json` datatype #974

matt-winkler · 2023-10-21T21:48:36Z

resolves #972

Problem

dbt show --select currently breaks when attempted on a json field because of this bigquery python client issue

Solution

Patch as suggested by @dbeatty10

# Override broken json deserializer for dbt show --inline
def _json_from_json(value, _):
    """NOOP string -> string coercion"""
    return json.loads(value)

bigquery._helpers._CELLDATA_FROM_JSON["JSON"] = _json_from_json

Checklist

I have read the contributing guide and understand what's expected of me
I have run this code in development and it appears to resolve the stated issue
This PR includes tests, or tests are not required/relevant for this PR
This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc)

dbeatty10 · 2023-10-23T14:16:00Z

@cla-bot check

cla-bot · 2023-10-23T14:16:06Z

The cla-bot has been summoned, and re-checked this pull request!

.changes/unreleased/Fixes-20231021-154237.yaml

dbt/adapters/bigquery/connections.py

colin-rogers-dbt · 2023-10-23T16:22:53Z

.changes/unreleased/Fixes-20231021-154237.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Patch for json inline --show


Looks like you generated a second changelog instead of updating this one, you can just delete this file

Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>

tests/functional/adapter/dbt_show/test_dbt_show.py

mikealfare

I left one minor comment that doesn't affect functionality. I'm providing an approval so that you can merge following that update.

…winkler/json-show-patch

dbeatty10 · 2023-10-23T17:35:53Z

.changes/unreleased/Fixes-20231023-082312.yaml

@@ -0,0 +1,6 @@
+kind: Fixes
+body: Patch for json inline --show


Suggested change

body: Patch for json inline --show

body: Patch for `dbt show` for `json` within a `struct`

…`json` datatype (#974) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458)

…`json` datatype (#974) (#976) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458) Co-authored-by: matt-winkler <75497565+matt-winkler@users.noreply.github.com>

…`json` datatype (#974) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458)

…`json` datatype (#974) (#978) * patch for --show with json * update matts handle in changelog * Update dbt/adapters/bigquery/connections.py Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> * add json struct functional test * remove old change log * add comment to test --------- Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com> Co-authored-by: Colin <colin.rogers@dbtlabs.com> (cherry picked from commit 6a3f458) Co-authored-by: matt-winkler <75497565+matt-winkler@users.noreply.github.com>

patch for --show with json

9b93327

matt-winkler requested a review from a team as a code owner October 21, 2023 21:48

matt-winkler requested a review from nathaniel-may October 21, 2023 21:48

cla-bot bot added the cla:yes label Oct 21, 2023

dbeatty10 added the ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering label Oct 23, 2023

dbeatty10 reviewed Oct 23, 2023

View reviewed changes

.changes/unreleased/Fixes-20231021-154237.yaml Outdated Show resolved Hide resolved

update matts handle in changelog

a5c9b8c

colin-rogers-dbt reviewed Oct 23, 2023

View reviewed changes

dbt/adapters/bigquery/connections.py Show resolved Hide resolved

colin-rogers-dbt reviewed Oct 23, 2023

View reviewed changes

matt-winkler and others added 3 commits October 23, 2023 10:27

Update dbt/adapters/bigquery/connections.py

e49cfc0

Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>

add json struct functional test

a32baf1

remove old change log

30f817e

mikealfare reviewed Oct 23, 2023

View reviewed changes

tests/functional/adapter/dbt_show/test_dbt_show.py Show resolved Hide resolved

mikealfare approved these changes Oct 23, 2023

View reviewed changes

dbeatty10 changed the title ~~patch for --show with json~~ Patch for dbt show for a model with a json column Oct 23, 2023

colin-rogers-dbt added 2 commits October 23, 2023 10:33

add comment to test

b01397e

Merge remote-tracking branch 'origin/mwinkler/json-show-patch' into m…

8b6bd15

…winkler/json-show-patch

dbeatty10 changed the title ~~Patch for dbt show for a model with a json column~~ Patch for dbt show for a model with a struct column containing a json datatype Oct 23, 2023

colin-rogers-dbt enabled auto-merge (squash) October 23, 2023 17:34

colin-rogers-dbt merged commit 6a3f458 into main Oct 23, 2023
11 checks passed

colin-rogers-dbt deleted the mwinkler/json-show-patch branch October 23, 2023 17:35

dbeatty10 reviewed Oct 23, 2023

View reviewed changes

colin-rogers-dbt added the backport 1.6.latest label Oct 23, 2023

github-actions bot mentioned this pull request Oct 23, 2023

[Backport 1.6.latest] Patch for dbt show for a model with a struct column containing a json datatype #976

Merged

colin-rogers-dbt added the backport 1.7.latest label Oct 24, 2023

github-actions bot mentioned this pull request Oct 24, 2023

[Backport 1.7.latest] Patch for dbt show for a model with a struct column containing a json datatype #978

Merged

dbeatty10 mentioned this pull request Feb 16, 2024

[Bug] Error when loading NULL result from JSON column #1101

Closed

2 tasks

colin-rogers-dbt mentioned this pull request Apr 24, 2024

backport 1187 to 1.6.latest #1193

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype #974

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype #974

matt-winkler commented Oct 21, 2023 •

edited by dbeatty10

Loading

dbeatty10 commented Oct 23, 2023

cla-bot bot commented Oct 23, 2023

colin-rogers-dbt Oct 23, 2023

mikealfare left a comment

dbeatty10 Oct 23, 2023

		@@ -0,0 +1,6 @@
		kind: Fixes
		body: Patch for json inline --show

	body: Patch for json inline --show
	body: Patch for `dbt show` for `json` within a `struct`

Patch for dbt show for a model with a struct column containing a json datatype #974

Patch for dbt show for a model with a struct column containing a json datatype #974

Conversation

matt-winkler commented Oct 21, 2023 • edited by dbeatty10 Loading

Problem

Solution

Checklist

dbeatty10 commented Oct 23, 2023

cla-bot bot commented Oct 23, 2023

colin-rogers-dbt Oct 23, 2023

Choose a reason for hiding this comment

mikealfare left a comment

Choose a reason for hiding this comment

dbeatty10 Oct 23, 2023

Choose a reason for hiding this comment

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype #974

Patch for `dbt show` for a model with a `struct` column containing a `json` datatype #974

matt-winkler commented Oct 21, 2023 •

edited by dbeatty10

Loading