Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-454] [Bug] api.Column.string_type causes error #715

Open
2 tasks done
rlh1994 opened this issue Apr 14, 2023 · 5 comments
Open
2 tasks done

[ADAP-454] [Bug] api.Column.string_type causes error #715

rlh1994 opened this issue Apr 14, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@rlh1994
Copy link

rlh1994 commented Apr 14, 2023

Is this a new bug in dbt-spark?

  • I believe this is a new bug in dbt-spark
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Because there is no overwritten function, it uses a the core class, which is character varying which doesn't exist in spark (or databricks), so this api call just errors in spark/databricks targets.

Expected Behavior

Model to pass, likely using string as the type.

Steps To Reproduce

The easiest way to see it is with a basic model that just casts something to the string type:

{{
  config(
    materialized = 'table',
    )
}}

select
    cast(10 as {{api.Column.string_type(4000)}}) as test_col

Which in the case of a databricks target leads to the following compiled code



select
    cast(10 as character varying(4000)) as test_col

and a dbt run gives the error output

dbt run --target databricks
10:05:06  Running with dbt=1.4.5
10:05:06  Unable to do partial parsing because config vars, config profile, or config target have changed
10:05:06  Unable to do partial parsing because a project dependency has been added
10:05:07  Found 1 model, 0 tests, 0 snapshots, 0 analyses, 374 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
10:05:07  
10:08:17  Concurrency: 1 threads (target='databricks')
10:08:17  
10:08:17  1 of 1 START sql table model dbt_ryan.test_models .............................. [RUN]
10:08:17  1 of 1 ERROR creating sql table model dbt_ryan.test_models ..................... [ERROR in 0.64s]
10:08:18  
10:08:18  Finished running 1 table model in 0 hours 3 minutes and 10.96 seconds (190.96s).
10:08:18  
10:08:18  Completed with 1 error and 0 warnings:
10:08:18  
10:08:18  Runtime Error in model test_models (models/test_models.sql)
10:08:18    
10:08:18    [PARSE_SYNTAX_ERROR] Syntax error at or near 'varying': missing ')'.(line 19, pos 25)
10:08:18    
10:08:18    == SQL ==
10:08:18    /* {"app": "dbt", "dbt_version": "1.4.5", "dbt_databricks_version": "1.4.2", "databricks_sql_connector_version": "2.3.0", "profile_name": "default", "target_name": "databricks", "node_id": "model.dbt_demo.test_models"} */
10:08:18    
10:08:18      
10:08:18        
10:08:18            create or replace table `dbt_ryan`.`test_models`
10:08:18          
10:08:18          
10:08:18        using delta
10:08:18          
10:08:18          
10:08:18          
10:08:18          
10:08:18          
10:08:18          
10:08:18          as
10:08:18          
10:08:18    
10:08:18    select
10:08:18        cast(10 as character varying(4000)) as test_col
10:08:18    -------------------------^^^
10:08:18      
10:08:18    
10:08:18  

Relevant log output

No response

Environment

- OS: MacOS Ventura 13.3.1 (22E261)
- Python: 3.9.13
- dbt-core: 1.4.5
- dbt-spark: 1.4.1

Additional Context

Issues in BQ: dbt-labs/dbt-bigquery#665

dbt-core relevant issue: dbt-labs/dbt-adapters#81

@rlh1994 rlh1994 added bug Something isn't working triage labels Apr 14, 2023
@github-actions github-actions bot changed the title [Bug] api.Column.string_type causes error [ADAP-454] [Bug] api.Column.string_type causes error Apr 14, 2023
@dbeatty10
Copy link
Contributor

Thanks @rlh1994 !

@github-actions
Copy link
Contributor

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Oct 12, 2023
@rlh1994
Copy link
Author

rlh1994 commented Oct 12, 2023

That Sign Wont Stop Me Because I Cant Read 6634

@dbeatty10 dbeatty10 removed the Stale label Oct 12, 2023
Copy link
Contributor

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Apr 10, 2024
@rlh1994
Copy link
Author

rlh1994 commented Apr 10, 2024

I Cant Have This Conversation Again 10042024082931

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants