Skip to content

Releases: dbt-labs/dbt-spark

dbt-spark v1.9.0b1

17 Oct 22:06
4863bdd
Compare
Choose a tag to compare
dbt-spark v1.9.0b1 Pre-release
Pre-release

dbt-spark 1.9.0-b1 - October 01, 2024

Features

  • Add tests for cross-database cast macro (#1028)
  • Cross-database date macro (#1031)
  • Allow configuring snapshot column names (#1096)
  • Support custom ODBC connection parameters via connection_string_suffix config (#1092)
  • Add Microbatch Strategy to dbt-spark (#1109)

Fixes

  • Fix incremental python models error where Databricks could not find the temp table transaction logs (#1033)

Under the Hood

  • Lazy load agate to improve performance (#1049)

Dependencies

  • Update freezegun requirement from ~=1.3 to ~=1.4 (#966)
  • Bump actions/download-artifact from 3 to 4 (#1010)
  • Bump actions/upload-artifact from 3 to 4 (#1011)
  • Bump dbt-labs/actions from 1.1.0 to 1.1.1 (#1012)
  • Update wheel requirement from ~=0.42 to ~=0.43 (#1035)
  • Update pytest-xdist requirement from ~=3.5 to ~=3.6 (#1073)
  • Loosen pin on pre-commit from 3.7.0 to 3.7 (#1074)

Contributors

dbt-spark v1.8.0

09 May 19:54
34451f4
Compare
Choose a tag to compare

dbt-spark 1.8.0 - May 09, 2024

Features

  • Implement spark__safe_cast and add functional tests for unit testing (#987)
  • : Add new workflow for internal patch releases (#38)

Fixes

  • Support new agate Integer type and empty seed test (#935)
  • Fix hardcoded file format for python models (#803)

Under the Hood

  • Add GitHub action for integration testing and use dagger-io to run tests. Remove CircleCI workflow. (#719)
  • Add tests for --empty flag (#949)
  • Remove unused invalid_insert_overwrite_delta_msg message (#962)
  • Update import paths and list_relations to support decoupling adapters/core (#972)
  • Update dependabot config to cover GHA and Docker (#1006)

Dependencies

  • Update pre-commit-hooks requirement from ~=4.4 to ~=4.5 (#903)
  • Bump mypy from 1.5.1 to 1.6.0 (#904)
  • Update pyodbc requirement from ~=4.0.39 to ~=5.0.0 (#905)
  • Update pre-commit requirement from ~=3.4 to ~=3.5 (#914)
  • Update pyodbc requirement from ~=5.0.0 to ~=5.0.1 (#925)
  • Bump mypy from 1.6.0 to 1.6.1 (#926)
  • Update black requirement from ~=23.9 to ~=23.10 (#927)
  • Update black requirement from ~=23.10 to ~=23.11 (#942)
  • Bump mypy from 1.6.1 to 1.7.0 (#946)
  • Update pytest-xdist requirement from ~=3.3 to ~=3.4 (#947)
  • Update pytest-xdist requirement from ~=3.4 to ~=3.5 (#951)
  • Update wheel requirement from ~=0.41 to ~=0.42 (#952)
  • Bump mypy from 1.7.0 to 1.7.1 (#953)
  • Update freezegun requirement from ~=1.2 to ~=1.3 (#956)
  • Update black requirement from ~=23.11 to ~=23.12 (#959)
  • Add dbt-core as a dependency to preserve backwards compatibility for installation (#1002)
  • Bump actions/setup-python from 4 to 5 (#1013)
  • Bump actions/checkout from 3 to 4 (#1014)

Security

  • Pin black>=24.3 in dev-requirements.txt (#1000)

Contributors

dbt-spark v1.8.0rc1

06 May 23:12
fbfd879
Compare
Choose a tag to compare
dbt-spark v1.8.0rc1 Pre-release
Pre-release

dbt-spark 1.8.0-rc1 - May 06, 2024

Under the Hood

  • Update dependabot config to cover GHA and Docker (#1006)

Dependencies

  • Bump actions/setup-python from 4 to 5 (#1013)
  • Bump actions/checkout from 3 to 4 (#1014)

dbt-spark v1.8.0b1

02 Mar 03:15
a2c4871
Compare
Choose a tag to compare
dbt-spark v1.8.0b1 Pre-release
Pre-release

dbt-spark 1.8.0-b1 - March 01, 2024

Features

  • Implement spark__safe_cast and add functional tests for unit testing (#987)

Fixes

  • Support new agate Integer type and empty seed test (#935)
  • Fix hardcoded file format for python models (#803)

Under the Hood

  • Add GitHub action for integration testing and use dagger-io to run tests. Remove CircleCI workflow. (#719)
  • Add tests for --empty flag (#949)
  • Remove unused invalid_insert_overwrite_delta_msg message (#962)
  • Update import paths and list_relations to support decoupling adapters/core (#972)

Dependencies

  • Update pre-commit-hooks requirement from ~=4.4 to ~=4.5 (#903)
  • Bump mypy from 1.5.1 to 1.6.0 (#904)
  • Update pyodbc requirement from ~=4.0.39 to ~=5.0.0 (#905)
  • Update pre-commit requirement from ~=3.4 to ~=3.5 (#914)
  • Update pyodbc requirement from ~=5.0.0 to ~=5.0.1 (#925)
  • Bump mypy from 1.6.0 to 1.6.1 (#926)
  • Update black requirement from ~=23.9 to ~=23.10 (#927)
  • Update black requirement from ~=23.10 to ~=23.11 (#942)
  • Bump mypy from 1.6.1 to 1.7.0 (#946)
  • Update pytest-xdist requirement from ~=3.3 to ~=3.4 (#947)
  • Update pytest-xdist requirement from ~=3.4 to ~=3.5 (#951)
  • Update wheel requirement from ~=0.41 to ~=0.42 (#952)
  • Bump mypy from 1.7.0 to 1.7.1 (#953)
  • Update freezegun requirement from ~=1.2 to ~=1.3 (#956)
  • Update black requirement from ~=23.11 to ~=23.12 (#959)

Contributors

dbt-spark v1.6.2

13 Dec 23:18
da81499
Compare
Choose a tag to compare

dbt-spark 1.6.2 - December 13, 2023

Dependencies

  • Replace sasl with pure-sasl for PyHive (#818)

dbt-spark v1.5.3

13 Dec 23:17
dacb520
Compare
Choose a tag to compare

dbt-spark 1.5.3 - December 13, 2023

Features

  • Persist Column level comments when creating views (#372)

Dependencies

  • Replace sasl with pure-sasl for PyHive (#818)

Contributors

dbt-spark v1.6.1

09 Nov 01:41
6fc3719
Compare
Choose a tag to compare

dbt-spark 1.6.1 - November 09, 2023

Features

  • Persist Column level comments when creating views (#372)

Fixes

  • Support new agate Integer type and empty seed test (#935)

Contributors

dbt-spark v1.7.1

08 Nov 00:34
db8d1f0
Compare
Choose a tag to compare

dbt-spark 1.7.1 - November 07, 2023

Fixes

  • Support new agate Integer type and empty seed test (#935)

dbt-spark v1.7.0

02 Nov 18:12
7d0f1fb
Compare
Choose a tag to compare

dbt-spark 1.7.0 - November 02, 2023

Features

  • Support server side parameters in thrift connection (#387)
  • Support server_side_parameters for Spark session connection method (#690)
  • Add server_side_parameters to HTTP connection method (#824)
  • Enforce server side parameters keys and values to be strings (#826)
  • Add SessionConnectionWrapper (#829)
  • Persist Column level comments when creating views (#372)
  • Support storing test failures as views (#6914)
  • Create temporary views with 'or replace' (#350)

Fixes

  • Wrap AnalysisException into DbtRuntimeError (#782)
  • include tblproperties macro in adapters.sql create table (#865)

Under the Hood

  • Update stale workflow to use centralized version (#842)
  • Remove dependency on hologram (#881)

Dependencies

  • Update tox requirement from ~=3.0 to ~=4.5 (#741)
  • Update pyodbc requirement from ~=4.0.30 to ~=4.0.39 (#742)
  • Update pre-commit requirement from ~=2.21 to ~=3.3 (#748)
  • Update types-requests requirement from ~=2.28 to ~=2.29 (#749)
  • Bump mypy from 1.2.0 to 1.3.0 (#768)
  • Replace sasl with pure-sasl for PyHive (#818)
  • Update flake8 requirement from ~=6.0 to ~=6.1 (#849)
  • Update types-requests requirement from ~=2.29 to ~=2.31 (#850)
  • Update pytest-xdist requirement from ~=3.2 to ~=3.3 (#851)
  • Update pytest requirement from ~=7.3 to ~=7.4 (#852)
  • Update pip-tools requirement from ~=6.13 to ~=7.2 (#856)
  • Update black requirement from ~=23.3 to ~=23.7 (#857)
  • Update wheel requirement from ~=0.40 to ~=0.41 (#858)
  • Update mock requirement from ~=5.0 to ~=5.1 (#859)
  • Bump mypy from 1.3.0 to 1.4.1 (#860)
  • Update tox requirement from ~=4.5 to ~=4.6 (#861)
  • Update pip-tools requirement from ~=7.2 to ~=7.3 (#863)
  • Update tox requirement from ~=4.6 to ~=4.7 (#867)
  • Update tox requirement from ~=4.7 to ~=4.8 (#871)
  • Bump mypy from 1.4.1 to 1.5.0 (#872)
  • Update tox requirement from ~=4.8 to ~=4.9 (#874)
  • Bump mypy from 1.5.0 to 1.5.1 (#875)
  • Update tox requirement from ~=4.9 to ~=4.10 (#879)
  • Update pre-commit requirement from ~=3.3 to ~=3.4 (#884)
  • Update black requirement from ~=23.7 to ~=23.9 (#886)
  • Update tox requirement from ~=4.10 to ~=4.11 (#887)

Security

  • Add docker image to the repo (#876)

Contributors

dbt-spark v1.7.0rc1

12 Oct 17:20
db0156e
Compare
Choose a tag to compare
dbt-spark v1.7.0rc1 Pre-release
Pre-release

dbt-spark 1.7.0-rc1 - October 12, 2023

Features

  • Support storing test failures as views (#6914)
  • Create temporary views with 'or replace' (#350)

Contributors