dbt-spark 0.15.3

This release contains a wide array of features, fixes, and quality-of-life improvements. It brings the Spark plugin closer to parity with core dbt functionality. It tracks dbt==0.15.3.

Features

• Add a merge strategy for incremental models stored in the Delta file format (#65)
• Use create or replace view for atomic replacement of models materialized as views (#65)
• Include object owner and table statistics in auto-generated docs site (#39, #41)
• Add location, clustered_by, and persist_docs as model configs (#43)

Fixes

• Reimplement get_relation to support dropping and recreating objects with custom schema/database (#52)
• Insert columns in same order as existing table for insert_overwrite incremental strategy (#60)

Quality of life

• Add docker-compose environment for containerized local Spark. Reimplement integration tests to run in docker (#58, #62, #64)
• Add support for creating and dropping target schema/database (#40)
• Faster metadata for multiple relations using show table extended in [database] like '*' (#50, #54)
• Add an organization config to support Azure Databricks (#34)
• Allow overriding hard-coded Spark configs with a pre-hook in incremental models (#37)
• Clearer requirements and instructions for pip installation (#44)

Under the hood

• Replace JSON Schemas with data classes (#65)
• Specify upper bound for jinja2 (#56)

Contributors:

• @aaronsteers (#44, #56)
• @bfil (#37)
• @Dandandan (#54)
• @dmateusp (#58, #62, #64)
• @Fokko (#39, #40, #41)
• @NielsZeilemaker (#43)
• @poidra02 (#34, #51)
• @SamKosky (#65)

Thank you to all members of the dbt + Spark community for your input, insight, and assistance with testing these changes.