[CT-2860] Add IAM Authentication to dbt-postgres

[christopherscholz]

resolves dbt-labs/dbt-postgres#14
docs #3798

Problem

It is not possible to use AWS IAM Authentication with dbt-postgres.

Solution

This adds IAM Authentication to dbt-postgres in order to connect to Amazon RDS Aurora PostgreSQL and Amazon RDS for PostgreSQL.
It does it in the same fashion as dbt-redshift. It is refactoring the connection function by providing it through a connection factory.

The following profile configurations change

• password becomes optional
• method is added: default database; options: database, iam
• iam_profile is added: default None, used to overwrite default AWS IAM profile
• region is added: default None, used to overwrite default AWS Region

If iam is chosen as method, then before connecting to the PostgreSQL db using psycopg2.connect, it would

• open boto3 session
• create rds client
• get the auth token by calling generate_db_auth_token and pass it to kwargs as password

if method is not set or set to database, the current password connection is used

Checklist

• I have read the contributing guide and understand what's expected of me
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.02%. Comparing base (518eb73) to head (81ba4a6).
Report is 377 commits behind head on main.

❗ Current head 81ba4a6 differs from pull request most recent head 8917638. Consider uploading reports for the commit 8917638 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8187      +/-   ##
==========================================
- Coverage   86.22%   83.02%   -3.21%     
==========================================
  Files         174      174              
  Lines       25515    25498      -17     
==========================================
- Hits        22001    21169     -832     
- Misses       3514     4329     +815     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[christopherscholz]

I have fixed the tests. Also tested it against a real Amazon RDS for PostgreSQL using IAM Auth, which worked for me without any issue.

[christopherscholz]

I have changed the Credentials validation flow a bit. My goal was to keep it as restrictive as before without changing too much code.

The classmethod PostgresCredentials.validate (hologram.JsonSchemaMixin.validate) validates a Dict against the class.

I have changed it, so that the dataclass PostgresCredentials is a union of all specific credential methods. But instead of only validate against this one, I have changed the validate classmethod to

1. validate against itself as before
2. if no error is risen then check which authentication method is chosen
3. validate also against the extenden PostgresCredentials dataclass for the specific method

This way password is still required for database authentication and at the same time is forbidden for IAM authentication.

[github-actions]

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

[dbeatty10]

Thanks for taking the time to open this PR @christopherscholz! Since opening, we've decoupled dbt Adapters from dbt Core, and this code now lives in a separate repo: dbt-postgres.

A consequence of the decoupling is that PR can't be merged anymore as is, so we're closing it. For more context see #9171.

The linked issue has already been transferred. Please don't hesitate to re-open your proposed code changes within this PR in the dbt-postgres repo.