Add OAuth2 authentication (57)

[jwhitlock]

Proposed changes

To fix issue #57, add OAuth2 authentication using client credentials.

A new endpoint /token implements the OAuth2 client credentials grant process. A caller passes a client_id and a client_secret, either in the form body or a header, and gets an OAuth2 token that expires. The token is implemented as a JSON Web Token or JWT, signed by the server (HMAC with SHA-256). When the token expires (default of 60 minutes), the client calls /token again.

All API endpoints now require a bearer token. The Swagger API docs already include a mechanism for authenticating in the browser.

There's a new database table api_client that stores the details for the API clients, along with an alembic migration. A script ctms/bin/client_credentials.py can create these. See below for an example.

There are new application settings:

• CTMS_SECRET_KEY - A long string used in JWT signing. There is a default for development, and it should be unique for each deployment.
• CTMS_TOKEN_EXPIRATION - How long, in seconds, that access tokens are valid. Default is 60 minutes.
• CTMS_SERVER_PREFIX - The prefix of the server URL, defaults to the dev value of http://localhost:8000. Currently used in the output of client_credentials.py, but it may be handy for CSP or other uses.

Types of changes

What types of changes does your code introduce?

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

• I have read the guides
• I have followed the Mozilla Lean Data Policies
• Lint and tests pass both locally and on the cicd with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• N/A Any dependent changes have been merged and published in downstream modules

Further comments

Sorry for the size of this PR. It is hard to do just a part of a security feature.

Here's how to use client_credentials.py in a local development environment:

make shell             # Enter a docker shell
poetry install         # Get ctms into Python path
alembic upgrade head   # Add the api_client table
ctms/bin/client_credentials.py test --email test@example.com

This will print something like:

Your OAuth2 client credentials are:

      client_id: id_test
  client_secret: secret_gPwcXQLSOXSYfpKxzjA5DBSQLDNL__MMkBBm2auWyxI

You can generate a token with an Authentication header:

  curl --user id_test:secret_gPwcXQLSOXSYfpKxzjA5DBSQLDNL__MMkBBm2auWyxI -F grant_type=client_credentials http://localhost:8000/token

or passing credentials in the form body:

  curl -v -F client_id=id_test -F client_secret=secret_gPwcXQLSOXSYfpKxzjA5DBSQLDNL__MMkBBm2auWyxI -F grant_type=client_credentials http://localhost:8000/token

The JSON response will have an access token, such as:

  {
    "access_token": "a_very_long_base64_string",
    "token_type": "bearer",
    "expires_in": 3600
  }

This can be used to access the API, such as:

  curl --oauth2-bearer a_very_long_base64_string http://localhost:8000/ctms?primary_email=test@example.com

You can then start the webserver (make start in a different terminal) and enter the credentials into http://localhost:8000/docs. The "Authorize" button in the Swagger docs is over by the right side:

You need to authorize before you can use the API endpoints, as hinted by the lock icons.

[jwhitlock]

Force-push to rebase on main and the package update PR

[jwhitlock]

@bsieber-mozilla it sounds like you have a database that was populated before we added alembic. If you don't care about the contents, run make setup to re-initialize your database. If you do care, then alembic stamp 9a91e36e6e6f might do the right thing (create the revision table and say that you already applied the initial commit)

[bsieber-mozilla]

I just killed all running docker containers to try; still was having errors, re-cloned, killed all docker containers and I think I'm now in business, albeit noticed this on setup...

docker-compose exec postgres bash -c 'while !</dev/tcp/postgres/5432; do sleep 1; done'
bash: connect: Connection refused
bash: /dev/tcp/postgres/5432: Connection refused
docker-compose exec postgres dropdb postgres --user postgres

Will get back to this after standup

[bsieber-mozilla]

• alembic upgrade head

• Results: SUCCESS
  INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
  INFO [alembic.runtime.migration] Will assume transactional DDL.

• root@774de875c06d:/app# ctms/bin/client_credentials.py test --email test@example.com

• Results: SUCCESS

---

• make start
• Results: Initially a failure, required a make build, then make start
• Also installed alembic and passlib to root, since both were complaining: "module not found" not sure if it was an order of operations or my operator err.

[bsieber-mozilla]

Screenshot for post-authorize in swagger

[bsieber-mozilla]

Screenshot of test creation post-authorization

Screenshot of test retrieval post-authorization

[bsieber-mozilla]

Curl test:

| =>   curl --user id_test:secret_FzYSmBbV3g1slWUsMdVrd2F5Xs0hiQcwscahN6UHIck -F grant_type=client_credentials http://localhost:8000/token
{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhcGlfY2xpZW50OmlkX3Rlc3QiLCJleHAiOjE2MTU0MDk5NDJ9.FXg2D_Gm8KREFusEByogxliNKIB9ASXivnRel91rgDs","token_type":"bearer","expires_in":3600}__

Ephemeral DB on local, so I imagine this is fine and not needed to be scrubbed.

Curl 2 test (form body):

| =>   curl -v -F client_id=id_test -F client_secret=secret_FzYSmBbV3g1slWUsMdVrd2F5Xs0hiQcwscahN6UHIck -F grant_type=client_credentials http://localhost:8000/token
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
> POST /token HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 420
> Content-Type: multipart/form-data; boundary=------------------------eb58970bd4bd0aa7
> 
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< date: Wed, 10 Mar 2021 20:01:04 GMT
< server: uvicorn
< content-length: 200
< content-type: application/json
< 
* Connection #0 to host localhost left intact
{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhcGlfY2xpZW50OmlkX3Rlc3QiLCJleHAiOjE2MTU0MTAwNjV9.DpnjQI78PUx4Ng3UnAGxX9zBgirUdFIbL8C1O4M2xaE","token_type":"bearer","expires_in":3600}* Closing connection 0
________________________________________________________________________________

[bsieber-mozilla]

I had a hiccup with my system, but don't believe it would be universal; it all appears to be running and functioning as expected.

LGTM, thanks for these efforts

[bsieber-mozilla]

Thanks for this documentation!

[jwhitlock]

Rebased on current main, no functional changes