Overhaul technical docs, split in to multiple other docs

* Add operation docs
* Extract "releasing" docs from development docs
* Extract third party service docs from development docs
* Update envvar docs

doc/how-to/development.md

@@ -1,28 +1,67 @@
-# Development
+---
+title: "Development"
+---
 
-## How to run the application for development
+## Quickstart
 
-
-### Symlink dev config
+Install [Docker](https://docs.docker.com/engine/install/) and
+[Docker Compose](https://docs.docker.com/compose/install/).
 
 ```bash
 ln -s compose.dev.yml compose.override.yml
+docker compose up -d
+
+# IMPORTANT: First time only. Initializing DB starts fresh, deleting all entered data:
+./scripts/invoke_in_container.sh db.init
 ```
 
+This will provide a hot-reloading dev server. Use `docker compose logs -f` to watch the
+logs.
+
+
+## The details
+
+It's easiest to work with and support Docker Compose here. Many other workflows are
+great options, but they are not in scope for this documentation.
 
-### Define environment variables
 
-Define the environment variables as specified in [envvars.md](/reference/envvars.md):
+### Install Docker Compose
+
+Install [Docker](https://docs.docker.com/engine/install/) and
+[Docker Compose](https://docs.docker.com/compose/install/).
+
+Docker Desktop provides both of these. There are alternatives, like Podman, but they are
+not in scope for this documentation.
+
+
+### Symlink dev Compose config
+
+Create a symbolic link to the dev Compose config with a magical name.
+`compose.override.yml`, if present is merged on top of `compose.yml`. This gives us a
+dev server instead of a production server, and other conveniences.
 
 ```bash
-export USAON_BENEFIT_TOOL_DB_HOST=...
-export USAON_BENEFIT_TOOL_DB_PORT=...
-export USAON_BENEFIT_TOOL_DB_USER=...
-export USAON_BENEFIT_TOOL_DB_PASSWORD=...
+ln -s compose.dev.yml compose.override.yml
 ```
 
-> :memo: these can be put in a `.env` file so that all variables are assigned. This file
-> is part of `.gitignore` so that no secrets are committed to github.
+
+### Environment variables
+
+Our development Compose file does not require any of
+[our application's environment variables](/references/envvars.md), but if you want to
+test with Google SSO or with a remote database, you may want to uncomment some lines in
+`compose.dev.yml`.
+
+> [!NOTE]
+>
+> Envvar values can be persisted in a `.env` file so they are passed to containers
+> automatically by Docker Compose. This file is part of `.gitignore` so that no secrets
+> are committed to github.
+
+> [!WARNING]
+> **Never put secret** (or non-constant!) **environment variables in Compose YAML,
+> because that makes it easy to accidentally expose those to the internet.**
+
 
 ### Start the service
 
@@ -32,17 +71,29 @@ Bring up the docker container:
 docker compose up -d
 ```
 
+You can follow the logs with:
+
+```bash
+docker compose logs -f
+```
+
 
 ### Initialize the database
 
-> :memo: In dev this will initialize the SQLite DB. In all other environments we deploy
-> the db using the [usaon-vta-db project](https://github.com/nsidc/usaon-vta-db).
+> [!NOTE]
+>
+> In dev this will initialize a PostgreSQL DB that is preserved in the `_db/` directory.
+> In NSIDC deployment environments we deploy the db on a separate host using the
+> [usaon-benefit-tool-db project](https://github.com/nsidc/usaon-benefit-tool-db).
 
 Run `./scripts/invoke_in_container.sh db.init`
 
-This can also be used to drop and recreate all the tables.
+This can also be used to drop and recreate all the tables, for example, after a model
+change.
 
-> :warning: This results in all the data in the database being deleted.
+> [!WARNING]
+>
+> This results in all the data in the database being deleted and starting fresh.
 
 
 ## How to develop
@@ -84,63 +135,9 @@ pre-commit run --all-files
 ```
 
 
-## How to release
-
-### CHANGELOG
-
-Author a new changelog section titled `NEXT_VERSION`. The `bumpversion` step will
-replace this magic string with a correct version identification.
-
-
-### Bump the version
-
-We're currently pre-1.0, so all bumps should look like:
-
-```bash
-bumpversion minor
-```
-
-Post-1.0, it's OK to bump other version parts.
-
-
-### Release
-
-After everything is merged, create a release in the GitHub UI.
-
-Releases that are `<1.0` or labeled `alpha`, `beta`, or `rc` must be marked as
-pre-releases.
-
-After the release is created, GitHub Actions will start building the container images.
-Once GitHub Actions is done, you can deploy the app (`deploy/deploy`).
-
-At NSIDC, this release should be done with Garrison.
-
-
-## Third-party services
-
-### Google SSO
-
-A Google OAuth application is required for login to work. Our app requires a client ID
-and client secret to communicate with the Google OAuth application.
-
-
-### Initial setup
-
-_TODO_
-
-
-### Redirect URIs
-
-> :warning: This configuration needs to be updated for every unique deployment URL of
-> this app.
-
-Our app's deployed URIs also must be registered with Google as ["Authorized redirect
-URIs"](https://console.cloud.google.com/apis/credentials) (from the link, click on your
-app under the "OAuth 2.0 Client IDs" section). The following URIs should be provided for
-local development:
-
-* `http://localhost:5000/google_oauth/google/authorized`
-* `http://127.0.0.1:5000/google_oauth/google/authorized`
+### Third-party services
 
-Add more URIs (substituting the protocol, hostname, and port number, and nothing else)
-for each separate deployment.
+See
+[our documentation on third-party services](/references/third-party-service-dependencies.md)
+for more, but the way they are currently set up should allow for development on
+`localhost`.

doc/how-to/operation.md

@@ -0,0 +1,62 @@
+---
+title: "Operation"
+---
+
+This application is operated by NSIDC. In the NSIDC environment, all of the values
+needed are provisioned on to deployment hosts with Puppet. The secret values are pulled
+from our secrets management system.
+
+
+## Application version
+
+To pull the correct image from DockerHub, the application version must be passed.
+See [our environment variable reference][envvars-doc] for more.
+
+
+## Session security
+
+The application needs a randomized secret key to secure sessions. Generating a new one
+every time would break all sessions every time the app is re-deployed, so we store a
+persistent value and pass it in as an environment variable. This is a setting for Flask
+(`FLASK_SECRET_KEY`), not something we've implemented specifically for this app.
+See [our environment variable reference][envvars-doc] for more.
+
+
+## Proxy switch
+
+When the app is running, for example behind a proxy, at any path other than `/`, it
+needs to be told by request headers (`X-Forwarded-Prefix`) what that path is. To enable
+that behavior, set `USAON_BENEFIT_TOOL_PROXY` envvar to `true`.
+See [our environment variable reference][envvars-doc] for more.
+
+
+## TLS certificate
+
+The application expects a TLS certificate to be provided to it from the host. Our
+Compose YAML uses environment variables to find these files on the host.
+See [our environment variable reference][envvars-doc] for more.
+
+These files are passed in to the container as
+[secrets](https://docs.docker.com/compose/use-secrets/).
+
+> [!NOTE]
+>
+> If they expire and need to be regenerated, the application may need to be restarted to
+> pick up the change.
+
+
+## Single Sign On (SSO)
+
+Currently, we only support Google as an SSO provider. We pass in the needed values as
+environment variables.
+See [our environment variable reference][envvars-doc] for more.
+
+
+## Database connectivity
+
+This application expects a PostgreSQL database to be running. The connection
+information, including credentials, are passed in as environment variables.
+See [our environment variable reference][envvars-doc] for more.
+
+
+[envvars-doc]: /reference/envvars.md

doc/how-to/release.md

@@ -0,0 +1,51 @@
+---
+title: "Releasing"
+---
+
+"Releasing" means sharing a completed code package to the world. It does not mean
+running that code package! That would be "deployment", and can look different depending
+on who is doing the deployment on on to what system.
+
+
+## CHANGELOG
+
+Author a new changelog section titled `NEXT_VERSION`. The `bump-my-version` step will
+replace this magic string with a correct version identification.
+
+
+## Bump the version
+
+This application users [semver](https://semver.org).
+
+The version number must be updated in many places, so we use `bump-my-version` to
+automate the process. To increase the minor version number:
+
+```bash
+bump-my-version bump minor
+```
+
+
+## Release
+
+After everything is merged, create a release in the GitHub UI.
+
+Releases that are labeled `alpha`, `beta`, or `rc` must be marked as
+pre-releases.
+
+After the release is created, GitHub Actions will start building the container images.
+Once GitHub Actions is done, you can deploy the app with the deploy script
+(`deploy/deploy`).
+
+> [!NOTE]
+>
+> At NSIDC, this deployment should be done with _Garrison_ after automations have
+> successfully completed.
+
+
+### Automations
+
+This will trigger [GitHub Actions](https://github.com/nsidc/usaon-benefit-tool/actions)
+to release new container images to Docker Hub and GitHub Packages. It should take at
+most 5-10 minutes.
+
+The ReadTheDocs `stable` page will also automatically update.

doc/reference/envvars.md

@@ -1,25 +1,47 @@
-# Environment variables
+---
+title: "Environment variables"
+---
+
+
+## Deployment
+
+* `USAON_BENEFIT_TOOL_VERSION`: Which tag on Docker Hub to deploy.
+
 
 ## Web
 
-* `FLASK_SECRET_KEY` (required): For securely signing session cookie and other security
+* `FLASK_SECRET_KEY`: For securely signing session cookie and other security
   needs ([doc](https://flask.palletsprojects.com/en/2.2.x/config/#SECRET_KEY)).
-* `USAON_BENEFIT_TOOL_PROXY`: True if we are deploying with a proxy to pass the header `X-Forwarded-Prefix`.
+* `USAON_BENEFIT_TOOL_PROXY`: "True" if we are deploying with a proxy to pass the header
+  `X-Forwarded-Prefix`.
+
+
+### Transport Level Security (TLS)
+
+* `TLS_CERT_FILE`: Path to `.crt` file.
+* `TLS_KEY_FILE`: Path to `.key` file.
 
 
 ## SSO
 
-* `OAUTHLIB_INSECURE_TRANSPORT=1`  NOTE: Do not set this in production.
-* `OAUTHLIB_RELAX_TOKEN_SCOPE=1` Set in Compose YAML.
-* `USAON_BENEFIT_TOOL_GOOGLE_CLIENT_ID`: The google credential id. (stored in vault)
-* `USAON_BENEFIT_TOOL_GOOGLE_CLIENT_SECRET`: The google sso credential secret. (stored in vault)
+* `USAON_BENEFIT_TOOL_GOOGLE_CLIENT_ID`: The google credential id.
+* `USAON_BENEFIT_TOOL_GOOGLE_CLIENT_SECRET`: The google sso credential secret.
+* `OAUTHLIB_RELAX_TOKEN_SCOPE=1`: Should always be set; hardcoded in Compose YAML.
+
+
+### Development only
+
+> [!IMPORTANT]
+>
+> **Do not set in production.**
+
+* `OAUTHLIB_INSECURE_TRANSPORT=1`: Enable SSO when developing without HTTPS.
 
 
 ## Database
 
-All of the below environment variables are required.
-* `USAON_BENEFIT_TOOL_DB_HOST`: The host on which the database is running (NOTE: Only postgres supported)
-* `USAON_BENEFIT_TOOL_DB_PORT`: The access port for the database
+* `USAON_BENEFIT_TOOL_DB_HOST`: The host on which the database is running.
+* `USAON_BENEFIT_TOOL_DB_PORT`: The access port for the database.
 * `USAON_BENEFIT_TOOL_DB_USER`: Username with permissions to create tables, delete tables, insert, update,
-    delete
-* `USAON_BENEFIT_TOOL_DB_PASSWORD`: Password for the above user
+    delete.
+* `USAON_BENEFIT_TOOL_DB_PASSWORD`: Password for the above user.

doc/reference/third-party-service-dependencies.md

@@ -0,0 +1,30 @@
+---
+title: "Third party services"
+---
+
+## Google SSO
+
+A Google OAuth application is required for login to work. Our app requires a client ID
+and client secret to communicate with the Google OAuth application.
+
+
+### Initial setup
+
+_TODO_
+
+
+### Redirect URIs
+
+> :warning: This configuration needs to be updated for every unique deployment URL of
+> this app.
+
+Our app's deployed URIs also must be registered with Google as ["Authorized redirect
+URIs"](https://console.cloud.google.com/apis/credentials) (from the link, click on your
+app under the "OAuth 2.0 Client IDs" section). The following URIs should be provided for
+local development:
+
+* `http://localhost:5000/google_oauth/google/authorized`
+* `http://127.0.0.1:5000/google_oauth/google/authorized`
+
+Add more URIs (substituting the protocol, hostname, and port number, and nothing else)
+for each separate deployment.