Merge pull request #4272 from open-formulieren/feature/4246-refactor-…

…return-flow

Rework OpenID Connect integration

.github/ISSUE_TEMPLATE/prepare-release.md

@@ -8,17 +8,32 @@ assignees: sergei-maertens
 
 - [ ] Resolve release blockers
   - [ ] ...
-- [ ] Re-generate VCR cassettes for API tests (see instructions on Taiga)
+- [ ] Re-generate VCR cassettes for API tests (see instructions on Taiga). You can find all test
+      cases with `grep OFVCRMixin -r src`
+
   - [ ] Appoinments: Qmatic
   - [ ] Suwinet
-  - [ ] DigiD/eHerkenning (Signicat)
+  - DigiD/eHerkenning (Signicat)
+
+    - [ ] `openforms.authentication.contrib.digid.tests.test_signicat_integration`
+    - [ ] `openforms.authentication.contrib.eherkenning.tests.test_signicat_integration`
+
+  - OIDC based authentication flows
+
+    - [ ] `openforms.authentication.tests.test_oidc`
+    - [ ] `openforms.authentication.contrib.digid_eherkenning_oidc`
+    - [ ] `openforms.authentication.contrib.org_oidc`
+    - [ ] `openforms.authentication.contrib.org_oidc`
+    - [ ] `openforms.tests.test_registrator_prefill`
+
   - [ ] Haal Centraal BRP Personen bevragen
-  - [ ] `src.soap.tests.test_client`
+  - [ ] `soap.tests.test_client`
   - [ ] BRK (Kadaster)
   - [ ] KVK
   - [ ] Ogone
   - [ ] Objects API (registration)
   - [ ] ZGW APIs (registration)
+
 - [ ] Release new SDK version
 - [ ] Correct SDK version pinned in `.sdk-release`
 - [ ] Check translations

CHANGELOG.rst

@@ -2,6 +2,39 @@
 Changelog
 =========
 
+2.7.0 "TBD" (2024-07-??)
+========================
+
+This release is in development, meaning it is not finished yet or suitable for
+production use.
+
+Upgrade notes
+-------------
+
+* ⚠️ The feature flag to disable backend validation is now removed, instances relying
+  on it should verify that their existing forms works with the validation enforced.
+
+* We're consolidating the OpenID Connect *Redirect URI* endpoints into a single
+  endpoint: ``/auth/oidc/callback/``. The legacy endpoints are still enabled (by default),
+  but scheduled for removal in Open Forms 3.0.
+
+  You can already opt-in to the new behaviour through three environment variables:
+
+  - ``USE_LEGACY_OIDC_ENDPOINTS=false``: admin login
+  - ``USE_LEGACY_DIGID_EH_OIDC_ENDPOINTS=false``: DigiD/eHerkenning plugins
+  - ``USE_LEGACY_ORG_OIDC_ENDPOINTS=false``: Organization OIDC plugin
+
+  Note that the OpenID applications need to be updated on the identity provider,
+  specifically the allowed "Redirect URIs" setting needs to be updated with the
+  following path replacements:
+
+  - ``/oidc/callback/`` -> ``/auth/oidc/callback/``
+  - ``/digid-oidc/callback/`` -> ``/auth/oidc/callback/``
+  - ``/eherkenning-oidc/callback/`` -> ``/auth/oidc/callback/``
+  - ``/digid-machtigen-oidc/callback/`` -> ``/auth/oidc/callback/``
+  - ``/eherkenning-bewindvoering-oidc/callback/`` -> ``/auth/oidc/callback/``
+  - ``/org-oidc/callback/`` -> ``/auth/oidc/callback/``
+
 2.6.9 (2024-06-14)
 ==================
 
@@ -61,7 +94,6 @@ Note that 2.4.8 was never published to Docker Hub.
 * [#4390] Fixed regression introduced by #4368 that would break template variables in
   hyperlinks inside WYSIWYG content.
 
-
 2.4.8 (2024-06-14)
 ==================
 
@@ -73,17 +105,6 @@ Bugfix release
 * [#4362] Fixed a crash in the form designer when a textfield/textarea allows multiple
   values in forms with translations enabled.
 
-2.7.0-alpha.1 (2024-XX-XX)
-==========================
-
-This is an alpha release, meaning it is not finished yet or suitable for production use.
-
-Upgrade notes
--------------
-
-* ⚠️ The feature flag to disable backend validation is now removed, instances relying
-  on it should verify that their existing forms works with the validation enforced.
-
 2.6.7 (2024-05-22)
 ==================
 

docker-compose.yml

@@ -63,7 +63,6 @@ services:
       - DB_HOST=db
       - CACHE_DEFAULT=redis:6379/0
       - CACHE_AXES=redis:6379/0
-      - CACHE_OIDC=redis:6379/0
       - CACHE_PORTALOCKER=redis:6379/0
       - ALLOWED_HOSTS=*
       - CELERY_BROKER_URL=redis://redis:6379/0
@@ -76,6 +75,11 @@ services:
       # settings used by workers and beat containers.
       - TZ=Europe/Amsterdam
       - CSP_REPORT_ONLY=yes
+      # Legacy endpoints are deprecated, new installs should opt-out of using legacy
+      # settings (the default is to use legacy endpoints).
+      - USE_LEGACY_OIDC_ENDPOINTS=no
+      - USE_LEGACY_DIGID_EH_OIDC_ENDPOINTS=no
+      - USE_LEGACY_ORG_OIDC_ENDPOINTS=no
     volumes: &web_volumes
       - media:/app/media
       - private_media:/app/private_media

docker/docker-compose.keycloak.yml

@@ -1,14 +1,25 @@
-version: '3.4'
+# Config taken from https://stackoverflow.com/a/77257732
+#
+# This docker-compose is for development and test purposes. Tests are recorded with
+# VCR against this instance.
+#
+#   Log in to http://localhost:8080/admin/master/console/ with `admin`/`admin`
+#   credentials.
+#
+# DO NOT USE THIS IN PRODUCTION.
+#
+
+version: '3.8'
 
 services:
   keycloak:
-    image: jboss/keycloak
+    image: quay.io/keycloak/keycloak:23.0
+    command: start-dev --import-realm
     environment:
-      - KEYCLOAK_USER=demo
-      - KEYCLOAK_PASSWORD=demo
-      - KEYCLOAK_IMPORT=/realm.json
+      - KEYCLOAK_ADMIN=admin
+      - KEYCLOAK_ADMIN_PASSWORD=admin
     volumes:
-      - ./keycloak/fixtures/realm.json:/realm.json
+      - ./keycloak/import:/opt/keycloak/data/import
     ports:
       - 8080:8080
     networks:

docker/keycloak/README.md

@@ -1,10 +1,10 @@
 # Keycloak infrastructure
 
-Open Forms supports OpenID Connect as an authentication protocol. Keycloak is
-an example of an Identity Provider that supports OIDC.
+Open Forms supports OpenID Connect as an authentication protocol. Keycloak is an example of an
+Identity Provider that supports OIDC.
 
-We include a compose stack for development and CI purposes. This is **NOT** suitable
-for production usage.
+We include a compose stack for development and testing/CI purposes. This is **NOT** suitable for
+production usage.
 
 ## docker compose
 
@@ -14,34 +14,63 @@ Start a Keycloak instance in your local environment from the parent directory:
 docker compose -f docker-compose.keycloak.yml up -d
 ```
 
-This brings up Keycloak, the admin interface is accessible at http://localhost:8080/.
-You can log in with `demo:demo`.
+This brings up Keycloak, the admin interface is accessible at http://localhost:8080/. You can now
+log in with the `admin`/`admin` credentials.
 
-In order to allow access to Keycloak via the same hostname via the Open Forms backend
-container and the browser, add the following entry to your `/etc/hosts` file:
+In order to allow access to Keycloak via the same hostname via the Open Forms backend container and
+the browser, add the following entry to your `/etc/hosts` file:
 
 ```
 127.0.0.1 keycloak.open-forms.local
 ```
 
+## Testing
 
-## Load fixtures
+This realm is used in the integration tests. We re-record the network traffic periodically (using
+VCR.py). The primary reason this setup exists, is for automated testing reasons.
 
-Before the DigiD login via OIDC can be tested, a fixture needs to be loaded.
-Assuming the docker containers specified in `docker-compose.yml` in the root directory
-are running, run the following commands:
+### Test data
 
-```bash
-cat docker/keycloak/fixtures/oidc.json | docker-compose exec web src/manage.py loaddata --format=json -
-```
+**Clients**
+
+- Client ID: `test-userinfo-jwt`, secret `ktGlGUELd1FR7dTXc84L7dJzUTjCtw9S`
+
+  Configured to return the user info as a JWT rather than JSON response.
+
+- Client ID: `testid`, secret: `7DB3KUAAizYCcmZufpHRVOcD0TOkNO3I`
+
+**Users**
+
+- `testuser` / `testuser`, has the `bsn`, `kvk`, `name_qualifier`, `legalSubjectID` and
+  `actingSubjectID` attributes (authentication plugins: DigiD, eHerkenning)
+- `digid-machtigen` / `digid-machtigen`, has the `aanvrager.bsn`, `gemachtigde.bsn` and
+  `service_id` attributes (for DigiD machtigen)
+- `eherkenning-bewindvoering` / `eherkenning-bewindvoering`, has the `legalSubjectID`
+  (kvk), `actingSubjectID` (pseudo ID), `representeeBSN`, `service_id`, `service_uuid`,
+  and `name_qualifier`  attributes (for eHerkenning bewindvoering)
+- `admin` / `admin`, intended to create as django user (can be made staff). The email
+  address is `admin@example.com`.
 
-This loads an example form configured to use DigiD via OIDC for authentication and
-it loads a configuration to connect to our Keycloak instance.
+## Exporting the Realm
 
-## Test login flow
+In short - exporting through the admin UI (rightfully) obfuscates client secrets and user
+credentials. However, for reproducible builds/environments, we want to include this data in the
+Realm export.
 
-To test the login flow, navigate to `http://127.0.0.1:8000/digid-oidc/`
-(not `localhost`, because this domain is not on the allowlist in the Keycloak config).
+Ensure the service is up and running through docker-compose.
 
-Click `Inloggen met DigiD` and fill in `testuser` for both username and password
-in the Keycloak login screen. If everything succeeded, you are now redirected back to the form.
+Ensure that UID `1000` can write to `./keycloak/import/`:
+
+```bash
+chmod o+rwx ./keycloak/import/
+```
+
+Then open another terminal and run:
+
+```bash
+docker-compose -f docker-compose.keycloak.yml exec keycloak \
+   /opt/keycloak/bin/kc.sh \
+   export \
+   --file /opt/keycloak/data/import/test-realm.json \
+   --realm test
+```