Merge branch 'develop' into shay/improve_downloads

element-hq · May 31, 2024 · 7a3bac3 · 7a3bac3
2 parents 830cbd9 + 5c2a837
commit 7a3bac3
Show file tree

Hide file tree

Showing 112 changed files with 5,977 additions and 1,915 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,3 +1,56 @@
+# Synapse 1.108.0 (2024-05-28)
+
+No significant changes since 1.108.0rc1.
+
+
+
+
+# Synapse 1.108.0rc1 (2024-05-21)
+
+### Features
+
+- Add a feature that allows clients to query the configured federation whitelist. Disabled by default. ([\#16848](https://github.com/element-hq/synapse/issues/16848), [\#17199](https://github.com/element-hq/synapse/issues/17199))
+- Add the ability to allow numeric user IDs with a specific prefix when in the CAS flow. Contributed by Aurélien Grimpard. ([\#17098](https://github.com/element-hq/synapse/issues/17098))
+
+### Bugfixes
+
+- Fix bug where push rules would be empty in `/sync` for some accounts. Introduced in v1.93.0. ([\#17142](https://github.com/element-hq/synapse/issues/17142))
+- Add support for optional whitespace around the Federation API's `Authorization` header's parameter commas. ([\#17145](https://github.com/element-hq/synapse/issues/17145))
+- Fix bug where disabling room publication prevented public rooms being created on workers. ([\#17177](https://github.com/element-hq/synapse/issues/17177), [\#17184](https://github.com/element-hq/synapse/issues/17184))
+
+### Improved Documentation
+
+- Document [`/v1/make_knock`](https://spec.matrix.org/v1.10/server-server-api/#get_matrixfederationv1make_knockroomiduserid) and [`/v1/send_knock/`](https://spec.matrix.org/v1.10/server-server-api/#put_matrixfederationv1send_knockroomideventid) federation endpoints as worker-compatible. ([\#17058](https://github.com/element-hq/synapse/issues/17058))
+- Update User Admin API with note about prefixing OIDC external_id providers. ([\#17139](https://github.com/element-hq/synapse/issues/17139))
+- Clarify the state of the created room when using the `autocreate_auto_join_room_preset` config option. ([\#17150](https://github.com/element-hq/synapse/issues/17150))
+- Update the Admin FAQ with the current libjemalloc version for latest Debian stable. Additionally update the name of the "push_rules" stream in the Workers documentation. ([\#17171](https://github.com/element-hq/synapse/issues/17171))
+
+### Internal Changes
+
+- Add note to reflect that [MSC3886](https://github.com/matrix-org/matrix-spec-proposals/pull/3886) is closed but will remain supported for some time. ([\#17151](https://github.com/element-hq/synapse/issues/17151))
+- Update dependency PyO3 to 0.21. ([\#17162](https://github.com/element-hq/synapse/issues/17162))
+- Fixes linter errors found in PR #17147. ([\#17166](https://github.com/element-hq/synapse/issues/17166))
+- Bump black from 24.2.0 to 24.4.2. ([\#17170](https://github.com/element-hq/synapse/issues/17170))
+- Cache literal sync filter validation for performance. ([\#17186](https://github.com/element-hq/synapse/issues/17186))
+- Improve performance by fixing a reactor pause. ([\#17192](https://github.com/element-hq/synapse/issues/17192))
+- Route `/make_knock` and `/send_knock` federation APIs to the federation reader worker in Complement test runs. ([\#17195](https://github.com/element-hq/synapse/issues/17195))
+- Prepare sync handler to be able to return different sync responses (`SyncVersion`). ([\#17200](https://github.com/element-hq/synapse/issues/17200))
+- Organize the sync cache key parameter outside of the sync config (separate concerns). ([\#17201](https://github.com/element-hq/synapse/issues/17201))
+- Refactor `SyncResultBuilder` assembly to its own function. ([\#17202](https://github.com/element-hq/synapse/issues/17202))
+- Rename to be obvious: `joined_rooms` -> `joined_room_ids`. ([\#17203](https://github.com/element-hq/synapse/issues/17203), [\#17208](https://github.com/element-hq/synapse/issues/17208))
+- Add a short pause when rate-limiting a request. ([\#17210](https://github.com/element-hq/synapse/issues/17210))
+
+
+
+### Updates to locked dependencies
+
+* Bump cryptography from 42.0.5 to 42.0.7. ([\#17180](https://github.com/element-hq/synapse/issues/17180))
+* Bump gitpython from 3.1.41 to 3.1.43. ([\#17181](https://github.com/element-hq/synapse/issues/17181))
+* Bump immutabledict from 4.1.0 to 4.2.0. ([\#17179](https://github.com/element-hq/synapse/issues/17179))
+* Bump sentry-sdk from 1.40.3 to 2.1.1. ([\#17178](https://github.com/element-hq/synapse/issues/17178))
+* Bump serde from 1.0.200 to 1.0.201. ([\#17183](https://github.com/element-hq/synapse/issues/17183))
+* Bump serde_json from 1.0.116 to 1.0.117. ([\#17182](https://github.com/element-hq/synapse/issues/17182))
+
 Synapse 1.107.0 (2024-05-14)
 ============================
 

diff --git a/Cargo.lock b/Cargo.lock
diff --git a/changelog.d/16848.feature b/changelog.d/16848.feature
diff --git a/changelog.d/17083.misc b/changelog.d/17083.misc
@@ -0,0 +1 @@
+Improve DB usage when fetching related events.
diff --git a/changelog.d/17098.feature b/changelog.d/17098.feature
diff --git a/changelog.d/17139.doc b/changelog.d/17139.doc
diff --git a/changelog.d/17145.bugfix b/changelog.d/17145.bugfix
diff --git a/changelog.d/17147.feature b/changelog.d/17147.feature
@@ -0,0 +1 @@
+Add the ability to auto-accept invites on the behalf of users. See the [`auto_accept_invites`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#auto-accept-invites) config option for details.
diff --git a/changelog.d/17150.doc b/changelog.d/17150.doc
diff --git a/changelog.d/17151.misc b/changelog.d/17151.misc
diff --git a/changelog.d/17162.misc b/changelog.d/17162.misc
diff --git a/changelog.d/17164.bugfix b/changelog.d/17164.bugfix
@@ -0,0 +1 @@
+Fix deduplicating of membership events to not create unused state groups.
diff --git a/changelog.d/17166.misc b/changelog.d/17166.misc
diff --git a/changelog.d/17167.feature b/changelog.d/17167.feature
@@ -0,0 +1 @@
+Add experimental [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) Sliding Sync `/sync/e2ee` endpoint for To-Device messages and device encryption info.
diff --git a/changelog.d/17170.misc b/changelog.d/17170.misc
diff --git a/changelog.d/17171.doc b/changelog.d/17171.doc
diff --git a/changelog.d/17176.misc b/changelog.d/17176.misc
@@ -0,0 +1 @@
+Log exceptions when failing to auto-join new user according to the `auto_join_rooms` option.
diff --git a/changelog.d/17177.bugfix b/changelog.d/17177.bugfix
diff --git a/changelog.d/17184.bugfix b/changelog.d/17184.bugfix
diff --git a/changelog.d/17186.misc b/changelog.d/17186.misc
diff --git a/changelog.d/17192.misc b/changelog.d/17192.misc
diff --git a/changelog.d/17204.doc b/changelog.d/17204.doc
@@ -0,0 +1 @@
+Update OIDC documentation: by default Matrix doesn't query userinfo endpoint, then claims should be put on id_token.
diff --git a/changelog.d/17211.misc b/changelog.d/17211.misc
@@ -0,0 +1 @@
+Reduce work of calculating outbound device lists updates.
diff --git a/changelog.d/17213.feature b/changelog.d/17213.feature
@@ -0,0 +1 @@
+Support MSC3916 by adding unstable media endpoints to `_matrix/client` (#17213).
diff --git a/changelog.d/17215.bugfix b/changelog.d/17215.bugfix
@@ -0,0 +1 @@
+Fix bug where duplicate events could be sent down sync when using workers that are overloaded.
diff --git a/changelog.d/17191.misc → changelog.d/17216.misc b/changelog.d/17191.misc → changelog.d/17216.misc
diff --git a/changelog.d/17219.feature b/changelog.d/17219.feature
@@ -0,0 +1 @@
+Add logging to tasks managed by the task scheduler, showing CPU and database usage.
diff --git a/changelog.d/17226.misc b/changelog.d/17226.misc
@@ -0,0 +1 @@
+Move towards using `MultiWriterIdGenerator` everywhere.
diff --git a/changelog.d/17229.misc b/changelog.d/17229.misc
@@ -0,0 +1 @@
+Replaces all usages of `StreamIdGenerator` with `MultiWriterIdGenerator`.
diff --git a/changelog.d/17238.misc b/changelog.d/17238.misc
@@ -0,0 +1 @@
+Change the `allow_unsafe_locale` config option to also apply when setting up new databases.
diff --git a/changelog.d/17239.misc b/changelog.d/17239.misc
@@ -0,0 +1 @@
+Fix errors in logs about closing incorrect logging contexts when media gets rejected by a module.
diff --git a/changelog.d/17240.bugfix b/changelog.d/17240.bugfix
@@ -0,0 +1 @@
+Ignore attempts to send to-device messages to bad users, to avoid log spam when we try to connect to the bad server.
diff --git a/changelog.d/17241.bugfix b/changelog.d/17241.bugfix
@@ -0,0 +1 @@
+Fix handling of duplicate concurrent uploading of device one-time-keys.
diff --git a/changelog.d/17242.misc b/changelog.d/17242.misc
@@ -0,0 +1 @@
+Clean out invalid destinations from `device_federation_outbox` table.
diff --git a/changelog.d/17246.misc b/changelog.d/17246.misc
@@ -0,0 +1 @@
+Fix errors in logs about closing incorrect logging contexts when media gets rejected by a module.
diff --git a/changelog.d/17250.misc b/changelog.d/17250.misc
@@ -0,0 +1 @@
+Stop logging errors when receiving invalid User IDs in key querys requests.
diff --git a/changelog.d/17251.bugfix b/changelog.d/17251.bugfix
@@ -0,0 +1 @@
+Fix reporting of default tags to Sentry, such as worker name. Broke in v1.108.0.
diff --git a/changelog.d/17252.bugfix b/changelog.d/17252.bugfix
@@ -0,0 +1 @@
+Fix bug where typing updates would not be sent when using workers after a restart.
diff --git a/debian/changelog b/debian/changelog
@@ -1,3 +1,15 @@
+matrix-synapse-py3 (1.108.0) stable; urgency=medium
+
+  * New Synapse release 1.108.0.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 28 May 2024 11:54:22 +0100
+
+matrix-synapse-py3 (1.108.0~rc1) stable; urgency=medium
+
+  * New Synapse release 1.108.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 21 May 2024 10:54:13 +0100
+
 matrix-synapse-py3 (1.107.0) stable; urgency=medium
 
   * New Synapse release 1.107.0.

diff --git a/docker/configure_workers_and_start.py b/docker/configure_workers_and_start.py
@@ -211,6 +211,8 @@
             "^/_matrix/federation/(v1|v2)/make_leave/",
             "^/_matrix/federation/(v1|v2)/send_join/",
             "^/_matrix/federation/(v1|v2)/send_leave/",
+            "^/_matrix/federation/v1/make_knock/",
+            "^/_matrix/federation/v1/send_knock/",
             "^/_matrix/federation/(v1|v2)/invite/",
             "^/_matrix/federation/(v1|v2)/query_auth/",
             "^/_matrix/federation/(v1|v2)/event_auth/",

diff --git a/docs/openid.md b/docs/openid.md
@@ -525,6 +525,8 @@ oidc_providers:
   (`Options > Security > ID Token signature algorithm` and `Options > Security >
   Access Token signature algorithm`)
 - Scopes: OpenID, Email and Profile
+- Force claims into `id_token`
+  (`Options > Advanced > Force claims to be returned in ID Token`)
 - Allowed redirection addresses for login (`Options > Basic > Allowed
   redirection addresses for login` ) :
   `[synapse public baseurl]/_synapse/client/oidc/callback`

diff --git a/docs/postgres.md b/docs/postgres.md
@@ -242,12 +242,11 @@ host    all         all             ::1/128     ident
 
 ### Fixing incorrect `COLLATE` or `CTYPE`
 
-Synapse will refuse to set up a new database if it has the wrong values of
-`COLLATE` and `CTYPE` set. Synapse will also refuse to start an existing database with incorrect values
-of `COLLATE` and `CTYPE` unless the config flag `allow_unsafe_locale`, found in the 
-`database` section of the config, is set to true. Using different locales can cause issues if the locale library is updated from
-underneath the database, or if a different version of the locale is used on any
-replicas.
+Synapse will refuse to start when using a database with incorrect values of
+`COLLATE` and `CTYPE` unless the config flag `allow_unsafe_locale`, found in the
+`database` section of the config, is set to true. Using different locales can
+cause issues if the locale library is updated from underneath the database, or
+if a different version of the locale is used on any replicas.
 
 If you have a database with an unsafe locale, the safest way to fix the issue is to dump the database and recreate it with
 the correct locale parameter (as shown above). It is also possible to change the

diff --git a/docs/usage/configuration/config_documentation.md b/docs/usage/configuration/config_documentation.md
@@ -1236,7 +1236,7 @@ federation_domain_whitelist:
 
 Enables an endpoint for fetching the federation whitelist config.
 
-The request method and path is `GET /_synapse/client/config/federation_whitelist`, and the
+The request method and path is `GET /_synapse/client/v1/config/federation_whitelist`, and the
 response format is:
 
 ```json
@@ -4613,3 +4613,32 @@ background_updates:
     min_batch_size: 10
     default_batch_size: 50
 ```
+---
+## Auto Accept Invites
+Configuration settings related to automatically accepting invites.
+
+---
+### `auto_accept_invites`
+
+Automatically accepting invites controls whether users are presented with an invite request or if they
+are instead automatically joined to a room when receiving an invite. Set the `enabled` sub-option to true to
+enable auto-accepting invites. Defaults to false.
+This setting has the following sub-options:
+* `enabled`: Whether to run the auto-accept invites logic. Defaults to false.
+* `only_for_direct_messages`: Whether invites should be automatically accepted for all room types, or only
+   for direct messages. Defaults to false.
+* `only_from_local_users`: Whether to only automatically accept invites from users on this homeserver. Defaults to false.
+* `worker_to_run_on`: Which worker to run this module on. This must match the "worker_name".
+
+NOTE: Care should be taken not to enable this setting if the `synapse_auto_accept_invite` module is enabled and installed.
+The two modules will compete to perform the same task and may result in undesired behaviour. For example, multiple join
+events could be generated from a single invite.
+
+Example configuration:
+```yaml
+auto_accept_invites:
+    enabled: true
+    only_for_direct_messages: true
+    only_from_local_users: true
+    worker_to_run_on: "worker_1"
+```
diff --git a/docs/workers.md b/docs/workers.md
@@ -211,6 +211,8 @@ information.
     ^/_matrix/federation/v1/make_leave/
     ^/_matrix/federation/(v1|v2)/send_join/
     ^/_matrix/federation/(v1|v2)/send_leave/
+    ^/_matrix/federation/v1/make_knock/
+    ^/_matrix/federation/v1/send_knock/
     ^/_matrix/federation/(v1|v2)/invite/
     ^/_matrix/federation/v1/event_auth/
     ^/_matrix/federation/v1/timestamp_to_event/