MSC4028: Push all encrypted events except for muted rooms

proposals/4028-push-all-encrypted-events-except-for-muted-rooms.md

@@ -0,0 +1,123 @@
+# MSC4028: Push all encrypted events except for muted rooms
+
+About notifications handling, it is currently not possible for mobile clients which get push notifications to properly implement a mentions-and-keywords-only room.
+
+Like it was already described in [MSC3996: Encrypted mentions-only rooms](https://github.com/matrix-org/matrix-spec-proposals/pull/3996): 
+
+Currently, every new event in an encrypted room might be pushed to mobile clients due to the default **underride** rule [`.m.rule.encrypted`](https://spec.matrix.org/v1.7/client-server-api/#default-underride-rules).
+
+However, a room can be configured to be mentions-and-keywords-only by creating a [room-specific push rule](https://spec.matrix.org/v1.7/client-server-api/#push-rules)
+with the `id` property set to the room ID & `actions` set to do nothing.[^1] Since this is evaluated *before* the `.m.rule.encrypted` rule (almost) **no events get pushed for a mentions-and-keywords-only room**.
+
+Additionally, a room can be configured to be muted by creating the **override** [push rule](https://spec.matrix.org/v1.7/client-server-api/#push-rules) which matches the room ID & has `actions` set to do nothing, e.g.:
+
+```json
+{
+    "rule_id": "!abcdef:example.com",
+    "conditions": [
+      {
+         "key": "room_id",
+         "kind": "event_match",
+         "pattern": "!abcdef:example.com"
+      }
+    ],
+    "default": false,
+    "enabled": true,
+    "actions": []
+}
+```
+
+## Proposal
+
+A new predefined **override** push rule (`.m.rule.encrypted_event`) is added just after the `.m.rule.master` push rule to be able to push all encrypted events:
+
+```json
+{
+    "rule_id": ".m.rule.encrypted_event",
+    "default": true,
+    "enabled": true,
+    "conditions": [
+      {
+         "kind": "event_match",
+         "key": "type",
+         "pattern": "m.room.encrypted"
+      }
+    ],
+    "actions": ["notify"]
+}
+```
+
+Note: The “user-defined rules” are evaluated with a higher priority than “server-default rules”, except for the `.m.rule.master` rule which has always a higher priority than any other rule (see [here](https://spec.matrix.org/v1.7/client-server-api/#predefined-rules)). So all the **override** push rules created to mute rooms will be evaluated before this new one. The new rule will be evaluated before all the other “server-default rules”. This new rule deprecates the existing underride rule [`.m.rule.encrypted`](https://spec.matrix.org/v1.7/client-server-api/#default-underride-rules), which will become useless.
+
+### Receiving notifications
+
+When this push rule matches then homeserver would push the event to all registered clients, similar to how other rules work. Clients would [decrypt the event and re-run push rules](https://spec.matrix.org/v1.7/client-server-api/#receiving-notifications) as normal. If the decrypted event results in no notification, the push is discarded [^2]. If the decryption failed, the new `.m.rule.encrypted_event` rule will match on client side, but there is no reason to notify the end user, so the push is discarded [^2] too. Globally the client must discard the push when the event stays encrypted locally because this use case should not happen except if the event was not encrypted for the current session, or if we are in front of an unexpected "Unable To Decrypt" (UTD) issue.
+
+The overall tradeoff is that clients will receive extra pushes some of the time.
+
+### Unread notification counts
+
+As soon as the rule `.m.rule.encrypted_event` is present and enabled in the account push rules set, the clients have to track unread notifications themselves in **encrypted rooms**. All the clients with or without push services will have to compute the unread notification count for each encrypted room during the client-server sync, by implementing the algorithm described below.
+
+This algorithm depends on the unread notification count `unread_notifications` received from the server for each joined room in the [sync response](https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientv3sync). It has to be adapted to the potential unread thread notification counts `unread_thread_notifications` that we will ignore here to simplify the description.
+
+Let's consider we receive the following data for a joined encrypted rooms in the sync response:
+- number of new encrypted events in the timeline: n
+- unread notification count: k
+
+1. if k = 0 -> reset the local notification counts for this room (the unread notification count and the highlight count).
+2. else if k <= n -> reset the local notification counts, then decrypt and run push rules on the k most recent events:
+   - if the decrypted event resulted in a highlight notification -> increment the local highlight notifications count and the local unread notification count.
+   - if the decrypted event results in a notification -> increment only the local unread notifications count.
+   - if the decrypted event results in no notification -> no change is required on the local notification counts.
+   - if the decryption failed -> increment the local unread notifications count.
+3. else if k > n and no gap is mentioned in the sync response for this room -> update the local notification counts by taking into account the potential change of the read receipt position of the current user, then decrypt and run push rules on the n received events by applying the same rules as above.
+4. else if k > n and a gap is mentioned in the sync response -> there is 2 options: 
+- flush the local room storage, this will reset the local notification counts, then decrypt and run push rules on the n received events by applying the same rules as above. In order to compute locally accurate notification counts, back paginate to retrieve (k-n) more encrypted events.
+
+or
+
+- update the local notification counts by taking into account the potential change of the read receipt position of the current user, then decrypt and run push rules on the n received events by applying the same rules as above. In order to compute locally accurate notification counts, back paginate to close the gap or at least retrieve (k-n) more encrypted events.
+
+About the case 4, the back pagination should be mandatory for mentions-and-keywords-only rooms to detect if there are some mentions in the gap. The client may decide to not trigger back pagination for rooms in "all messages" mode, but in that case the local unread notification count is unknown. No badge can be displayed at the application side, only an unread marker.
+
+Note: some unencrypted events (for example `m.room.redaction`) may be taken into account or not in this unreads count k. This depends on the push rules set. If you force "all messages" mode on a room by creating a room-specific push rule, some unencrypted events (like redaction events) will be included in this count. This is not the case if "all messages" mode is obtained by the combination of server predefined rules. The client have locally all the information (= the push rules) to handle this edge case during the implementation of the algorithm.
+
+### Listening notifications
+
+This new push rule will impact the existing endpoint [`GET /_matrix/client/v3/notifications`](https://spec.matrix.org/v1.7/client-server-api/#get_matrixclientv3notifications).
+
+When this rule will be present and enabled in the account push rules set, the clients which retrieve the list of events that it has been notified about with this endpoint, will receive most of the encrypted events (except for muted rooms). They will be able to decrypt and re-run push rules locally. This should fix the notifications panel observed in some web clients where currently the notifications of encrypted events are missing.
+
+## Potential issues
+
+Until a client is updated to support this MSC, it may display a high number of unread notifications for encrypted rooms. The first workaround would be to not display the count for the encrypted rooms when the rule `.m.rule.encrypted_event` is present and enabled in the account push rules set, until the suggested algorithm is implemented.
+
+## Alternatives
+
+[MSC3996: Encrypted mentions-only rooms](https://github.com/matrix-org/matrix-spec-proposals/pull/3996): the suggested top-level property `m.has_mentions` may be an option to reduce the volume of pushes. But we would not be able to support notifications on keywords then.
+
+Another alternative would be to define a new [`action`](https://spec.matrix.org/v1.7/client-server-api/#actions): `push_without_notify` or `silently_notify` in order to push all encrypted events without incrementing the notifications count. The client implementation to handle these counts would be then less complex. But this would need to implement more changes in Synapse. The current proposal is just to add a new push rules without changing the rules handling server side. This alternative should be discussed as soon as possible.
+
+## Security considerations
+
+None
+
+## Future extensions
+
+None
+
+## Unstable prefix
+
+During development the new push rule shall use `org.matrix.msc4028.encrypted_event` instead of `.m.rule.encrypted_event`.
+
+## Dependencies
+
+None
+
+
+[^1]: Via either an explicit `"dont_notify"` action or an empty array. These are
+equivalent, see [issue #643](https://github.com/matrix-org/matrix-spec/issues/643).
+
+[^2]: In the past it was not possible to discard notifications on iOS: if a push
+notification was received it *had to be displayed*. This is [no longer the case](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_usernotifications_filtering).