From 845704d4d217cf46b50c97b1fc8a53f8986da49a Mon Sep 17 00:00:00 2001 From: gnuxie Date: Thu, 26 Sep 2024 23:06:19 +0100 Subject: [PATCH 1/5] first draft. Need to make it clear what the changes are from `m.ban`. This will make it easier to talk about what the more severe consequences are that need manual approval. --- proposals/0000-takedown-recommendation.md | 137 ++++++++++++++++++++++ 1 file changed, 137 insertions(+) create mode 100644 proposals/0000-takedown-recommendation.md diff --git a/proposals/0000-takedown-recommendation.md b/proposals/0000-takedown-recommendation.md new file mode 100644 index 00000000000..a6e974d1e02 --- /dev/null +++ b/proposals/0000-takedown-recommendation.md @@ -0,0 +1,137 @@ +# MSC0000: `m.takedown` moderation policy recommendation + +Currently there is one specific moderation policy recommendation, `m.ban`[^spec-ban-recommendation]. + +> When this recommendation is used, the entities affected by the rule +> should be banned from participation where possible. The enforcement +> of this is deliberately left as an implementation detail to avoid +> the protocol imposing its opinion on how the policy list is to be +> interpreted. However, a suggestion for a simple implementation is as +> follows: + +[^spec-ban-recommendation]: https://spec.matrix.org/v1.11/client-server-api/#mban-recommendation + +In practice this recommendation is used by the moderation bots to +store banned rooms, users and servers alongside a reason for the ban. + +This has provided the ability to create versatile policy lists that +can ban matrix users for any recorded reason. Not only for spam, +illegal content, csam, but softer reasons such as moderation disputes, +timeouts, inrventions, disagreements. + +Unfortunately, because the `m.ban` recommendation has such a broad and +consequently has the same power for any consequence, +it is the expectation that a reason is provided for each `m.ban`. + +This reason inadvertently allows the subject of the moderation policy +to be classified as an undesirable consequence. For example, a user called +`@yarrgh:example.com` could be banned for the reason `piracy`. Which +would identify `@yarrgh:example.com` as a pirate. Which is problematic. + +Moderation bots currently also use the `reason` of the ban to determine +whether to redact a user's events or not. Which doesn't allow +for a free-form use of the `reason` where redaction is required. + +## Proposal + +We introduce a new policy recommendation, `m.takedown`, reserved for +purging entities and any associated content. When `m.takedown` is +used, a `reason` SHOULD NOT be embedded into the same policy event[^bluesky]. + +The enforcement of the `m.takedown` recommendation deliberately left +as an implementation detail to avoid the protocol imposing its opinion +on how the `recommendation` is to be interpreted. +However, examples of how the recommendation is expected to be used +by implementations is as follows: + +If the entity of the rule is a user: + ++ Applied to a user: The user and all associated messages, media, invitations + are hidden and removed from local storage or caches. + ++ Applied to a room: The user is banned from the room and all of their + recent messages redacted. + ++ Applied to a server: The user is not allowed to send invites to + users on the server. Any associated media is quarantined or removed, + if the user is resident then the user is considered for + deactivation. + +If the entity of the rule is a room: + ++ Applied to a user: The user is parted from the room, + and be unable to rejoin it. An invitations to the room are + rejected or ignored. + ++ Applied to a room: No-op because a room can't take itself down. + ++ Applied to a server: The room is purged from the server and + resident users are prevented from joining. Invitations + to the room are blocked. + +If the entity of the rule is a server: + ++ Applied to a user: Invitations originating from from users on the server are hidden. + Media uploaded from users resident to the target server are removed from + local storage or caches and is never be displayed. + Messages originating from the server could be hidden behind spoiler text. + ++ Applied to a room: The server is added as a denied server in the ACLs. + All room members resident to the target server are banned from the room + and have their recent messages redacted. + ++ Applied to a server: The subscriber should avoid federating with the + server as much as possible by blocking invites from the server and + not sending traffic unless strictly required (no outbound invites). + Any media associated with the server is quarantined or removed. + + +[^bluesky] This is inspired by work going into bluesky +https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone + +## Potential issues + ++ Some consequences of recommending `m.takedown` against an entity are irreversible, + and can have a huge impact on the history of a Matrix room if implemented naively. + ++ Because the `reason` is no longer present, a higher degree of trust is required + when applying some consequences to these policies. + ++ The lack of classification from the `reason` field is insufficient + to prevent direct identification of some entities. For example + `@yarrgh:example.com` is evidently a pirate because of the presence + of the phrase `yarrgh` embeded within their mxid. + +## Alternatives + ++ `m.ban` Could continue to be used with a more specific reason, but + this presents problems for automated and semi-automated responses + because the reasons are not standardised. + ++ `m.ban` A `takedown` reason could be given on `m.ban` policies. + This is pretty adhoc. + + +## Security considerations + ++ `m.takedown` can have severe consequences for entities, policies + could be created maliciously against innocent users on popular + policy rooms. This could increasce the reward of infiltrating + moderation focussed communities. Tools that naively implement + the recommendation without safeguards, such as manual approval, + for the most severe consequences could be exploited. + +## Unstable prefix + +*If a proposal is implemented before it is included in the spec, then implementers must ensure that the +implementation is compatible with the final version that lands in the spec. This generally means that +experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary. +For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section +should be used to document things such as what endpoints and names are being used while the feature is +in development, the name of the unstable feature flag to use to detect support for the feature, or what +migration steps are needed to switch to newer versions of the proposal.* + +## Dependencies + +This MSC builds on MSCxxxx, MSCyyyy and MSCzzzz (which at the time of writing have not yet been accepted +into the spec). From f0b4eea9a8d19fe60c2bc4bda0335b7057721ec2 Mon Sep 17 00:00:00 2001 From: gnuxie Date: Wed, 2 Oct 2024 12:20:26 +0100 Subject: [PATCH 2/5] touch up. --- proposals/0000-takedown-recommendation.md | 93 +++++++++++++++-------- 1 file changed, 60 insertions(+), 33 deletions(-) diff --git a/proposals/0000-takedown-recommendation.md b/proposals/0000-takedown-recommendation.md index a6e974d1e02..c4341808c1b 100644 --- a/proposals/0000-takedown-recommendation.md +++ b/proposals/0000-takedown-recommendation.md @@ -1,6 +1,6 @@ # MSC0000: `m.takedown` moderation policy recommendation -Currently there is one specific moderation policy recommendation, `m.ban`[^spec-ban-recommendation]. +Currently there is one specified moderation policy recommendation, `m.ban`[^spec-ban-recommendation]. > When this recommendation is used, the entities affected by the rule > should be banned from participation where possible. The enforcement @@ -19,18 +19,28 @@ can ban matrix users for any recorded reason. Not only for spam, illegal content, csam, but softer reasons such as moderation disputes, timeouts, inrventions, disagreements. -Unfortunately, because the `m.ban` recommendation has such a broad and -consequently has the same power for any consequence, -it is the expectation that a reason is provided for each `m.ban`. +Unfortunately, because the `m.ban` recommendation has such a broad +use, and its implementation has the same outcome for any consequence, +it is the expectation that a `reason` is provided for each `m.ban`. This reason inadvertently allows the subject of the moderation policy -to be classified as an undesirable consequence. For example, a user called -`@yarrgh:example.com` could be banned for the reason `piracy`. Which -would identify `@yarrgh:example.com` as a pirate. Which is problematic. - -Moderation bots currently also use the `reason` of the ban to determine -whether to redact a user's events or not. Which doesn't allow -for a free-form use of the `reason` where redaction is required. +to be classified among catagories of abuse, which can be +undesirable. For example, a user called `@yarrgh:example.com` could be +banned for the reason `piracy`. Which would identify +`@yarrgh:example.com` as a pirate. This is problematic because +it would allow other pirates to identify `@yarrgh:example.com` and +join forces. In other situations, it may cause harm or legal +ramifications if `@yarrgh:example.com`'s reputation has been damaged +and they are not a pirate at all. + +Additionally, if `@yarrgh:example.com` is known to been smuggling +contraband via matrix events or media, then there is no mechanism +to mark this content as plunder that should be ceased. + +A compunding issue is Moderation bots currently use the `reason` of +the ban to determine whether to redact a user's events or not. Which +doesn't allow for a free-form use of the `reason` where redaction is +required. ## Proposal @@ -38,16 +48,23 @@ We introduce a new policy recommendation, `m.takedown`, reserved for purging entities and any associated content. When `m.takedown` is used, a `reason` SHOULD NOT be embedded into the same policy event[^bluesky]. -The enforcement of the `m.takedown` recommendation deliberately left -as an implementation detail to avoid the protocol imposing its opinion -on how the `recommendation` is to be interpreted. -However, examples of how the recommendation is expected to be used -by implementations is as follows: +We provide guidance for how the `m.takedown` recommendation is +expected to be consumed by typical moderation tooling or clients. +This is to set expectations for policy list curators for how the +recommendation should behave and the severity of the consequences. We +also provide guidance to allow tool implementers to understand the +context of the recommendations use. Moderation tools or clients are +free to interpret the recommendation differently, for example if they +are aware of a conflciting policy or they are configured to implement +harsher or more liberal consequences for the recommendation. If the entity of the rule is a user: -+ Applied to a user: The user and all associated messages, media, invitations - are hidden and removed from local storage or caches. ++ Applied to a user: The user and all associated messages, media, + invitations are hidden and removed from local storage or caches. + If the user enforcing the recommendation is a moderator in the + room in common with the target, then messages may be shown + with a warning behind spoiler text. + Applied to a room: The user is banned from the room and all of their recent messages redacted. @@ -55,13 +72,14 @@ If the entity of the rule is a user: + Applied to a server: The user is not allowed to send invites to users on the server. Any associated media is quarantined or removed, if the user is resident then the user is considered for - deactivation. + deactivation or suspension. If the entity of the rule is a room: + Applied to a user: The user is parted from the room, - and be unable to rejoin it. An invitations to the room are - rejected or ignored. + and be unable to rejoin it. Any invitations to the room are + hidden and ignored. Any media associated with the room are + purged from local storage or caches. + Applied to a room: No-op because a room can't take itself down. @@ -71,19 +89,21 @@ If the entity of the rule is a room: If the entity of the rule is a server: -+ Applied to a user: Invitations originating from from users on the server are hidden. - Media uploaded from users resident to the target server are removed from - local storage or caches and is never be displayed. - Messages originating from the server could be hidden behind spoiler text. ++ Applied to a user: Invitations originating from from users on the + server are hidden. Media uploaded from users resident to the target + server are removed from local storage or caches and never + displayed. Messages originating from the server could be hidden + entirely or behind spoiler text. + Applied to a room: The server is added as a denied server in the ACLs. All room members resident to the target server are banned from the room and have their recent messages redacted. -+ Applied to a server: The subscriber should avoid federating with the - server as much as possible by blocking invites from the server and - not sending traffic unless strictly required (no outbound invites). - Any media associated with the server is quarantined or removed. ++ Applied to a server: The enforcing server should avoid federating + with the target server as much as possible by blocking invites from + the target server and not sending traffic unless strictly required + (no outbound invites). Any media associated with the server is + quarantined or removed. [^bluesky] This is inspired by work going into bluesky @@ -91,11 +111,15 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone ## Potential issues -+ Some consequences of recommending `m.takedown` against an entity are irreversible, - and can have a huge impact on the history of a Matrix room if implemented naively. ++ It is anticipaited that some consequences of recommending + `m.takedown` against an entity are irreversible, and can have a huge + impact on the history of a Matrix room if implemented naively. The + most common use case for the recommendation will be to replace the + use of `m.ban` with the reason `spam` targeting a user, which in + Mjolinr causes the target user to be banned from protected rooms and -+ Because the `reason` is no longer present, a higher degree of trust is required - when applying some consequences to these policies. ++ Because the `reason` is no longer present, a higher degree of trust + is required when applying some consequences to these policies. + The lack of classification from the `reason` field is insufficient to prevent direct identification of some entities. For example @@ -120,6 +144,9 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone moderation focussed communities. Tools that naively implement the recommendation without safeguards, such as manual approval, for the most severe consequences could be exploited. + Additionally, moderation tools ask for confirmation when the + associated entity is known to be an active participant of the + community being protected. ## Unstable prefix From 3dc314690c6f3353aa82fa0f452e949e7e47212d Mon Sep 17 00:00:00 2001 From: gnuxie Date: Wed, 2 Oct 2024 14:38:18 +0100 Subject: [PATCH 3/5] Link up the MSC with its relatives. --- ...ion.md => 4204-takedown-recommendation.md} | 24 +++++++------------ 1 file changed, 9 insertions(+), 15 deletions(-) rename proposals/{0000-takedown-recommendation.md => 4204-takedown-recommendation.md} (87%) diff --git a/proposals/0000-takedown-recommendation.md b/proposals/4204-takedown-recommendation.md similarity index 87% rename from proposals/0000-takedown-recommendation.md rename to proposals/4204-takedown-recommendation.md index c4341808c1b..0abe7acb60a 100644 --- a/proposals/0000-takedown-recommendation.md +++ b/proposals/4204-takedown-recommendation.md @@ -1,4 +1,4 @@ -# MSC0000: `m.takedown` moderation policy recommendation +# MSC4204: `m.takedown` moderation policy recommendation Currently there is one specified moderation policy recommendation, `m.ban`[^spec-ban-recommendation]. @@ -119,12 +119,17 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone Mjolinr causes the target user to be banned from protected rooms and + Because the `reason` is no longer present, a higher degree of trust - is required when applying some consequences to these policies. + is required when applying some consequences to these policies. An + attempt to document the reason privately exists through [MSC4206 + Moderation policy auditing and + context](https://github.com/matrix-org/matrix-spec-proposals/pull/4206). + The lack of classification from the `reason` field is insufficient to prevent direct identification of some entities. For example `@yarrgh:example.com` is evidently a pirate because of the presence - of the phrase `yarrgh` embeded within their mxid. + of the phrase `yarrgh` embeded within their mxid. An attempt to + alleviate this concern exists through [MSC4205 Hashed moderation policy + entities](https://github.com/matrix-org/matrix-spec-proposals/pull/4205). ## Alternatives @@ -150,15 +155,4 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone ## Unstable prefix -*If a proposal is implemented before it is included in the spec, then implementers must ensure that the -implementation is compatible with the final version that lands in the spec. This generally means that -experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary. -For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section -should be used to document things such as what endpoints and names are being used while the feature is -in development, the name of the unstable feature flag to use to detect support for the feature, or what -migration steps are needed to switch to newer versions of the proposal.* - -## Dependencies - -This MSC builds on MSCxxxx, MSCyyyy and MSCzzzz (which at the time of writing have not yet been accepted -into the spec). +`org.matrix.msc4204.takedown` -> `m.takedown`. From 1ec87a60d8961e889d0308f770f559e6606aee31 Mon Sep 17 00:00:00 2001 From: gnuxie Date: Wed, 2 Oct 2024 14:42:16 +0100 Subject: [PATCH 4/5] spelling --- proposals/4204-takedown-recommendation.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/proposals/4204-takedown-recommendation.md b/proposals/4204-takedown-recommendation.md index 0abe7acb60a..546062718ba 100644 --- a/proposals/4204-takedown-recommendation.md +++ b/proposals/4204-takedown-recommendation.md @@ -17,14 +17,14 @@ store banned rooms, users and servers alongside a reason for the ban. This has provided the ability to create versatile policy lists that can ban matrix users for any recorded reason. Not only for spam, illegal content, csam, but softer reasons such as moderation disputes, -timeouts, inrventions, disagreements. +timeouts, interventions, disagreements. Unfortunately, because the `m.ban` recommendation has such a broad use, and its implementation has the same outcome for any consequence, it is the expectation that a `reason` is provided for each `m.ban`. This reason inadvertently allows the subject of the moderation policy -to be classified among catagories of abuse, which can be +to be classified among categories of abuse, which can be undesirable. For example, a user called `@yarrgh:example.com` could be banned for the reason `piracy`. Which would identify `@yarrgh:example.com` as a pirate. This is problematic because @@ -37,7 +37,7 @@ Additionally, if `@yarrgh:example.com` is known to been smuggling contraband via matrix events or media, then there is no mechanism to mark this content as plunder that should be ceased. -A compunding issue is Moderation bots currently use the `reason` of +A compounding issue is Moderation bots currently use the `reason` of the ban to determine whether to redact a user's events or not. Which doesn't allow for a free-form use of the `reason` where redaction is required. @@ -55,7 +55,7 @@ recommendation should behave and the severity of the consequences. We also provide guidance to allow tool implementers to understand the context of the recommendations use. Moderation tools or clients are free to interpret the recommendation differently, for example if they -are aware of a conflciting policy or they are configured to implement +are aware of a conflicting policy or they are configured to implement harsher or more liberal consequences for the recommendation. If the entity of the rule is a user: @@ -111,12 +111,12 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone ## Potential issues -+ It is anticipaited that some consequences of recommending ++ It is anticipated that some consequences of recommending `m.takedown` against an entity are irreversible, and can have a huge impact on the history of a Matrix room if implemented naively. The most common use case for the recommendation will be to replace the use of `m.ban` with the reason `spam` targeting a user, which in - Mjolinr causes the target user to be banned from protected rooms and + Mjolnir causes the target user to be banned from protected rooms and + Because the `reason` is no longer present, a higher degree of trust is required when applying some consequences to these policies. An @@ -127,7 +127,7 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone + The lack of classification from the `reason` field is insufficient to prevent direct identification of some entities. For example `@yarrgh:example.com` is evidently a pirate because of the presence - of the phrase `yarrgh` embeded within their mxid. An attempt to + of the phrase `yarrgh` embedded within their mxid. An attempt to alleviate this concern exists through [MSC4205 Hashed moderation policy entities](https://github.com/matrix-org/matrix-spec-proposals/pull/4205). @@ -138,14 +138,14 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone because the reasons are not standardised. + `m.ban` A `takedown` reason could be given on `m.ban` policies. - This is pretty adhoc. + This is pretty ad-hoc. ## Security considerations + `m.takedown` can have severe consequences for entities, policies could be created maliciously against innocent users on popular - policy rooms. This could increasce the reward of infiltrating + policy rooms. This could increase the reward of infiltrating moderation focussed communities. Tools that naively implement the recommendation without safeguards, such as manual approval, for the most severe consequences could be exploited. From 670f48a9d8e069c919b3a55c3d1a8cb6bd645cf4 Mon Sep 17 00:00:00 2001 From: gnuxie Date: Wed, 2 Oct 2024 16:26:09 +0100 Subject: [PATCH 5/5] typos from another reading. --- proposals/4204-takedown-recommendation.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4204-takedown-recommendation.md b/proposals/4204-takedown-recommendation.md index 546062718ba..eefc79ec4de 100644 --- a/proposals/4204-takedown-recommendation.md +++ b/proposals/4204-takedown-recommendation.md @@ -35,7 +35,7 @@ and they are not a pirate at all. Additionally, if `@yarrgh:example.com` is known to been smuggling contraband via matrix events or media, then there is no mechanism -to mark this content as plunder that should be ceased. +to mark this content as plunder that should be seized. A compounding issue is Moderation bots currently use the `reason` of the ban to determine whether to redact a user's events or not. Which @@ -53,7 +53,7 @@ expected to be consumed by typical moderation tooling or clients. This is to set expectations for policy list curators for how the recommendation should behave and the severity of the consequences. We also provide guidance to allow tool implementers to understand the -context of the recommendations use. Moderation tools or clients are +context of the recommendation's use. Moderation tools or clients are free to interpret the recommendation differently, for example if they are aware of a conflicting policy or they are configured to implement harsher or more liberal consequences for the recommendation. @@ -106,7 +106,7 @@ If the entity of the rule is a server: quarantined or removed. -[^bluesky] This is inspired by work going into bluesky +[^bluesky]: This is inspired by work going into bluesky https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone ## Potential issues @@ -117,6 +117,7 @@ https://docs.bsky.app/blog/2024-protocol-roadmap#protocol-stability-milestone most common use case for the recommendation will be to replace the use of `m.ban` with the reason `spam` targeting a user, which in Mjolnir causes the target user to be banned from protected rooms and + their messages to be redacted. + Because the `reason` is no longer present, a higher degree of trust is required when applying some consequences to these policies. An