Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an Admin API endpoint to redact all a user's events #17506

Merged
merged 15 commits into from
Sep 18, 2024

Conversation

H-Shay
Copy link
Contributor

@H-Shay H-Shay commented Jul 30, 2024

Adds an Admin API to redact all the events of a given user in the provided rooms. If an empty dict is provided, all the events of user in all the rooms they are a member of will be redacted.

@H-Shay H-Shay requested a review from a team as a code owner July 30, 2024 21:55
@H-Shay H-Shay changed the title Add Admin API to redact all a user's events Add an Admin API endpoint to redact all a user's events Jul 30, 2024
@github-actions github-actions bot deployed to PR Documentation Preview July 30, 2024 21:56 Active
@github-actions github-actions bot deployed to PR Documentation Preview July 30, 2024 21:57 Active
@github-actions github-actions bot deployed to PR Documentation Preview July 30, 2024 22:29 Active
@MatMaul
Copy link
Contributor

MatMaul commented Jul 31, 2024

This looks like a potentially long process, you should probably use the resumable task scheduler so that it survives a restart, and move to an async friendly API, similarly to the delete room v2 endpoint.

@MatMaul
Copy link
Contributor

MatMaul commented Jul 31, 2024

Also feel free to ping me in Matrix (@mat:tout.im) if you need some clarity on how the resumable task scheduler work.

@H-Shay
Copy link
Contributor Author

H-Shay commented Aug 1, 2024

This looks like a potentially long process, you should probably use the resumable task scheduler

Great point thanks for the tip - I've gone ahead and done so :)

@github-actions github-actions bot deployed to PR Documentation Preview August 1, 2024 17:49 Active
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
tests/rest/admin/test_user.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
synapse/rest/admin/users.py Outdated Show resolved Hide resolved
synapse/rest/admin/users.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Show resolved Hide resolved
docs/admin_api/user_admin_api.md Show resolved Hide resolved
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty good so far, thanks for continuing to make sysadmin's lives easier!

synapse/rest/admin/users.py Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/rest/admin/users.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
@anoadragon453
Copy link
Member

sidenote: MSC4084: Mass redactions would make this a lot more efficient.

@github-actions github-actions bot deployed to PR Documentation Preview August 20, 2024 19:43 Active
@H-Shay
Copy link
Contributor Author

H-Shay commented Aug 20, 2024

complement failures are flakey partial-state join tests - unrelated

synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the impersonation of the event sender explored before merging this PR, as I agree with @MatMaul that you're likely to need that functionality to redact the majority of a user's messages. Perhaps start with a test with a room that contains the user, but not the admin requester, and check whether their messages are redacted.

I'd also like to see an allowlist of event types to redact. I don't think it makes sense to redact state events that a user has sent, for instance (except in the case of m.room.member joins, which could contain offensive display name data).

synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
@github-actions github-actions bot deployed to PR Documentation Preview August 29, 2024 20:27 Active
@H-Shay
Copy link
Contributor Author

H-Shay commented Aug 29, 2024

Right so I think this is ready for another look. My only concern is the filter mechanism for event types is a bit noddy, any suggestions on how to make that more graceful (or if it matters) are appreciated.

@H-Shay
Copy link
Contributor Author

H-Shay commented Sep 6, 2024

@anoadragon453 just wondering if the request review notif got lost? This should be ready for another go-around and is hopefully very close to done.

Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it looks like I got halfway through a review here and then got distracted :|

This PR is close! Just a few things now.

synapse/handlers/admin.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
synapse/rest/admin/users.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
tests/rest/admin/test_user.py Outdated Show resolved Hide resolved
@github-actions github-actions bot deployed to PR Documentation Preview September 11, 2024 21:14 Active
@github-actions github-actions bot deployed to PR Documentation Preview September 11, 2024 21:46 Active
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple things now, but overall this is looking nearly ready to go!

docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
if content:
if content.get("membership") == "join":
if content.get("membership") == "Membership.JOIN":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh err, Membership.JOIN is a type. Looks like we may be missing a test for this codepath? 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right I have switched it back and ensured that the tests are verifying that we are redacting the join event.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the tests! Using Membership.JOIN (without quotes) should work though? It's equivalent to "join", but uses a Final instead of a loose string.

@github-actions github-actions bot deployed to PR Documentation Preview September 12, 2024 19:38 Active
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small things now, lgtm otherwise!

tests/rest/admin/test_user.py Outdated Show resolved Hide resolved
tests/rest/admin/test_user.py Outdated Show resolved Hide resolved
if content:
if content.get("membership") == "join":
if content.get("membership") == "Membership.JOIN":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the tests! Using Membership.JOIN (without quotes) should work though? It's equivalent to "join", but uses a Final instead of a loose string.

synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
synapse/storage/databases/main/events_worker.py Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
synapse/handlers/admin.py Outdated Show resolved Hide resolved
Comment on lines 1437 to 1438
reason = body.get("reason")
limit = body.get("limit")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you can use parse_integer_from_args and parse_string_from_args to ensure the type of these arguments. Otherwise we may start generating events with a non-string reason field.

It's also a good idea to verify that limit is not 0 or negative.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly confused as this function seems to parse an integer from a request string, and I am trying to get the values from the request's json object - am I missing something here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oop, you have every right to be confused. Those functions indeed won't be helpful. Could you manually add the mentioned checks instead?

changelog.d/17506.feature Outdated Show resolved Hide resolved
@anoadragon453
Copy link
Member

This just needs a couple very minor points addressed, and then it's good to go: #17506 (comment), #17506 (comment)

@github-actions github-actions bot deployed to PR Documentation Preview September 17, 2024 19:50 Active
@H-Shay
Copy link
Contributor Author

H-Shay commented Sep 17, 2024

riiight sorry for the delay - I think this is ready now.

Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, this LGTM! Thanks for your patience, this is a really welcome change :)

docs/admin_api/user_admin_api.md Outdated Show resolved Hide resolved
@anoadragon453 anoadragon453 enabled auto-merge (squash) September 18, 2024 09:45
@github-actions github-actions bot deployed to PR Documentation Preview September 18, 2024 09:46 Active
@anoadragon453 anoadragon453 merged commit 51dd4df into element-hq:develop Sep 18, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants