Admin Purge history does not purge last message

[gfiury]

Description

Calling the Admin Purge history API of a room, does not purge all messages, it leaves the last message behind.

Steps to reproduce

4 Messages written on the room
hi
hello
bye
this is the last message

Then Call
https://homeserver/_synapse/admin/v1/purge_history/<room_id>

With a greater ts than the messages
Body

{
     "delete_local_events": true,
    "purge_up_to_ts": 1582827526998
}

The last message of the room is not purged. Even if you try to invoke the API with a greater ts, it does not work.

Checking the logs

2020-02-27 18:15:38,200 - synapse.storage.data_stores.main.events - 1498 - INFO - POST-1384 - [purge] looking for events to delete
2020-02-27 18:15:38,203 - synapse.storage.data_stores.main.events - 1539 - INFO - POST-1384 - [purge] found 4 events before cutoff, of which 4 can be deleted
2020-02-27 18:15:38,203 - synapse.storage.data_stores.main.events - 1542 - INFO - POST-1384 - [purge] Finding new backward extremities
2020-02-27 18:15:38,204 - synapse.storage.data_stores.main.events - 1555 - INFO - POST-1384 - [purge] replacing backward extremities: [('$EZ54ZeVy4H5bYxUVpLpmJkeDrgvKebqT95TH7w59WI0',)]
2020-02-27 18:15:38,205 - synapse.storage.data_stores.main.events - 1568 - INFO - POST-1384 - [purge] finding state groups referenced by deleted events
2020-02-27 18:15:38,206 - synapse.storage.data_stores.main.events - 1581 - INFO - POST-1384 - [purge] found 1 referenced state groups
2020-02-27 18:15:38,206 - synapse.storage.data_stores.main.events - 1584 - INFO - POST-1384 - [purge] removing events from event_to_state_groups
2020-02-27 18:15:38,207 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from events
2020-02-27 18:15:38,207 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_json
2020-02-27 18:15:38,208 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_auth
2020-02-27 18:15:38,209 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_edges
2020-02-27 18:15:38,210 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_forward_extremities
2020-02-27 18:15:38,211 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_reference_hashes
2020-02-27 18:15:38,211 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from event_search
2020-02-27 18:15:38,212 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1384 - [purge] removing events from rejections
2020-02-27 18:15:38,213 - synapse.storage.data_stores.main.events - 1614 - INFO - POST-1384 - [purge] removing events from event_push_actions
2020-02-27 18:15:38,214 - synapse.storage.data_stores.main.events - 1624 - INFO - POST-1384 - [purge] marking remaining events as outliers
2020-02-27 18:15:38,215 - synapse.storage.data_stores.main.events - 1657 - INFO - POST-1384 - [purge] updating room_depth to 43
2020-02-27 18:15:38,217 - synapse.storage.data_stores.main.events - 1668 - INFO - POST-1384 - [purge] done

It says "found 4 events before cutoff", so it is correct, but it does not delete 4th. The stream_ordering is fine, because it finds the 4 events.

Calling again with a greater ts the logs shows

2020-02-27 18:18:58,459 - synapse.storage.data_stores.main.events - 1498 - INFO - POST-1444 - [purge] looking for events to delete
2020-02-27 18:18:58,462 - synapse.storage.data_stores.main.events - 1539 - INFO - POST-1444 - [purge] found 0 events before cutoff, of which 0 can be deleted
2020-02-27 18:18:58,462 - synapse.storage.data_stores.main.events - 1542 - INFO - POST-1444 - [purge] Finding new backward extremities
2020-02-27 18:18:58,463 - synapse.storage.data_stores.main.events - 1555 - INFO - POST-1444 - [purge] replacing backward extremities: []
2020-02-27 18:18:58,463 - synapse.storage.data_stores.main.events - 1568 - INFO - POST-1444 - [purge] finding state groups referenced by deleted events
2020-02-27 18:18:58,464 - synapse.storage.data_stores.main.events - 1581 - INFO - POST-1444 - [purge] found 0 referenced state groups
2020-02-27 18:18:58,464 - synapse.storage.data_stores.main.events - 1584 - INFO - POST-1444 - [purge] removing events from event_to_state_groups
2020-02-27 18:18:58,465 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from events
2020-02-27 18:18:58,465 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_json
2020-02-27 18:18:58,466 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_auth
2020-02-27 18:18:58,467 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_edges
2020-02-27 18:18:58,467 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_forward_extremities
2020-02-27 18:18:58,468 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_reference_hashes
2020-02-27 18:18:58,480 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from event_search
2020-02-27 18:18:58,481 - synapse.storage.data_stores.main.events - 1603 - INFO - POST-1444 - [purge] removing events from rejections
2020-02-27 18:18:58,481 - synapse.storage.data_stores.main.events - 1614 - INFO - POST-1444 - [purge] removing events from event_push_actions
2020-02-27 18:18:58,482 - synapse.storage.data_stores.main.events - 1624 - INFO - POST-1444 - [purge] marking remaining events as outliers
2020-02-27 18:18:58,484 - synapse.storage.data_stores.main.events - 1657 - INFO - POST-1444 - [purge] updating room_depth to 0
2020-02-27 18:18:58,485 - synapse.storage.data_stores.main.events - 1668 - INFO - POST-1444 - [purge] done
2020-02-27 18:18:58,497 - synapse.storage.purge_events - 57 - INFO - POST-1444 - [purge] finding state groups that can be deleted
2020-02-27 18:18:58,498 - synapse.storage.data_stores.state.store - 511 - INFO - POST-1444 - [purge] found 0 state groups to delete
2020-02-27 18:18:58,498 - synapse.storage.data_stores.state.store - 531 - INFO - POST-1444 - [purge] de-delta-ing 0 remaining state groups
2020-02-27 18:18:58,498 - synapse.storage.data_stores.state.store - 564 - INFO - POST-1444 - [purge] removing redundant state groups
2020-02-27 18:18:58,499 - synapse.handlers.pagination - 252 - INFO - POST-1444 - [purge] complete

The stream_ordering of the two calls are the same.

Version information

Synapse: 1.9.1

[richvdh]

yes, this is documented at https://github.com/matrix-org/synapse/blob/master/docs/admin_api/purge_history_api.rst:

Note that Synapse requires at least one message in each room, so it will never delete the last message in a room.

[richvdh]

(If you want to completely remove a room, consider the purge_room api)