Concurrency: Changes Synchronization & Deletion should be atomic

[Zhiyuan-Amos]

Changes Synchronization & Deletion can technically be executed concurrently by the same user. This shouldn't be occurring as there is no use case for such a scenario. Also, this can lead to race conditions. For example, when executing Synchronization, which contains the following logic

Get all Changes by UserId
Process Changes
Delete Changes

If Process 1 is completes Delete Changes and Process 2 runs Process Changes, then double processing will be executed. Retrieval from external sources such as Images might fail as these resources might have been deleted already.

Let's add a new container Synchronizing, which contains {userId, state}. Both id & partition key will use userId.

1. When Synchronization begins, add the userId with state isSynchronizing to Synchronizing. If an entry with the userId exists, the operation fails.
2. Run Synchronization.
3. On Synchronization complete, set existing state to pendingDeletion.
4. When Deletion begins, set existing state to isDeleting. If an entry with the userId exists and the state is not pendingDeletion, or if an entry does not exist, the operation fails.
5. On Deletion complete, delete entry.

To handle the scenario where either the Client or Server errors out, let's set ttl to 30s for auto-timeout. A race condition might occur if the ttl expires too quickly before the operation completes. To mitigate this, let's update ttl periodically while Synchronization is occurring.