Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
derived_data_service: make enqueuing items into the derivation queue …
…resilient against races Summary: When enqueuing an item into the derivation queue, the item being inserted might represent a batch of changesets that share some changesets with another batch that's simultaneously being inserted e.g.: ``` a d \ / b | c First batch: [a, b, c] Second batch: [d, b, c] ``` In this case one of the two batches will lose the race and will fail to get enqueued in the derivation queue. If the second batch failed to get enqueued while the first batch was still in the queue, then the logic in `build_underived_batched_graph` will handle this correctly by creating a new batch that includes only changesets that are in the second batch but not in the first and trying to enqueue it. There's however another case that's not currently handled correctly which is that the first batch was processed and was partially removed from the queue before the attempt to enqueue the second batch. In this case the enqueue attempt will fail but we will not find the first batch in the queue so this will count as a failure and we will retry enqueuing the same batch and potentially hit the same failure again. Instead let's check if the batch was partially derived and in that case try to enqueue only the part of the batch that was not derived and not count this case as a failure. Reviewed By: singhsrb Differential Revision: D64899662 fbshipit-source-id: c45188061de0ca24397eade4fc790154ceef4748
- Loading branch information