Skip to content

Commit

Permalink
derived_data_utils: Fix duplicated derived data dependency graph
Browse files Browse the repository at this point in the history
Summary:
Recently, we had a report from AOSP that they saw this error:
```
D0314 14:55:17.055516   307 fbcode/scm/grepo/branch_forest/src/derive_data.rs:98] derived graph in 14.239055403s: Err(failed to derive bssm_v3 batch (start:9ff0c1071da6397408981fab35b69323c8f2900fc76b4b323bdb7edace0dc849, end:7b9d79e5f2ab5dacb04e7c7a2e0354163be485e98c6f726978d2cba960cc421c)
Caused by:
    dependency 'skeleton_manifests' of 66a9c86eb6dc728992307e5f7d83bb54c290246405b508f9dabaa3ae30f1c11a was not already derived
```

Full trace: P1196413182

At the moment, `branch_forest` uses the old "derived_data/utils" interface to interact with data derivation.
This code duplicates the dependency graph that is established at compile time in the definition of each derivable data type and performs runtime traversals.

For this reason, it is error prone and we want to use recent refactorings in derived data to replace it with the `BulkDerivation` interface and delete it.

We are not quite there in terms of refactorings, yet, so for now patch the bug in `derived_data/utils`.

This is the source of truth for this particular derived data dependency: https://www.internalfb.com/code/fbsource/[97d3f14298ef3390a03e4f5051b093f9813f119d]/fbcode/eden/mononoke/derived_data/basename_suffix_skeleton_manifest_v3/mapping.rs?lines=75

Wihout this, the ordering of derivations won't deterministically ensure we derive `skeleton_manifests` before `bssm_v3`, so we can hit a race condition that results in the error observed in prod above.

Reviewed By: RajivTS

Differential Revision: D54949896

fbshipit-source-id: b8c5bbe28e9754845b79a3a71f1f9ee2ab239cb0
  • Loading branch information
Pierre Chevalier authored and facebook-github-bot committed Mar 15, 2024
1 parent 482c5ec commit 096a0cd
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion eden/mononoke/derived_data/utils/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ lazy_static! {
dag.insert(fsnodes, vec![]);
dag.insert(deleted_mf_v2, vec![unodes]);
dag.insert(skeleton_mf, vec![]);
dag.insert(bssm_v3, vec![]);
dag.insert(bssm_v3, vec![skeleton_mf]);
dag.insert(git_tree, vec![]);
dag.insert(git_commit, vec![git_tree]);
dag.insert(git_delta_manifest, vec![git_tree, git_commit, unodes]);
Expand Down

0 comments on commit 096a0cd

Please sign in to comment.