-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[server][dvc] Open store before bootstrapping from blob transfer. #1290
Conversation
@@ -79,6 +80,7 @@ public void startConsumption(VeniceStoreVersionConfig storeConfig, int partition | |||
if (!storeAndVersion.getFirst().isBlobTransferEnabled() || blobTransferManager == null) { | |||
runnable.run(); | |||
} else { | |||
storageService.openStore(storeConfig, svsSupplier); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@FelixGV
I saw you modified this interface to pass svsSupplier
in the past:
#44
For the usage in this class, it is a little confusing:
Supplier<StoreVersionState> svsSupplier = () -> storageMetadataService.getStoreVersionState(storeVersion);
syncStoreVersionConfig(storeAndVersion.getFirst(), storeConfig);
Runnable runnable = () -> {
AbstractStorageEngine storageEngine =
storageService.openStoreForNewPartition(storeConfig, partition, svsSupplier);
So it first looks up SVS state from the metadata partition and then pass it to StorageService#openStoreForNewPartition
, I wondered whether it is necessary to use this pattern as internally if versionStateCache
is null
in the get path, it will try to do the lookup internally.
My main question is that whether this param: svsSupplier
is necessary or not in openStore
and openStoreForNewPartition
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the user can pass a null
supplier for such usage, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code change itself looks good and I left an open-question to @FelixGV .
Summary
Open the store before bootstrapping from blob transfer, otherwise the offset record and StorageVersionState can not update successfully.
How was this PR tested?
Run at 2 dvc hosts.
Does this PR introduce any user-facing changes?