Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data size synchronization worker (CADC 13214) #561

Merged
merged 6 commits into from
Mar 18, 2024

Conversation

andamian
Copy link
Contributor

No description provided.

node.bytesUsed = artifact.getContentLength();
tm.startTransaction();
try {
nodeDAO.put(node);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inside the txn but before the put(node), this should lock the the node before updating in case state changed since the getDataNode call.

The lock method returns another instance of the node that might be different from getDataNode if another process also updated it (eg NodePersistenceImpl.get(...) if we do that "cache update " or validation) and it might return null if the node was deleted... all part of race condition handling (but not easily testable without a lot of effort that's not worth it). Once you have started and txn and locked the node, there is not even much point in rechecking the bytesUsed: just assign the value and put. The metaChecksum will optimize whether the update is needed or not.


for harvestStateDAO.put(...) you do not need the second argument to force the timestamp update. Here it should just rely on changing the fields of the object.

Logically, the harvestStateDAO.put(...) needs to happen for every artifact event handled, not just the ones that cause an update so it needs to go outside the if (node size needs update). That's OK because the harvestStateDAO.put(...) call can happen outside the transaction (it risks re-evaluating an artifact event, which is idempotent so harmless). Because I will probably configure the HarvestStateDAO to buffer updates and possibly even run the maintenance it must be outside the transaction because that happens inside the put().

@pdowler pdowler merged commit 742bc6b into opencadc:vos2 Mar 18, 2024
1 check failed
@andamian andamian deleted the CADC-13214 branch July 9, 2024 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants