-
Notifications
You must be signed in to change notification settings - Fork 2
Fixing an Incomplete Moab Upload
Johnathan Martin edited this page May 21, 2020
·
13 revisions
A large Moab (>10GB) has only uploaded a portion of its archive files. The sdr-transfer volume ran out of space during transfers/upload. The parts_count value of the uploaded segments is incorrect.
- Delete all remaining pieces of the Moab from /sdr-transfers. (can be done by dev or ops from any of pres cat prod worker boxes, i.e. any other than -01)
- Delete any pieces of the failed Moab-version upload that made it to an endpoint from that one endpoint. (ops task)
- Delete & re-create the ZMV of the failing Moab from PresCat. (dev task, see below)
[1] pry(main)> zmv = ZippedMoabVersion.by_druid(druid).where(zip_endpoint: ZipEndpoint.where(endpoint_name: endpoint_name)).find_by!(version: version)
[2] pry(main)> zmv.destroy
[3] pry(main)> CompleteMoab.by_druid('dc156hp0190').find_each(&:create_zipped_moab_versions!)
This forces PresCat to re-zip that version and correctly record metadata.
- Replication errors
- Validate moab step fails during preservationIngestWF
- ZipmakerJob failures
- Moab Audit Failures
- Ceph Errors
- Job queues
- Deposit bag was missing
- ActiveRecord and Replication intro
- 2018 Work Cycle Documentation
- Fixing a stuck Moab
- Adding a new cloud provider
- Audits (how to run as needed)
- Extracting segmented zipfiles
- AWS credentials, S3 configuration
- Zip Creation
- Storage Migration Additional Information
- Useful ActiveRecord queries
- IO against Ceph backed preservation storage is hanging indefinitely (steps to address IO problems, and follow on cleanup)