-
Notifications
You must be signed in to change notification settings - Fork 2
Audits (basic info)
Basic Information about audits
- Moab to Catalog (M2C) existence/version check
- Catalog to Moab (C2M) existence/version check
- Checksum Validation (CV)
- Catalog to Archive (C2A) also known as replication audit
- (TBD) Moab Validation (which is in the moab-versioning gem, but called by PreservationCatalog)
For each object encountered on a storage_root, performs Moab Validation (moab-versioning
gem code confirming the Moab object is consistent with itself - has correct manifests and versions, etc.) and ensures the Preservation Catalog db has correct information for a Moab (e.g. which version), etc.
See:
- app/models/moab_stroage_root.rb (method
m2c_check!
) - app/jobs/moab_to_catalog_job.rb
- lib/audit/preserved_object_handler.rb (method
check_existence
in particular) - this has moved - lib/services/moab_validation_handler.rb (method
set_status_as_seen_on_disk
in particular) - this has moved
- cron job via
whenever
gem, seeconfig/schedule.rb
; M2C is triggered for every Moab found on storage roots. - can be triggered manually: https://github.com/sul-dlss/preservation_catalog/#moab-to-catalog-m2c-existenceversion-check
For each CompleteMoab object in PreservationCatalog, ensures the prescat db has correct information for the Moab (e.g. which version) and also performs Moab Validation (moab-versioning
gem code confirming the Moab object is consistent with itself - has correct manifests and versions, etc.).
See:
- app/models/moab_stroage_root.rg (method
c2m_check!
) - app/jobs/catalog_to_moab_job.rb
- lib/audit/catalog_to_moab.rb
- lib/services/moab_validation_handler.rb (method
set_status_as_seen_on_disk
in particular) - this has moved
- cron job via
whenever
gem, seeconfig/schedule.rb
; C2M is triggered for every Moab found on storage roots. - if CV is triggered, and Moab passes CV (does not have
invalid_checksum
status), then C2M will be triggered. - can be triggered manually: https://github.com/sul-dlss/preservation_catalog/#catalog-to-moab-c2m-existenceversion-check
Fixity checking of the Moab. Parse all manifestInventory.xml
and most recent signatureCatalog.xml
for stored checksums and verify against computed checksums.
NOTE: if CV is triggered, and Moab passes CV (does not have invalid_checksum
status), then C2M will be triggered.
See:
- app/models/moab_stroage_root.rg (method
validate_expired_checksums!
) - app/jobs/checksum_validation_job.rb
- app/services/checksum_validator.rb - this has moved
- cron job via
whenever
gem, seeconfig/schedule.rb
; it looks for "expired"last_checksum_validation
dates for CompleteMoabs and re-runs CV on them. - can be triggered manually: https://github.com/sul-dlss/preservation_catalog/#checksum-validation-cv
Checks that all parts of zip file (replicated Moab objects are zipped and split into parts as necessary) have been replicated.
- cron job via
whenever
gem, seeconfig/schedule.rb
; it looks for "expired"last_archive_audit
dates for CompleteMoabs and re-runs C2A on them.
if CV is triggered, and Moab passes CV (does not have invalid_checksum
status), then C2M will be triggered.
- Replication errors
- Validate moab step fails during preservationIngestWF
- ZipmakerJob failures
- Moab Audit Failures
- Ceph Errors
- Job queues
- Deposit bag was missing
- ActiveRecord and Replication intro
- 2018 Work Cycle Documentation
- Fixing a stuck Moab
- Adding a new cloud provider
- Audits (how to run as needed)
- Extracting segmented zipfiles
- AWS credentials, S3 configuration
- Zip Creation
- Storage Migration Additional Information
- Useful ActiveRecord queries
- IO against Ceph backed preservation storage is hanging indefinitely (steps to address IO problems, and follow on cleanup)