You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We need to adapt the GcpUploader implementation to support uploading of immutable file archives only if they don't already exist in the bucket. We also need to make sure that these archives are not deleted after 14 days by the GCP bucket life-cycle rules implemented for the bucket
What
Have GcpUploader support an overwriting feature (simple, no hash comparison at first) and adapt the GCP bucket life-cycle rules.
How
Add an overwriting option in the GcpUploader which avoids uploading a file that already exists remotely (check on existence only)
Add a destination folder option in the GcpUploader which specifies the folder in the GCP bucket where the file is uploaded
Modify the dependency injection of the GCP uploaders to support the new options
Upload the CardanoImmutableFilesFull snapshot archive to a cardano-immutable-files-full folder
Upload the CardanoDatabase ancillary archives to a cardano-database/ancillary folder
Upload the CardanoDatabase immutable archives to a cardano-database/immutable folder
Upload the CardanoDatabase digests file to a cardano-database/digests folder
Update terraform scripts to support prefixes of files deleted after 14 days
Later
Extend the overwriting option to verify the hash of the local file vs remote file
Manual clean of the non deleted archives located at the root of the bucket 14 days after deployment
The text was updated successfully, but these errors were encountered:
Why
We need to adapt the
GcpUploader
implementation to support uploading of immutable file archives only if they don't already exist in the bucket. We also need to make sure that these archives are not deleted after 14 days by the GCP bucket life-cycle rules implemented for the bucketWhat
Have
GcpUploader
support an overwriting feature (simple, no hash comparison at first) and adapt the GCP bucket life-cycle rules.How
GcpUploader
which avoids uploading a file that already exists remotely (check on existence only)GcpUploader
which specifies the folder in the GCP bucket where the file is uploadedCardanoImmutableFilesFull
snapshot archive to acardano-immutable-files-full
folderCardanoDatabase
ancillary archives to acardano-database/ancillary
folderCardanoDatabase
immutable archives to acardano-database/immutable
folderCardanoDatabase
digests file to acardano-database/digests
folderLater
The text was updated successfully, but these errors were encountered: