Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CADC-13234 fix ringhold for ArtifactDAO API change #564

Merged
merged 7 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ jobs:
- name: java build -- ratik
run: cd ratik && ../gradlew --info clean build javadoc checkstyleMain

## disabled until updated for cadc-inventory-db API changes
# - name: java build -- ringhold
# run: cd ringhold && ../gradlew --info clean build javadoc checkstyleMain
- name: java build -- ringhold
run: cd ringhold && ../gradlew --info clean build javadoc checkstyleMain

25 changes: 17 additions & 8 deletions ringhold/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Storage Inventory local artifact removal process (ringhold)
# Storage Inventory local artifact deletion process (ringhold)

Process to remove local artifacts that are no longer being synchronised by fenwick. This tool is used
to perform quick cleanup at a storage site after changing the fenwick artifact-filter policy.
Process to remove the local copy of artifacts from a storage site inventory database and
generate DeletedStorageLocationEvent(s) so the removal will propagate correctly to a global inventory.
This does not remove the files from storage (see `tantar`).

## configuration
See the [cadc-java](https://github.com/opencadc/docker-base/tree/master/cadc-java) image docs for general config requirements.
Expand All @@ -18,17 +19,25 @@ org.opencadc.ringhold.inventory.schema={schema for inventory database objects}
org.opencadc.ringhold.inventory.username={username for inventory admin}
org.opencadc.ringhold.inventory.password={password for inventory admin}
org.opencadc.ringhold.inventory.url=jdbc:postgresql://{server}/{database}

# artifact namespace(s) to remove
org.opencadc.ringhold.namespace={storage site namespace}

# artifact uri bucket filter (optional)
org.opencadc.ringhold.buckets={uriBucket prefix or range of prefixes}
```
The `inventory` account owns and manages all the content (insert, update, delete) in the inventory schema. Unlike
other components that modify inventory content, this component **does not initialise** the database objects because
it never makes sense to run this in a new/empty database. The database is specified in the JDBC URL. Failure to
connect to a pre-initialised database will show up in logs.

### artifact-deselector.sql
Contains a SQL clause used as a WHERE constraint. The clause returns Artifact's that match the URI pattern.
```
WHERE uri LIKE 'cadc:CFHT/%'
```
The `namespace` is the prefix of the Artifact URI's to be deleted. The `namespace` must end with a colon (:)
or slash (/) so one namespace cannot accidentally match (be a prefix of) another namespace. Multiple values
of `namespace` may be specified, one per line.

The `buckets` value indicates a subset of artifacts to delete. The range of uri bucket prefixes is specified
with two values separated by a single - (dash) character; whitespace is ignored. Multiple instances of `ringhold`
can be run (in parallel) to subdivide the work as long as the range of buckets do not overlap.

## building it
```
Expand Down
2 changes: 1 addition & 1 deletion ringhold/VERSION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## deployable containers have a semantic and build tag
# semantic version tag: major.minor[.patch]
# build version tag: timestamp
VER=0.2.0
VER=0.3.0
TAGS="${VER} ${VER}-$(date --utc +"%Y%m%dT%H%M%S")"
unset VER
Loading
Loading