Tco17 historical snapshots aggregations #49
Merged
Commits on Jul 10, 2024
-
Update dev debugging and fixup issn detection
Why are these changes being introduced: * The ISSN detection was returning a key set to `nil` when an ISSN-like pattern (probably a year span) was detected but it failed the ISSN validation checksum. This differed from other standard_identifiers and caused issues when using the standard_identifiers detection for our montly statistics. This just deletes the key instead of setting it to `nil` in conditions when the checksum fails. * Our development logging wasn't showing a useful loglevel for development. I set it to debug here, but we may want to allow it to be controlled by ENV and also consider using lograge like we do on other apps
-
Adds support for filtering SearchEvents to a month
Why are these changes being introduced: * The ability to filter to a specific month is a feature needed for the montly aggregation of algorithm matches Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/TCO-17 How does this address that need: * Creates a scope on the model
-
Why are these changes being introduced: * Implement data models for counting algorithm matches for the current month Terms Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/TCO-17 See also: * https://github.com/MITLibraries/tacos/blob/main/docs/architecture-decisions/0005-use-multiple-minimal-historical-analytics-models.md How does this address that need: * Creates a new model `MontlyMatch` * Adds methods to run each (current) StandardIdentifier algorithm on each Term (via the SearchEvents) Document any side effects to this change: * A schedulable job to run this automatically is out of scope and will be added under a separate ticket
-
Add parallel test support to coverage
Why are these changes being introduced: * We'll hit 50 tests in the next commit which is the Rails default before using parallel tests. How does this address that need: * This updates the coverage config to work with parallel tests.
-
Why are these changes being introduced: * Implement data models for counting algorithm matches for all Terms Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/TCO-17 See also: * https://github.com/MITLibraries/tacos/blob/main/docs/architecture-decisions/0005-use-multiple-minimal-historical-analytics-models.md How does this address that need: * Creates a new model `AggregateMatch` * Adds methods to run each (current) StandardIdentifier algorithm on each Term (via the SearchEvents) * Adjusts `MontlyMatch` counting algorithm to be useful for both cases and extracts it to a module which is imported into both Classes Document any side effects to this change: * A schedulable job to run this automatically is out of scope and will be added under a separate ticket * The tests are identical between this and `MontlyMatch`. There may be a way to avoid the duplication and thus ensure both get relevant updates but it was not clear to me how to do that in an obvious way at the time of this work.
-
-
Refactors to Metrics::Algorithms
Moves both types of metrics on algorithms to use a single class and associated database table. Creating the namespace `Metrics` should allow us to better organize code associated with various metrics into a common space that is both intuitive and useful. Relevant ticket(s): https://mitlibraries.atlassian.net/browse/TCO-17
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.