Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate cost metric #471

Merged
merged 6 commits into from
Jul 13, 2023
Merged

Aggregate cost metric #471

merged 6 commits into from
Jul 13, 2023

Conversation

kathy-t
Copy link
Contributor

@kathy-t kathy-t commented Jun 28, 2023

Description
This PR aggregates the cost metric.

Companion PRs:

Review Instructions

  • Submit a run execution metric that specifies a cost for a workflow. Read this doc for more info if you're unfamiliar with submitting metrics. You must be a curator or admin to be able to submit metrics.
  • Perform a release of the metrics aggregator so that the new JAR is in artifactory. View the wiki for how to do this
    • Note to self: release after this is merged so reviewing is easier for the verifier. EDIT: I released the 1.15.0-alpha.6 version of the metrics aggregator on artifactory to be used to verify this PR. It can be found here
  • SSM onto the deployer and wget the metrics aggregator JAR from artifactory using the following command:
    wget https://artifacts.oicr.on.ca/artifactory/collab-release/io/dockstore/metricsaggregator/1.15.0-alpha.6/metricsaggregator-1.15.0-alpha.6.jar
    
  • Add the Deployer instance's public IP to the WAF's exempted IP set so that the Deployer doesn't get rate limited when running the aggregate-metrics command.
    • Navigate to the EC2 console and click on the Deployer instance. In the Details tab, copy the Public IPv4 address value. Remember this value because you'll be deleting this IP once we're done with the metrics aggregator.
    • Go to the WAF console and click on IP sets on the left hand menu. Make sure that you're in the us-east-2 region.
    • Click on qa-core-ExemptedIPSet and add the Deployer's IP to this set. You may need to append /32 to the end of the IP address
  • Aggregate the metrics. View the README for instructions on setting up the config file and running the aggregator.
  • Go to the workflow version that you submitted metrics for in the UI and enable the metrics flag. Verify that the cost metric shows up.
  • Remove the Deployer's IP address from the WAF's exempted IP set when done verifying.

Issue
SEAB-5625

Security
If there are any concerns that require extra attention from the security team, highlight them here.

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that you pass the basic style checks and unit tests by running mvn clean install in the project that you have modified (until https://ucsc-cgl.atlassian.net/browse/SEAB-5300 adds multi-module support properly)
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • If you are changing dependencies, check with dependabot to ensure you are not introducing new high/critical vulnerabilities
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

@kathy-t kathy-t self-assigned this Jun 28, 2023
@codecov
Copy link

codecov bot commented Jun 28, 2023

Codecov Report

Patch coverage: 94.91% and project coverage change: +3.70 🎉

Comparison is base (cdb0c1c) 38.96% compared to head (62c9e7c) 42.66%.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #471      +/-   ##
=============================================
+ Coverage      38.96%   42.66%   +3.70%     
- Complexity       143      172      +29     
=============================================
  Files             22       24       +2     
  Lines           1481     1596     +115     
  Branches         128      131       +3     
=============================================
+ Hits             577      681     +104     
- Misses           868      877       +9     
- Partials          36       38       +2     
Flag Coverage Δ
metricsaggregator 42.66% <94.91%> (+3.70%) ⬆️
toolbackup 32.33% <92.09%> (+5.32%) ⬆️
tooltester 32.33% <92.09%> (+5.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ava/io/dockstore/metricsaggregator/Statistics.java 82.97% <84.61%> (-17.03%) ⬇️
...re/metricsaggregator/helper/AggregationHelper.java 92.74% <96.29%> (-0.12%) ⬇️
.../dockstore/metricsaggregator/DoubleStatistics.java 100.00% <100.00%> (ø)
...e/metricsaggregator/MetricsAggregatorS3Client.java 81.81% <100.00%> (+0.42%) ⬆️
...o/dockstore/metricsaggregator/MoneyStatistics.java 100.00% <100.00%> (ø)
...aggregator/client/cli/MetricsAggregatorClient.java 69.30% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@kathy-t kathy-t marked this pull request as ready for review June 28, 2023 15:55
@kathy-t
Copy link
Contributor Author

kathy-t commented Jul 7, 2023

Changes:

  • Based on feedback in Add cost metric dockstore#5577 (comment), changed the aggregation of the cost metrics to use the Java Money library to preserve accuracy. In doing so, I removed the Statistics record and made it an abstract class that is inherited by DoubleStatistics and MoneyStatistics. I thought this was a better approach than adding a bunch of Money specific functions to the Statistics record

<artifactId>moneta-core</artifactId>
</dependency>
<dependency>
<groupId>javax.money</groupId>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not this existed. :)

@kathy-t kathy-t requested a review from denis-yuen July 10, 2023 19:54
Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very close, the SonarCloud bug warnings seem worth looking into (basically check whether an optional exists before using it)

@sonarcloud
Copy link

sonarcloud bot commented Jul 12, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

@kathy-t kathy-t merged commit d82c92b into develop Jul 13, 2023
5 of 6 checks passed
@kathy-t kathy-t deleted the feature/seab-5625/cost-metric branch July 13, 2023 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants