Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Record number of skipped executions during aggregation #486

Merged
merged 11 commits into from
Mar 8, 2024
Merged

Record number of skipped executions during aggregation #486

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5e68df0
Count number of skipped executions
kathy-t Feb 23, 2024
26df5da
Clean up
kathy-t Feb 23, 2024
11a89d6
Refactor some things
kathy-t Feb 23, 2024
648f988
Clean up
kathy-t Feb 26, 2024
cadb648
Add clarification
kathy-t Feb 26, 2024
28e6686
Put this back
kathy-t Feb 26, 2024
3fd2cd5
Merge branch 'develop' into feature/seab-6046/aggregate-even-if-invalid
kathy-t Feb 28, 2024
cdccbb5
Validate execution metrics using webservice's Java Bean validation co…
kathy-t Feb 28, 2024
6f31449
Merge branch 'develop' into feature/seab-6046/aggregate-even-if-invalid
kathy-t Mar 6, 2024
f007056
PR feedback
kathy-t Mar 7, 2024
6696cc6
Use 1.16.0-alpha.2 webservice version
kathy-t Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions THIRD-PARTY-LICENSES.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Lists of 417 third-party dependencies.
(Apache License, Version 2.0) Apache Commons BeanUtils (commons-beanutils:commons-beanutils:1.9.4 - https://commons.apache.org/proper/commons-beanutils/)
(Apache License, Version 2.0) Apache Commons Codec (commons-codec:commons-codec:1.15 - https://commons.apache.org/proper/commons-codec/)
(Apache License, Version 2.0) Apache Commons Collections (commons-collections:commons-collections:3.2.2 - http://commons.apache.org/collections/)
(Apache License, Version 2.0) Apache Commons Compress (org.apache.commons:commons-compress:1.22 - https://commons.apache.org/proper/commons-compress/)
(Apache-2.0) Apache Commons Compress (org.apache.commons:commons-compress:1.24.0 - https://commons.apache.org/proper/commons-compress/)
(Apache License, Version 2.0) (The Apache Software License, Version 2.0) Apache Commons Configuration (org.apache.commons:commons-configuration2:2.8.0 - https://commons.apache.org/proper/commons-configuration/)
(Apache-2.0) Apache Commons CSV (org.apache.commons:commons-csv:1.10.0 - https://commons.apache.org/proper/commons-csv/)
(Apache License, Version 2.0) Apache Commons Exec (org.apache.commons:commons-exec:1.3 - http://commons.apache.org/proper/commons-exec/)
Expand Down Expand Up @@ -90,7 +90,7 @@ Lists of 417 third-party dependencies.
(MIT) Cats kernel (org.typelevel:cats-kernel_2.13:2.7.0 - https://github.com/typelevel/cats)
(MIT) Cats macros (org.typelevel:cats-macros_2.13:2.1.1 - https://github.com/typelevel/cats)
(Apache-2.0) cats-effect (org.typelevel:cats-effect_2.13:2.5.3 - https://typelevel.org/cats-effect/)
(The MIT License) Checker Qual (org.checkerframework:checker-qual:3.38.0 - https://checkerframework.org/)
(The MIT License) Checker Qual (org.checkerframework:checker-qual:3.42.0 - https://checkerframework.org/)
(Apache 2.0) Circe core (io.circe:circe-core_2.13:0.14.1 - https://github.com/circe/circe)
(Apache 2.0) Circe generic (io.circe:circe-generic_2.13:0.14.1 - https://github.com/circe/circe)
(Apache 2.0) Circe jawn (io.circe:circe-jawn_2.13:0.14.1 - https://github.com/circe/circe)
Expand Down Expand Up @@ -131,10 +131,10 @@ Lists of 417 third-party dependencies.
(The Apache Software License, Version 2.0) docker-java-core (com.github.docker-java:docker-java-core:3.3.0 - https://github.com/docker-java/docker-java)
(The Apache Software License, Version 2.0) docker-java-transport (com.github.docker-java:docker-java-transport:3.3.0 - https://github.com/docker-java/docker-java)
(The Apache Software License, Version 2.0) docker-java-transport-httpclient5 (com.github.docker-java:docker-java-transport-httpclient5:3.3.0 - https://github.com/docker-java/docker-java)
(Apache Software License, Version 2.0) dockstore-common (io.dockstore:dockstore-common:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) dockstore-integration-testing (io.dockstore:dockstore-integration-testing:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) dockstore-language-plugin-parent (io.dockstore:dockstore-language-plugin-parent:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) dockstore-webservice (io.dockstore:dockstore-webservice:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) dockstore-common (io.dockstore:dockstore-common:1.16.0-SNAPSHOT - no url defined)
(Apache Software License, Version 2.0) dockstore-integration-testing (io.dockstore:dockstore-integration-testing:1.16.0-SNAPSHOT - no url defined)
(Apache Software License, Version 2.0) dockstore-language-plugin-parent (io.dockstore:dockstore-language-plugin-parent:1.16.0-SNAPSHOT - no url defined)
(Apache Software License, Version 2.0) dockstore-webservice (io.dockstore:dockstore-webservice:1.16.0-SNAPSHOT - no url defined)
(Apache License 2.0) Dropwizard (io.dropwizard:dropwizard-core:4.0.2 - http://www.dropwizard.io/4.0.2/dropwizard-bom/dropwizard-dependencies/dropwizard-parent/dropwizard-core)
(Apache License 2.0) Dropwizard Asset Bundle (io.dropwizard:dropwizard-assets:4.0.2 - http://www.dropwizard.io/4.0.2/dropwizard-bom/dropwizard-dependencies/dropwizard-parent/dropwizard-assets)
(Apache License 2.0) Dropwizard Authentication (io.dropwizard:dropwizard-auth:4.0.2 - http://www.dropwizard.io/4.0.2/dropwizard-bom/dropwizard-dependencies/dropwizard-parent/dropwizard-auth)
Expand Down Expand Up @@ -230,7 +230,7 @@ Lists of 417 third-party dependencies.
(The Apache Software License, Version 2.0) Java 6 (and higher) extensions to the Google OAuth Client Library for Java. (com.google.oauth-client:google-oauth-client-java6:1.34.0 - https://github.com/googleapis/google-oauth-java-client/google-oauth-client-java6)
(Apache License, Version 2.0) Java Annotation Indexer (org.jboss:jandex:2.4.2.Final - http://www.jboss.org/jandex)
(The MIT License (MIT)) java jwt (com.auth0:java-jwt:3.8.1 - https://github.com/auth0/java-jwt)
(Apache-2.0) (LGPL-2.1-or-later) Java Native Access (net.java.dev.jna:jna:5.12.1 - https://github.com/java-native-access/jna)
(Apache-2.0) (LGPL-2.1-or-later) Java Native Access (net.java.dev.jna:jna:5.13.0 - https://github.com/java-native-access/jna)
(The MIT License) Java SemVer (com.github.zafarkhaja:java-semver:0.9.0 - https://github.com/zafarkhaja/jsemver)
(The Apache Software License, Version 2.0) java-diff-utils (io.github.java-diff-utils:java-diff-utils:4.12 - https://github.com/java-diff-utils/java-diff-utils/java-diff-utils)
(Common Development and Distribution License (CDDL) v1.0) JavaBeans Activation Framework (JAF) (javax.activation:activation:1.1 - http://java.sun.com/products/javabeans/jaf/index.jsp)
Expand Down Expand Up @@ -327,7 +327,7 @@ Lists of 417 third-party dependencies.
(Apache License 2.0) Metrics Utility Jakarta Servlets (io.dropwizard.metrics:metrics-jakarta-servlets:4.2.19 - https://metrics.dropwizard.io/metrics-jakarta-servlets)
(The Apache Software License, Version 2.0) metrics3-statsd (com.readytalk:metrics3-statsd:4.2.0 - no url defined)
(Apache 2) metrics4-scala (nl.grons:metrics4-scala_2.13:4.2.8 - https://github.com/erikvanoosten/metrics-scala)
(GNU General Public License (GPLv3)) metricsaggregator (io.dockstore:metricsaggregator:1.15.0 - https://github.com/dockstore/dockstore-support)
(GNU General Public License (GPLv3)) metricsaggregator (io.dockstore:metricsaggregator:1.16.0-SNAPSHOT - https://github.com/dockstore/dockstore-support)
(Eclipse Distribution License - v 1.0) MIME streaming extension (org.jvnet.mimepull:mimepull:1.9.15 - https://github.com/eclipse-ee4j/metro-mimepull)
(The MIT License) mockito-core (org.mockito:mockito-core:3.12.4 - https://github.com/mockito/mockito)
(The MIT License) mockito-inline (org.mockito:mockito-inline:3.12.4 - https://github.com/mockito/mockito)
Expand All @@ -354,7 +354,7 @@ Lists of 417 third-party dependencies.
(Apache License, Version 2.0) Objenesis (org.objenesis:objenesis:3.2 - http://objenesis.org/objenesis)
(The Apache Software License, Version 2.0) okhttp (com.squareup.okhttp3:okhttp:4.10.0 - https://square.github.io/okhttp/)
(The Apache Software License, Version 2.0) okio (com.squareup.okio:okio-jvm:3.0.0 - https://github.com/square/okio/)
(Apache Software License, Version 2.0) openapi-java-client (io.dockstore:openapi-java-client:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) openapi-java-client (io.dockstore:openapi-java-client:1.16.0-SNAPSHOT - no url defined)
(The Apache License, Version 2.0) OpenCensus (io.opencensus:opencensus-api:0.31.0 - https://github.com/census-instrumentation/opencensus-java)
(Apache 2) opencsv (com.opencsv:opencsv:5.7.1 - http://opencsv.sf.net)
(Apache 2.0) optics (io.circe:circe-optics_2.13:0.14.1 - https://github.com/circe/circe-optics)
Expand All @@ -363,7 +363,7 @@ Lists of 417 third-party dependencies.
(The Apache License, Version 2.0) org.opentest4j:opentest4j (org.opentest4j:opentest4j:1.3.0 - https://github.com/ota4j-team/opentest4j)
(EPL 2.0) (GPL2 w/ CPE) OSGi resource locator (org.glassfish.hk2:osgi-resource-locator:1.0.3 - https://projects.eclipse.org/projects/ee4j/osgi-resource-locator)
(The Apache Software License, Version 2.0) PF4J (org.pf4j:pf4j:3.10.0 - https://pf4j.org/pf4j)
(BSD-2-Clause) PostgreSQL JDBC Driver (org.postgresql:postgresql:42.4.3 - https://jdbc.postgresql.org)
(BSD-2-Clause) PostgreSQL JDBC Driver (org.postgresql:postgresql:42.7.2 - https://jdbc.postgresql.org)
(MIT) pprint_2.13 (com.lihaoyi:pprint_2.13:0.7.3 - https://github.com/lihaoyi/PPrint)
(The Apache Software License, Version 2.0) rank-eval (org.elasticsearch.plugin:rank-eval-client:7.10.2 - https://github.com/elastic/elasticsearch)
(CC0) reactive-streams (org.reactivestreams:reactive-streams:1.0.3 - http://www.reactive-streams.org/)
Expand Down Expand Up @@ -395,7 +395,7 @@ Lists of 417 third-party dependencies.
(Apache License 2.0) swagger-core-jakarta (io.swagger.core.v3:swagger-core-jakarta:2.2.15 - https://github.com/swagger-api/swagger-core/modules/swagger-core-jakarta)
(Apache License 2.0) swagger-integration-jakarta (io.swagger.core.v3:swagger-integration-jakarta:2.2.15 - https://github.com/swagger-api/swagger-core/modules/swagger-integration-jakarta)
(Apache Software License, Version 2.0) swagger-java-bitbucket-client (io.dockstore:swagger-java-bitbucket-client:2.0.3 - no url defined)
(Apache Software License, Version 2.0) swagger-java-client (io.dockstore:swagger-java-client:1.15.0-rc.2 - no url defined)
(Apache Software License, Version 2.0) swagger-java-client (io.dockstore:swagger-java-client:1.16.0-SNAPSHOT - no url defined)
(Apache Software License, Version 2.0) swagger-java-discourse-client (io.dockstore:swagger-java-discourse-client:2.0.1 - no url defined)
(Apache Software License, Version 2.0) swagger-java-quay-client (io.dockstore:swagger-java-quay-client:2.0.2 - no url defined)
(Apache Software License, Version 2.0) swagger-java-sam-client (io.dockstore:swagger-java-sam-client:2.0.2 - no url defined)
Expand All @@ -410,10 +410,10 @@ Lists of 417 third-party dependencies.
(Apache License 2.0) Throttling Appender (io.dropwizard.logback:logback-throttling-appender:1.4.0 - https://github.com/dropwizard/logback-throttling-appender/)
(Apache License, Version 2.0) tomcat-jdbc (org.apache.tomcat:tomcat-jdbc:10.1.13 - https://tomcat.apache.org/)
(Apache License, Version 2.0) tomcat-juli (org.apache.tomcat:tomcat-juli:10.1.13 - https://tomcat.apache.org/)
(GNU General Public License (GPLv3)) toolbackup (io.dockstore:toolbackup:1.15.0 - https://github.com/dockstore/dockstore-support)
(GNU General Public License (GPLv3)) tooltester (io.dockstore:tooltester:1.15.0 - https://github.com/dockstore/dockstore-support)
(GNU General Public License (GPLv3)) toolbackup (io.dockstore:toolbackup:1.16.0-SNAPSHOT - https://github.com/dockstore/dockstore-support)
(GNU General Public License (GPLv3)) tooltester (io.dockstore:tooltester:1.16.0-SNAPSHOT - https://github.com/dockstore/dockstore-support)
(Eclipse Distribution License - v 1.0) TXW2 Runtime (org.glassfish.jaxb:txw2:3.0.2 - https://eclipse-ee4j.github.io/jaxb-ri/)
(GNU General Public License (GPLv3)) utils (io.dockstore:utils:1.15.0 - https://github.com/dockstore/dockstore-support)
(GNU General Public License (GPLv3)) utils (io.dockstore:utils:1.16.0-SNAPSHOT - https://github.com/dockstore/dockstore-support)
(WDL License https://github.com/openwdl/wdl/blob/master/LICENSE) wdl-biscayne (org.broadinstitute:wdl-biscayne_2.13:85 - no url defined)
(WDL License https://github.com/openwdl/wdl/blob/master/LICENSE) wdl-draft2 (org.broadinstitute:wdl-draft2_2.13:85 - no url defined)
(WDL License https://github.com/openwdl/wdl/blob/master/LICENSE) wdl-draft3 (org.broadinstitute:wdl-draft3_2.13:85 - no url defined)
34 changes: 18 additions & 16 deletions metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CostAggregator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,41 @@
import io.dockstore.metricsaggregator.MoneyStatistics;
import io.dockstore.openapi.client.model.Cost;
import io.dockstore.openapi.client.model.CostMetric;
import io.dockstore.openapi.client.model.Metrics;
import io.dockstore.openapi.client.model.RunExecution;
import io.dockstore.openapi.client.model.TaskExecutions;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import org.javamoney.moneta.Money;

public class CostAggregator implements ExecutionAggregator<RunExecution, CostMetric, Cost> {
public class CostAggregator extends RunExecutionAggregator<CostMetric, Cost> {

@Override
public Cost getMetricFromExecution(RunExecution execution) {
return execution.getCost();
}

@Override
public CostMetric getMetricFromMetrics(Metrics metrics) {
return null; // There is no CostMetric in Metrics

Check warning on line 26 in metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CostAggregator.java

View check run for this annotation

Codecov / codecov/patch

metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CostAggregator.java#L26 

Added line #L26 was not covered by tests
}

@Override
public boolean validateExecutionMetric(Cost executionMetric) {
return executionMetric != null && isValidCurrencyCode(executionMetric.getCurrency()) && executionMetric.getValue() >= 0;
}

@Override
public Optional<RunExecution> getWorkflowExecutionFromTaskExecutions(TaskExecutions taskExecutionsForOneWorkflowRun) {
final List<RunExecution> taskExecutions = taskExecutionsForOneWorkflowRun.getTaskExecutions();
if (taskExecutions != null && taskExecutions.stream().map(RunExecution::getCost).allMatch(Objects::nonNull)) {
// Get the overall cost by summing up the cost of each task
List<Cost> taskCosts = taskExecutions.stream()
.map(RunExecution::getCost)
.filter(Objects::nonNull)
.toList();
boolean containsMalformedCurrencies = taskCosts.stream().anyMatch(cost -> !isValidCurrencyCode(cost.getCurrency()));
// This shouldn't happen until we allow users to submit any currency they want
if (!containsMalformedCurrencies && !taskCosts.isEmpty()) {
if (!taskCosts.isEmpty()) {
Money totalCost = taskCosts.stream()
.map(cost -> Money.of(cost.getValue(), cost.getCurrency()))
.reduce(Money.of(0, CURRENCY), Money::add);
Expand All @@ -41,17 +51,9 @@
}

@Override
public Optional<CostMetric> getAggregatedMetricFromExecutions(List<RunExecution> executions) {
List<Cost> submittedCosts = getNonNullMetricsFromExecutions(executions);

boolean containsMalformedCurrencies = submittedCosts.stream().anyMatch(cost -> !isValidCurrencyCode(cost.getCurrency()));
// This shouldn't happen until we allow users to submit any currency they want
if (containsMalformedCurrencies) {
return Optional.empty(); // Don't aggregate if there's malformed data
}

if (!submittedCosts.isEmpty()) {
List<Money> costs = submittedCosts.stream()
protected Optional<CostMetric> calculateAggregatedMetricFromExecutionMetrics(List<Cost> executionMetrics) {
if (!executionMetrics.isEmpty()) {
List<Money> costs = executionMetrics.stream()
.map(cost -> Money.of(cost.getValue(), cost.getCurrency()))
.toList();
MoneyStatistics statistics = new MoneyStatistics(costs);
Expand All @@ -65,7 +67,7 @@
}

@Override
public Optional<CostMetric> getAggregatedMetricsFromAggregatedMetrics(List<CostMetric> aggregatedMetrics) {
protected Optional<CostMetric> calculateAggregatedMetricFromAggregatedMetrics(List<CostMetric> aggregatedMetrics) {
if (!aggregatedMetrics.isEmpty()) {
List<MoneyStatistics> statistics = aggregatedMetrics.stream()
.map(metric -> new MoneyStatistics(
Expand Down
19 changes: 15 additions & 4 deletions metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CpuAggregator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import io.dockstore.metricsaggregator.DoubleStatistics;
import io.dockstore.openapi.client.model.CpuMetric;
import io.dockstore.openapi.client.model.Metrics;
import io.dockstore.openapi.client.model.RunExecution;
import io.dockstore.openapi.client.model.TaskExecutions;
import java.util.List;
Expand All @@ -12,13 +13,23 @@
* Aggregate CPU metrics by calculating the minimum, maximum, and average.
* @return
*/
public class CpuAggregator implements ExecutionAggregator<RunExecution, CpuMetric, Integer> {
public class CpuAggregator extends RunExecutionAggregator<CpuMetric, Integer> {

@Override
public Integer getMetricFromExecution(RunExecution execution) {
return execution.getCpuRequirements();
}

@Override
public CpuMetric getMetricFromMetrics(Metrics metrics) {
return null; // There is no CpuMetric in Metrics

Check warning on line 25 in metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CpuAggregator.java

View check run for this annotation

Codecov / codecov/patch

metricsaggregator/src/main/java/io/dockstore/metricsaggregator/helper/CpuAggregator.java#L25 

Added line #L25 was not covered by tests
}

@Override
public boolean validateExecutionMetric(Integer executionMetric) {
return executionMetric != null && executionMetric >= 0;
}

@Override
public Optional<RunExecution> getWorkflowExecutionFromTaskExecutions(TaskExecutions taskExecutionsForOneWorkflowRun) {
final List<RunExecution> taskExecutions = taskExecutionsForOneWorkflowRun.getTaskExecutions();
Expand All @@ -36,8 +47,8 @@
}

@Override
public Optional<CpuMetric> getAggregatedMetricFromExecutions(List<RunExecution> executions) {
List<Double> cpuRequirements = getNonNullMetricsFromExecutions(executions).stream()
protected Optional<CpuMetric> calculateAggregatedMetricFromExecutionMetrics(List<Integer> executionMetrics) {
List<Double> cpuRequirements = executionMetrics.stream()
.map(Integer::doubleValue)
.toList();
if (!cpuRequirements.isEmpty()) {
Expand All @@ -52,7 +63,7 @@
}

@Override
public Optional<CpuMetric> getAggregatedMetricsFromAggregatedMetrics(List<CpuMetric> aggregatedMetrics) {
protected Optional<CpuMetric> calculateAggregatedMetricFromAggregatedMetrics(List<CpuMetric> aggregatedMetrics) {
if (!aggregatedMetrics.isEmpty()) {
List<DoubleStatistics> statistics = aggregatedMetrics.stream()
.map(metric -> new DoubleStatistics(metric.getMinimum(), metric.getMaximum(), metric.getAverage(), metric.getNumberOfDataPointsForAverage())).toList();
Expand Down
Loading