Add a config flag to make ITR skip every test #8226

nikita-tkachenko-datadog · 2025-01-16T10:29:47Z

What Does This Do

Adds DD_CIVISIBILITY_SKIP_ALL_TESTS diagnostic flag to make ITR skip everything.

Motivation

Debugging ITR issues.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

LANGTOOLS-3125

pr-commenter · 2025-01-16T11:12:23Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/itr-skip-all-tests
git_commit_date	1737020324	1737023282
git_commit_sha	`148aef9`	`e12a433`
release_version	1.46.0-SNAPSHOT~148aef92ab	1.46.0-SNAPSHOT~e12a4330b2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737025643	1737025643
ci_job_id	768667806	768667806
ci_pipeline_id	53163874	53163874
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058089
Total [baseline] (10.423 s) : 0, 10422775
Agent [candidate] (1.059 s) : 0, 1058520
Total [candidate] (10.413 s) : 0, 10413241
section appsec
Agent [baseline] (1.193 s) : 0, 1192660
Total [baseline] (10.711 s) : 0, 10711475
Agent [candidate] (1.191 s) : 0, 1191494
Total [candidate] (10.701 s) : 0, 10701471
section iast
Agent [baseline] (1.186 s) : 0, 1186028
Total [baseline] (10.98 s) : 0, 10979680
Agent [candidate] (1.184 s) : 0, 1184083
Total [candidate] (11.039 s) : 0, 11039178
section profiling
Agent [baseline] (1.253 s) : 0, 1253114
Total [baseline] (10.832 s) : 0, 10832409
Agent [candidate] (1.277 s) : 0, 1276577
Total [candidate] (10.895 s) : 0, 10894882

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	appsec	1.193 s	134.571 ms (12.7%)
Agent	iast	1.186 s	127.939 ms (12.1%)
Agent	profiling	1.253 s	195.024 ms (18.4%)
Total	tracing	10.423 s	-
Total	appsec	10.711 s	288.7 ms (2.8%)
Total	iast	10.98 s	556.905 ms (5.3%)
Total	profiling	10.832 s	409.633 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.191 s	132.974 ms (12.6%)
Agent	iast	1.184 s	125.564 ms (11.9%)
Agent	profiling	1.277 s	218.057 ms (20.6%)
Total	tracing	10.413 s	-
Total	appsec	10.701 s	288.23 ms (2.8%)
Total	iast	11.039 s	625.938 ms (6.0%)
Total	profiling	10.895 s	481.641 ms (4.6%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.833 ms) : 0, 714833
BytebuddyAgent [candidate] (716.556 ms) : 0, 716556
GlobalTracer [baseline] (256.374 ms) : 0, 256374
GlobalTracer [candidate] (257.001 ms) : 0, 257001
AppSec [baseline] (56.204 ms) : 0, 56204
AppSec [candidate] (56.146 ms) : 0, 56146
Remote Config [baseline] (728.799 µs) : 0, 729
Remote Config [candidate] (731.311 µs) : 0, 731
Telemetry [baseline] (14.959 ms) : 0, 14959
Telemetry [candidate] (13.041 ms) : 0, 13041
section appsec
BytebuddyAgent [baseline] (734.125 ms) : 0, 734125
BytebuddyAgent [candidate] (733.124 ms) : 0, 733124
GlobalTracer [baseline] (253.955 ms) : 0, 253955
GlobalTracer [candidate] (253.712 ms) : 0, 253712
AppSec [baseline] (170.837 ms) : 0, 170837
AppSec [candidate] (170.786 ms) : 0, 170786
IAST [baseline] (19.541 ms) : 0, 19541
IAST [candidate] (19.472 ms) : 0, 19472
Remote Config [baseline] (665.201 µs) : 0, 665
Remote Config [candidate] (659.194 µs) : 0, 659
Telemetry [baseline] (8.289 ms) : 0, 8289
Telemetry [candidate] (8.508 ms) : 0, 8508
section iast
BytebuddyAgent [baseline] (834.262 ms) : 0, 834262
BytebuddyAgent [candidate] (832.871 ms) : 0, 832871
GlobalTracer [baseline] (247.714 ms) : 0, 247714
GlobalTracer [candidate] (247.385 ms) : 0, 247385
AppSec [baseline] (57.853 ms) : 0, 57853
AppSec [candidate] (57.893 ms) : 0, 57893
IAST [baseline] (21.617 ms) : 0, 21617
IAST [candidate] (21.461 ms) : 0, 21461
Remote Config [baseline] (674.494 µs) : 0, 674
Remote Config [candidate] (666.532 µs) : 0, 667
Telemetry [baseline] (8.804 ms) : 0, 8804
Telemetry [candidate] (8.788 ms) : 0, 8788
section profiling
BytebuddyAgent [baseline] (702.744 ms) : 0, 702744
BytebuddyAgent [candidate] (717.532 ms) : 0, 717532
GlobalTracer [baseline] (348.823 ms) : 0, 348823
GlobalTracer [candidate] (355.338 ms) : 0, 355338
AppSec [baseline] (54.646 ms) : 0, 54646
AppSec [candidate] (54.394 ms) : 0, 54394
Remote Config [baseline] (673.497 µs) : 0, 673
Remote Config [candidate] (677.907 µs) : 0, 678
Telemetry [baseline] (8.907 ms) : 0, 8907
Telemetry [candidate] (9.072 ms) : 0, 9072
ProfilingAgent [baseline] (95.431 ms) : 0, 95431
ProfilingAgent [candidate] (96.768 ms) : 0, 96768
Profiling [baseline] (95.458 ms) : 0, 95458
Profiling [candidate] (96.796 ms) : 0, 96796

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1055416
Total [baseline] (8.612 s) : 0, 8612147
Agent [candidate] (1.056 s) : 0, 1055724
Total [candidate] (8.613 s) : 0, 8612512
section iast
Agent [baseline] (1.192 s) : 0, 1192170
Total [baseline] (9.177 s) : 0, 9177095
Agent [candidate] (1.2 s) : 0, 1199977
Total [candidate] (9.225 s) : 0, 9224816
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.189 s) : 0, 1188713
Total [baseline] (9.154 s) : 0, 9153528
Agent [candidate] (1.186 s) : 0, 1185710
Total [candidate] (9.185 s) : 0, 9185473
section iast_TELEMETRY_OFF
Agent [baseline] (1.183 s) : 0, 1183092
Total [baseline] (9.157 s) : 0, 9156900
Agent [candidate] (1.198 s) : 0, 1197859
Total [candidate] (9.238 s) : 0, 9238263

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	iast	1.192 s	136.754 ms (13.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.189 s	133.297 ms (12.6%)
Agent	iast_TELEMETRY_OFF	1.183 s	127.676 ms (12.1%)
Total	tracing	8.612 s	-
Total	iast	9.177 s	564.947 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.154 s	541.381 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.157 s	544.752 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	iast	1.2 s	144.253 ms (13.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.186 s	129.986 ms (12.3%)
Agent	iast_TELEMETRY_OFF	1.198 s	142.136 ms (13.5%)
Total	tracing	8.613 s	-
Total	iast	9.225 s	612.304 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.185 s	572.961 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.238 s	625.751 ms (7.3%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (713.83 ms) : 0, 713830
BytebuddyAgent [candidate] (713.514 ms) : 0, 713514
GlobalTracer [baseline] (255.423 ms) : 0, 255423
GlobalTracer [candidate] (255.995 ms) : 0, 255995
AppSec [baseline] (55.389 ms) : 0, 55389
AppSec [candidate] (55.197 ms) : 0, 55197
Remote Config [baseline] (717.405 µs) : 0, 717
Remote Config [candidate] (720.328 µs) : 0, 720
Telemetry [baseline] (15.002 ms) : 0, 15002
Telemetry [candidate] (15.176 ms) : 0, 15176
section iast
BytebuddyAgent [baseline] (839.061 ms) : 0, 839061
BytebuddyAgent [candidate] (845.209 ms) : 0, 845209
GlobalTracer [baseline] (248.098 ms) : 0, 248098
GlobalTracer [candidate] (249.968 ms) : 0, 249968
AppSec [baseline] (58.248 ms) : 0, 58248
AppSec [candidate] (58.146 ms) : 0, 58146
IAST [baseline] (21.855 ms) : 0, 21855
IAST [candidate] (21.639 ms) : 0, 21639
Remote Config [baseline] (678.198 µs) : 0, 678
Remote Config [candidate] (682.603 µs) : 0, 683
Telemetry [baseline] (9.039 ms) : 0, 9039
Telemetry [candidate] (8.963 ms) : 0, 8963
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (836.931 ms) : 0, 836931
BytebuddyAgent [candidate] (833.921 ms) : 0, 833921
GlobalTracer [baseline] (247.681 ms) : 0, 247681
GlobalTracer [candidate] (247.348 ms) : 0, 247348
AppSec [baseline] (58.191 ms) : 0, 58191
AppSec [candidate] (58.243 ms) : 0, 58243
IAST [baseline] (21.467 ms) : 0, 21467
IAST [candidate] (21.697 ms) : 0, 21697
Remote Config [baseline] (670.025 µs) : 0, 670
Remote Config [candidate] (676.505 µs) : 0, 677
Telemetry [baseline] (8.736 ms) : 0, 8736
Telemetry [candidate] (8.785 ms) : 0, 8785
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (833.486 ms) : 0, 833486
BytebuddyAgent [candidate] (842.779 ms) : 0, 842779
GlobalTracer [baseline] (246.487 ms) : 0, 246487
GlobalTracer [candidate] (250.089 ms) : 0, 250089
AppSec [baseline] (57.714 ms) : 0, 57714
AppSec [candidate] (58.814 ms) : 0, 58814
IAST [baseline] (21.049 ms) : 0, 21049
IAST [candidate] (21.4 ms) : 0, 21400
Remote Config [baseline] (665.112 µs) : 0, 665
Remote Config [candidate] (683.752 µs) : 0, 684
Telemetry [baseline] (8.646 ms) : 0, 8646
Telemetry [candidate] (8.837 ms) : 0, 8837

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-16T10:37:25	2025-01-16T10:44:29
git_branch	master	nikita-tkachenko/itr-skip-all-tests
git_commit_date	1737020324	1737023282
git_commit_sha	`148aef9`	`e12a433`
release_version	1.46.0-SNAPSHOT~148aef92ab	1.46.0-SNAPSHOT~e12a4330b2
start_time	2025-01-16T10:37:11	2025-01-16T10:44:15

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737024629	1737024629
ci_job_id	768667808	768667808
ci_pipeline_id	53163874	53163874
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.582 µs) : 352, 393
.   : milestone, 373,
iast (517.793 µs) : 496, 540
.   : milestone, 518,
iast_FULL (748.275 µs) : 726, 770
.   : milestone, 748,
iast_GLOBAL (553.278 µs) : 532, 575
.   : milestone, 553,
iast_HARDCODED_SECRET_DISABLED (508.302 µs) : 487, 530
.   : milestone, 508,
iast_INACTIVE (456.648 µs) : 436, 478
.   : milestone, 457,
iast_TELEMETRY_OFF (499.546 µs) : 478, 521
.   : milestone, 500,
tracing (459.643 µs) : 438, 481
.   : milestone, 460,
section candidate
no_agent (376.271 µs) : 356, 396
.   : milestone, 376,
iast (514.578 µs) : 492, 537
.   : milestone, 515,
iast_FULL (739.984 µs) : 718, 762
.   : milestone, 740,
iast_GLOBAL (556.085 µs) : 534, 578
.   : milestone, 556,
iast_HARDCODED_SECRET_DISABLED (509.657 µs) : 488, 532
.   : milestone, 510,
iast_INACTIVE (463.397 µs) : 442, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (497.068 µs) : 475, 519
.   : milestone, 497,
tracing (450.672 µs) : 430, 471
.   : milestone, 451,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	372.582 µs [352.362 µs, 392.802 µs]	-
iast	517.793 µs [495.766 µs, 539.819 µs]	145.21 µs (39.0%)
iast_FULL	748.275 µs [726.18 µs, 770.369 µs]	375.692 µs (100.8%)
iast_GLOBAL	553.278 µs [531.656 µs, 574.9 µs]	180.696 µs (48.5%)
iast_HARDCODED_SECRET_DISABLED	508.302 µs [486.86 µs, 529.745 µs]	135.72 µs (36.4%)
iast_INACTIVE	456.648 µs [435.724 µs, 477.573 µs]	84.066 µs (22.6%)
iast_TELEMETRY_OFF	499.546 µs [477.94 µs, 521.151 µs]	126.963 µs (34.1%)
tracing	459.643 µs [438.359 µs, 480.928 µs]	87.061 µs (23.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	376.271 µs [356.333 µs, 396.209 µs]	-
iast	514.578 µs [492.294 µs, 536.862 µs]	138.307 µs (36.8%)
iast_FULL	739.984 µs [717.9 µs, 762.069 µs]	363.713 µs (96.7%)
iast_GLOBAL	556.085 µs [534.329 µs, 577.841 µs]	179.814 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED	509.657 µs [487.8 µs, 531.514 µs]	133.386 µs (35.4%)
iast_INACTIVE	463.397 µs [441.697 µs, 485.097 µs]	87.126 µs (23.2%)
iast_TELEMETRY_OFF	497.068 µs [475.001 µs, 519.135 µs]	120.797 µs (32.1%)
tracing	450.672 µs [429.874 µs, 471.471 µs]	74.401 µs (19.8%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1373
.   : milestone, 1353,
appsec (1.759 ms) : 1735, 1783
.   : milestone, 1759,
appsec_no_iast (1.78 ms) : 1757, 1804
.   : milestone, 1780,
iast (1.51 ms) : 1486, 1535
.   : milestone, 1510,
profiling (1.496 ms) : 1473, 1519
.   : milestone, 1496,
tracing (1.471 ms) : 1446, 1496
.   : milestone, 1471,
section candidate
no_agent (1.373 ms) : 1354, 1393
.   : milestone, 1373,
appsec (1.758 ms) : 1735, 1781
.   : milestone, 1758,
appsec_no_iast (1.772 ms) : 1748, 1797
.   : milestone, 1772,
iast (1.525 ms) : 1500, 1550
.   : milestone, 1525,
profiling (1.527 ms) : 1503, 1552
.   : milestone, 1527,
tracing (1.499 ms) : 1475, 1523
.   : milestone, 1499,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.353 ms [1.333 ms, 1.373 ms]	-
appsec	1.759 ms [1.735 ms, 1.783 ms]	405.93 µs (30.0%)
appsec_no_iast	1.78 ms [1.757 ms, 1.804 ms]	427.397 µs (31.6%)
iast	1.51 ms [1.486 ms, 1.535 ms]	157.407 µs (11.6%)
profiling	1.496 ms [1.473 ms, 1.519 ms]	142.879 µs (10.6%)
tracing	1.471 ms [1.446 ms, 1.496 ms]	117.874 µs (8.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.373 ms [1.354 ms, 1.393 ms]	-
appsec	1.758 ms [1.735 ms, 1.781 ms]	384.367 µs (28.0%)
appsec_no_iast	1.772 ms [1.748 ms, 1.797 ms]	398.878 µs (29.0%)
iast	1.525 ms [1.5 ms, 1.55 ms]	151.854 µs (11.1%)
profiling	1.527 ms [1.503 ms, 1.552 ms]	153.88 µs (11.2%)
tracing	1.499 ms [1.475 ms, 1.523 ms]	125.715 µs (9.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/itr-skip-all-tests
git_commit_date	1737020324	1737023282
git_commit_sha	`148aef9`	`e12a433`
release_version	1.46.0-SNAPSHOT~148aef92ab	1.46.0-SNAPSHOT~e12a4330b2

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1737025209	1737025209
ci_job_id	768667810	768667810
ci_pipeline_id	53163874	53163874
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.486 s) : 15486000, 15486000
.   : milestone, 15486000,
appsec (15.213 s) : 15213000, 15213000
.   : milestone, 15213000,
iast (18.903 s) : 18903000, 18903000
.   : milestone, 18903000,
iast_GLOBAL (18.257 s) : 18257000, 18257000
.   : milestone, 18257000,
profiling (15.418 s) : 15418000, 15418000
.   : milestone, 15418000,
tracing (14.925 s) : 14925000, 14925000
.   : milestone, 14925000,
section candidate
no_agent (14.899 s) : 14899000, 14899000
.   : milestone, 14899000,
appsec (15.19 s) : 15190000, 15190000
.   : milestone, 15190000,
iast (18.979 s) : 18979000, 18979000
.   : milestone, 18979000,
iast_GLOBAL (17.817 s) : 17817000, 17817000
.   : milestone, 17817000,
profiling (15.14 s) : 15140000, 15140000
.   : milestone, 15140000,
tracing (14.889 s) : 14889000, 14889000
.   : milestone, 14889000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.486 s [15.486 s, 15.486 s]	-
appsec	15.213 s [15.213 s, 15.213 s]	-273.0 ms (-1.8%)
iast	18.903 s [18.903 s, 18.903 s]	3.417 s (22.1%)
iast_GLOBAL	18.257 s [18.257 s, 18.257 s]	2.771 s (17.9%)
profiling	15.418 s [15.418 s, 15.418 s]	-68.0 ms (-0.4%)
tracing	14.925 s [14.925 s, 14.925 s]	-561.0 ms (-3.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.899 s [14.899 s, 14.899 s]	-
appsec	15.19 s [15.19 s, 15.19 s]	291.0 ms (2.0%)
iast	18.979 s [18.979 s, 18.979 s]	4.08 s (27.4%)
iast_GLOBAL	17.817 s [17.817 s, 17.817 s]	2.918 s (19.6%)
profiling	15.14 s [15.14 s, 15.14 s]	241.0 ms (1.6%)
tracing	14.889 s [14.889 s, 14.889 s]	-10.0 ms (-0.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~e12a4330b2, baseline=1.46.0-SNAPSHOT~148aef92ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (2.363 ms) : 2320, 2406
.   : milestone, 2363,
iast (2.108 ms) : 2054, 2163
.   : milestone, 2108,
iast_GLOBAL (2.152 ms) : 2097, 2207
.   : milestone, 2152,
profiling (1.978 ms) : 1934, 2022
.   : milestone, 1978,
tracing (1.955 ms) : 1913, 1997
.   : milestone, 1955,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.372 ms) : 2329, 2415
.   : milestone, 2372,
iast (2.104 ms) : 2050, 2159
.   : milestone, 2104,
iast_GLOBAL (2.158 ms) : 2103, 2213
.   : milestone, 2158,
profiling (1.982 ms) : 1937, 2026
.   : milestone, 1982,
tracing (1.944 ms) : 1902, 1986
.   : milestone, 1944,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.459 ms, 1.482 ms]	-
appsec	2.363 ms [2.32 ms, 2.406 ms]	892.423 µs (60.7%)
iast	2.108 ms [2.054 ms, 2.163 ms]	637.46 µs (43.3%)
iast_GLOBAL	2.152 ms [2.097 ms, 2.207 ms]	681.348 µs (46.3%)
profiling	1.978 ms [1.934 ms, 2.022 ms]	507.618 µs (34.5%)
tracing	1.955 ms [1.913 ms, 1.997 ms]	484.025 µs (32.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	2.372 ms [2.329 ms, 2.415 ms]	900.942 µs (61.2%)
iast	2.104 ms [2.05 ms, 2.159 ms]	633.166 µs (43.0%)
iast_GLOBAL	2.158 ms [2.103 ms, 2.213 ms]	686.851 µs (46.7%)
profiling	1.982 ms [1.937 ms, 2.026 ms]	510.462 µs (34.7%)
tracing	1.944 ms [1.902 ms, 1.986 ms]	472.835 µs (32.1%)

Add a config flag to make ITR skip every test

e12a433

nikita-tkachenko-datadog added tag: no release notes Changes to exclude from release notes tag: diagnostics Diagnostics related changes comp: ci visibility Continuous Integration Visibility labels Jan 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a config flag to make ITR skip every test #8226

Add a config flag to make ITR skip every test #8226

nikita-tkachenko-datadog commented Jan 16, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Jan 16, 2025

Add a config flag to make ITR skip every test #8226

Are you sure you want to change the base?

Add a config flag to make ITR skip every test #8226

Conversation

nikita-tkachenko-datadog commented Jan 16, 2025 • edited by jira bot Loading

What Does This Do

Motivation

Contributor Checklist

pr-commenter bot commented Jan 16, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

nikita-tkachenko-datadog commented Jan 16, 2025 •

edited by jira bot

Loading