Avoid performing blocking I/O operation on application thread #8120

mcculls · 2024-12-20T00:44:18Z

Motivation

Agent discovery potentially involves a blocking I/O operation. This should only be run from our own background threads and not from application threads. Note we shouldn't need to wait for the result on the application thread because the tracer code already handles asynchronous upgrade/downgrade scenarios.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APMLP-326

pr-commenter · 2024-12-20T01:11:54Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/move-discovery-off-app-thread
git_commit_date	1734688094	1734688708
git_commit_sha	`ab205f6`	`a6cbc55`
release_version	1.45.0-SNAPSHOT~ab205f6a56	1.45.0-SNAPSHOT~a6cbc55ef7

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734691334	1734691334
ci_job_id	746800313	746800313
ci_pipeline_id	51642656	51642656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 16 performance improvements and 3 performance regressions! Performance is the same for 38 metrics, 6 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start	better [-65.638ms; -55.837ms] or [-5.303%; -4.511%]	1.177s	1.238s
scenario:startup:insecure-bank:iast:GlobalTracer	better [-66.735ms; -61.330ms] or [-21.516%; -19.773%]	246.137ms	310.169ms
scenario:startup:insecure-bank:iast:Remote Config	worse [+31.013µs; +76.085µs] or [+5.237%; +12.848%]	645.738µs	592.189µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start	better [-83.026ms; -48.501ms] or [-6.673%; -3.898%]	1.178s	1.244s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer	better [-68.110ms; -61.052ms] or [-21.868%; -19.603%]	246.871ms	311.452ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config	worse [+41.185µs; +81.989µs] or [+7.000%; +13.936%]	649.924µs	588.337µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Agent.start	better [-62.982ms; -55.774ms] or [-5.099%; -4.516%]	1.176s	1.235s
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer	better [-62.854ms; -61.172ms] or [-20.339%; -19.795%]	247.023ms	309.036ms
scenario:startup:insecure-bank:tracing:Agent.start	better [-63.826ms; -53.729ms] or [-5.742%; -4.834%]	1.053s	1.112s
scenario:startup:insecure-bank:tracing:GlobalTracer	better [-65.872ms; -62.304ms] or [-20.598%; -19.483%]	255.704ms	319.792ms
scenario:startup:insecure-bank:tracing:AppSec	worse [+1.774ms; +6.166ms] or [+3.210%; +11.157%]	59.232ms	55.263ms
scenario:startup:petclinic:appsec:Agent.start	better [-66.390ms; -42.371ms] or [-5.328%; -3.400%]	1.192s	1.246s
scenario:startup:petclinic:appsec:GlobalTracer	better [-64.237ms; -58.673ms] or [-20.339%; -18.577%]	254.379ms	315.834ms
scenario:startup:petclinic:iast:Agent.start	better [-70.513ms; -60.562ms] or [-5.682%; -4.880%]	1.175s	1.241s
scenario:startup:petclinic:iast:GlobalTracer	better [-63.710ms; -61.309ms] or [-20.646%; -19.868%]	246.070ms	308.579ms
scenario:startup:petclinic:profiling:Agent.start	better [-72.125ms; -35.862ms] or [-5.394%; -2.682%]	1.283s	1.337s
scenario:startup:petclinic:profiling:GlobalTracer	better [-68.632ms; -60.251ms] or [-15.642%; -13.732%]	374.334ms	438.776ms
scenario:startup:petclinic:tracing:Agent.start	better [-73.087ms; -43.671ms] or [-6.561%; -3.920%]	1.056s	1.114s
scenario:startup:petclinic:tracing:GlobalTracer	better [-67.827ms; -59.290ms] or [-21.177%; -18.511%]	256.731ms	320.289ms

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.112 s) : 0, 1111536
Total [baseline] (8.706 s) : 0, 8706123
Agent [candidate] (1.053 s) : 0, 1052759
Total [candidate] (8.601 s) : 0, 8601160
section iast
Agent [baseline] (1.238 s) : 0, 1237864
Total [baseline] (9.269 s) : 0, 9268595
Agent [candidate] (1.177 s) : 0, 1177126
Total [candidate] (9.209 s) : 0, 9208712
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.244 s) : 0, 1244243
Total [baseline] (9.284 s) : 0, 9283709
Agent [candidate] (1.178 s) : 0, 1178480
Total [candidate] (9.171 s) : 0, 9170857
section iast_TELEMETRY_OFF
Agent [baseline] (1.235 s) : 0, 1235117
Total [baseline] (9.23 s) : 0, 9229869
Agent [candidate] (1.176 s) : 0, 1175739
Total [candidate] (9.262 s) : 0, 9261907

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.112 s	-
Agent	iast	1.238 s	126.327 ms (11.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.244 s	132.707 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.235 s	123.581 ms (11.1%)
Total	tracing	8.706 s	-
Total	iast	9.269 s	562.472 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.284 s	577.586 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.23 s	523.746 ms (6.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	iast	1.177 s	124.368 ms (11.8%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.178 s	125.722 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.176 s	122.98 ms (11.7%)
Total	tracing	8.601 s	-
Total	iast	9.209 s	607.553 ms (7.1%)
Total	iast_HARDCODED_SECRET_DISABLED	9.171 s	569.697 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.262 s	660.747 ms (7.7%)

gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (711.895 ms) : 0, 711895
BytebuddyAgent [candidate] (711.745 ms) : 0, 711745
GlobalTracer [baseline] (319.792 ms) : 0, 319792
GlobalTracer [candidate] (255.704 ms) : 0, 255704
AppSec [baseline] (55.263 ms) : 0, 55263
AppSec [candidate] (59.232 ms) : 0, 59232
Remote Config [baseline] (677.691 µs) : 0, 678
Remote Config [candidate] (684.887 µs) : 0, 685
Telemetry [baseline] (8.937 ms) : 0, 8937
Telemetry [candidate] (10.41 ms) : 0, 10410
section iast
BytebuddyAgent [baseline] (826.798 ms) : 0, 826798
BytebuddyAgent [candidate] (828.337 ms) : 0, 828337
GlobalTracer [baseline] (310.169 ms) : 0, 310169
GlobalTracer [candidate] (246.137 ms) : 0, 246137
AppSec [baseline] (56.464 ms) : 0, 56464
AppSec [candidate] (57.558 ms) : 0, 57558
IAST [baseline] (20.794 ms) : 0, 20794
IAST [candidate] (20.996 ms) : 0, 20996
Remote Config [baseline] (592.189 µs) : 0, 592
Remote Config [candidate] (645.738 µs) : 0, 646
Telemetry [baseline] (8.216 ms) : 0, 8216
Telemetry [candidate] (8.487 ms) : 0, 8487
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (831.118 ms) : 0, 831118
BytebuddyAgent [candidate] (828.119 ms) : 0, 828119
GlobalTracer [baseline] (311.452 ms) : 0, 311452
GlobalTracer [candidate] (246.871 ms) : 0, 246871
AppSec [baseline] (56.769 ms) : 0, 56769
AppSec [candidate] (58.28 ms) : 0, 58280
IAST [baseline] (21.079 ms) : 0, 21079
IAST [candidate] (21.131 ms) : 0, 21131
Remote Config [baseline] (588.337 µs) : 0, 588
Remote Config [candidate] (649.924 µs) : 0, 650
Telemetry [baseline] (8.278 ms) : 0, 8278
Telemetry [candidate] (8.481 ms) : 0, 8481
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (823.474 ms) : 0, 823474
BytebuddyAgent [candidate] (826.178 ms) : 0, 826178
GlobalTracer [baseline] (309.036 ms) : 0, 309036
GlobalTracer [candidate] (247.023 ms) : 0, 247023
AppSec [baseline] (58.305 ms) : 0, 58305
AppSec [candidate] (57.951 ms) : 0, 57951
IAST [baseline] (20.634 ms) : 0, 20634
IAST [candidate] (20.65 ms) : 0, 20650
Remote Config [baseline] (597.379 µs) : 0, 597
Remote Config [candidate] (654.09 µs) : 0, 654
Telemetry [baseline] (8.152 ms) : 0, 8152
Telemetry [candidate] (8.342 ms) : 0, 8342

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.114 s) : 0, 1113973
Total [baseline] (10.486 s) : 0, 10486404
Agent [candidate] (1.056 s) : 0, 1055593
Total [candidate] (10.453 s) : 0, 10452752
section appsec
Agent [baseline] (1.246 s) : 0, 1246095
Total [baseline] (10.755 s) : 0, 10754797
Agent [candidate] (1.192 s) : 0, 1191715
Total [candidate] (10.757 s) : 0, 10756970
section iast
Agent [baseline] (1.241 s) : 0, 1240986
Total [baseline] (11.108 s) : 0, 11108340
Agent [candidate] (1.175 s) : 0, 1175449
Total [candidate] (10.965 s) : 0, 10964871
section profiling
Agent [baseline] (1.337 s) : 0, 1337249
Total [baseline] (10.901 s) : 0, 10900596
Agent [candidate] (1.283 s) : 0, 1283256
Total [candidate] (10.907 s) : 0, 10906592

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.114 s	-
Agent	appsec	1.246 s	132.122 ms (11.9%)
Agent	iast	1.241 s	127.014 ms (11.4%)
Agent	profiling	1.337 s	223.277 ms (20.0%)
Total	tracing	10.486 s	-
Total	appsec	10.755 s	268.393 ms (2.6%)
Total	iast	11.108 s	621.936 ms (5.9%)
Total	profiling	10.901 s	414.192 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.192 s	136.122 ms (12.9%)
Agent	iast	1.175 s	119.855 ms (11.4%)
Agent	profiling	1.283 s	227.663 ms (21.6%)
Total	tracing	10.453 s	-
Total	appsec	10.757 s	304.218 ms (2.9%)
Total	iast	10.965 s	512.119 ms (4.9%)
Total	profiling	10.907 s	453.84 ms (4.3%)

gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.019 ms) : 0, 714019
BytebuddyAgent [candidate] (711.931 ms) : 0, 711931
GlobalTracer [baseline] (320.289 ms) : 0, 320289
GlobalTracer [candidate] (256.731 ms) : 0, 256731
AppSec [baseline] (55.076 ms) : 0, 55076
AppSec [candidate] (57.707 ms) : 0, 57707
Remote Config [baseline] (674.757 µs) : 0, 675
Remote Config [candidate] (709.683 µs) : 0, 710
Telemetry [baseline] (8.937 ms) : 0, 8937
Telemetry [candidate] (13.447 ms) : 0, 13447
section appsec
BytebuddyAgent [baseline] (727.63 ms) : 0, 727630
BytebuddyAgent [candidate] (733.507 ms) : 0, 733507
GlobalTracer [baseline] (315.834 ms) : 0, 315834
GlobalTracer [candidate] (254.379 ms) : 0, 254379
AppSec [baseline] (167.155 ms) : 0, 167155
AppSec [candidate] (170.512 ms) : 0, 170512
IAST [baseline] (21.324 ms) : 0, 21324
IAST [candidate] (19.145 ms) : 0, 19145
Remote Config [baseline] (646.267 µs) : 0, 646
Remote Config [candidate] (658.988 µs) : 0, 659
Telemetry [baseline] (9.297 ms) : 0, 9297
Telemetry [candidate] (7.931 ms) : 0, 7931
section iast
BytebuddyAgent [baseline] (828.644 ms) : 0, 828644
BytebuddyAgent [candidate] (826.691 ms) : 0, 826691
GlobalTracer [baseline] (308.579 ms) : 0, 308579
GlobalTracer [candidate] (246.07 ms) : 0, 246070
AppSec [baseline] (58.533 ms) : 0, 58533
AppSec [candidate] (57.707 ms) : 0, 57707
IAST [baseline] (21.234 ms) : 0, 21234
IAST [candidate] (21.014 ms) : 0, 21014
Remote Config [baseline] (610.663 µs) : 0, 611
Remote Config [candidate] (636.646 µs) : 0, 637
Telemetry [baseline] (8.35 ms) : 0, 8350
Telemetry [candidate] (8.382 ms) : 0, 8382
section profiling
BytebuddyAgent [baseline] (699.83 ms) : 0, 699830
BytebuddyAgent [candidate] (708.002 ms) : 0, 708002
GlobalTracer [baseline] (438.776 ms) : 0, 438776
GlobalTracer [candidate] (374.334 ms) : 0, 374334
AppSec [baseline] (54.02 ms) : 0, 54020
AppSec [candidate] (54.189 ms) : 0, 54189
Remote Config [baseline] (653.021 µs) : 0, 653
Remote Config [candidate] (710.678 µs) : 0, 711
Telemetry [baseline] (7.823 ms) : 0, 7823
Telemetry [candidate] (7.839 ms) : 0, 7839
ProfilingAgent [baseline] (94.479 ms) : 0, 94479
ProfilingAgent [candidate] (96.297 ms) : 0, 96297
Profiling [baseline] (94.503 ms) : 0, 94503
Profiling [candidate] (96.321 ms) : 0, 96321

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-12-20T10:12:21	2024-12-20T10:19:19
git_branch	master	mcculls/move-discovery-off-app-thread
git_commit_date	1734688094	1734688708
git_commit_sha	`ab205f6`	`a6cbc55`
release_version	1.45.0-SNAPSHOT~ab205f6a56	1.45.0-SNAPSHOT~a6cbc55ef7
start_time	2024-12-20T10:12:07	2024-12-20T10:19:05

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1734690309	1734690309
ci_job_id	746800314	746800314
ci_pipeline_id	51642656	51642656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.188 µs) : 350, 390
.   : milestone, 370,
iast (484.026 µs) : 463, 505
.   : milestone, 484,
iast_FULL (649.124 µs) : 627, 671
.   : milestone, 649,
iast_GLOBAL (528.605 µs) : 505, 553
.   : milestone, 529,
iast_HARDCODED_SECRET_DISABLED (493.298 µs) : 471, 515
.   : milestone, 493,
iast_INACTIVE (453.399 µs) : 432, 475
.   : milestone, 453,
iast_TELEMETRY_OFF (483.763 µs) : 462, 505
.   : milestone, 484,
tracing (449.465 µs) : 429, 470
.   : milestone, 449,
section candidate
no_agent (380.489 µs) : 359, 402
.   : milestone, 380,
iast (494.906 µs) : 473, 517
.   : milestone, 495,
iast_FULL (644.622 µs) : 623, 666
.   : milestone, 645,
iast_GLOBAL (520.604 µs) : 499, 543
.   : milestone, 521,
iast_HARDCODED_SECRET_DISABLED (488.643 µs) : 467, 510
.   : milestone, 489,
iast_INACTIVE (449.247 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (475.951 µs) : 455, 497
.   : milestone, 476,
tracing (444.084 µs) : 423, 465
.   : milestone, 444,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	370.188 µs [350.346 µs, 390.029 µs]	-
iast	484.026 µs [462.755 µs, 505.297 µs]	113.839 µs (30.8%)
iast_FULL	649.124 µs [627.412 µs, 670.837 µs]	278.937 µs (75.4%)
iast_GLOBAL	528.605 µs [504.554 µs, 552.655 µs]	158.417 µs (42.8%)
iast_HARDCODED_SECRET_DISABLED	493.298 µs [471.484 µs, 515.111 µs]	123.11 µs (33.3%)
iast_INACTIVE	453.399 µs [432.285 µs, 474.514 µs]	83.212 µs (22.5%)
iast_TELEMETRY_OFF	483.763 µs [462.326 µs, 505.201 µs]	113.576 µs (30.7%)
tracing	449.465 µs [428.512 µs, 470.419 µs]	79.278 µs (21.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	380.489 µs [358.732 µs, 402.247 µs]	-
iast	494.906 µs [473.075 µs, 516.737 µs]	114.417 µs (30.1%)
iast_FULL	644.622 µs [623.016 µs, 666.228 µs]	264.133 µs (69.4%)
iast_GLOBAL	520.604 µs [498.543 µs, 542.665 µs]	140.115 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED	488.643 µs [467.25 µs, 510.036 µs]	108.154 µs (28.4%)
iast_INACTIVE	449.247 µs [428.441 µs, 470.053 µs]	68.758 µs (18.1%)
iast_TELEMETRY_OFF	475.951 µs [454.699 µs, 497.203 µs]	95.462 µs (25.1%)
tracing	444.084 µs [422.916 µs, 465.252 µs]	63.595 µs (16.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.745 ms) : 1721, 1769
.   : milestone, 1745,
appsec_no_iast (1.727 ms) : 1701, 1752
.   : milestone, 1727,
iast (1.487 ms) : 1465, 1510
.   : milestone, 1487,
profiling (1.505 ms) : 1481, 1529
.   : milestone, 1505,
tracing (1.484 ms) : 1459, 1508
.   : milestone, 1484,
section candidate
no_agent (1.35 ms) : 1331, 1370
.   : milestone, 1350,
appsec (1.736 ms) : 1712, 1759
.   : milestone, 1736,
appsec_no_iast (1.758 ms) : 1735, 1782
.   : milestone, 1758,
iast (1.486 ms) : 1463, 1510
.   : milestone, 1486,
profiling (1.505 ms) : 1482, 1528
.   : milestone, 1505,
tracing (1.491 ms) : 1467, 1515
.   : milestone, 1491,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.36 ms]	-
appsec	1.745 ms [1.721 ms, 1.769 ms]	404.292 µs (30.1%)
appsec_no_iast	1.727 ms [1.701 ms, 1.752 ms]	385.447 µs (28.7%)
iast	1.487 ms [1.465 ms, 1.51 ms]	146.252 µs (10.9%)
profiling	1.505 ms [1.481 ms, 1.529 ms]	164.037 µs (12.2%)
tracing	1.484 ms [1.459 ms, 1.508 ms]	142.571 µs (10.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.35 ms [1.331 ms, 1.37 ms]	-
appsec	1.736 ms [1.712 ms, 1.759 ms]	385.269 µs (28.5%)
appsec_no_iast	1.758 ms [1.735 ms, 1.782 ms]	408.133 µs (30.2%)
iast	1.486 ms [1.463 ms, 1.51 ms]	136.001 µs (10.1%)
profiling	1.505 ms [1.482 ms, 1.528 ms]	154.734 µs (11.5%)
tracing	1.491 ms [1.467 ms, 1.515 ms]	140.636 µs (10.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mcculls/move-discovery-off-app-thread
git_commit_date	1734688094	1734688708
git_commit_sha	`ab205f6`	`a6cbc55`
release_version	1.45.0-SNAPSHOT~ab205f6a56	1.45.0-SNAPSHOT~a6cbc55ef7

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1734690796	1734690796
ci_job_id	746800315	746800315
ci_pipeline_id	51642656	51642656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.18 s) : 15180000, 15180000
.   : milestone, 15180000,
appsec (15.217 s) : 15217000, 15217000
.   : milestone, 15217000,
iast (18.79 s) : 18790000, 18790000
.   : milestone, 18790000,
iast_GLOBAL (17.984 s) : 17984000, 17984000
.   : milestone, 17984000,
profiling (15.598 s) : 15598000, 15598000
.   : milestone, 15598000,
tracing (15.082 s) : 15082000, 15082000
.   : milestone, 15082000,
section candidate
no_agent (15.04 s) : 15040000, 15040000
.   : milestone, 15040000,
appsec (14.975 s) : 14975000, 14975000
.   : milestone, 14975000,
iast (18.995 s) : 18995000, 18995000
.   : milestone, 18995000,
iast_GLOBAL (17.722 s) : 17722000, 17722000
.   : milestone, 17722000,
profiling (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,
tracing (15.025 s) : 15025000, 15025000
.   : milestone, 15025000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.18 s [15.18 s, 15.18 s]	-
appsec	15.217 s [15.217 s, 15.217 s]	37.0 ms (0.2%)
iast	18.79 s [18.79 s, 18.79 s]	3.61 s (23.8%)
iast_GLOBAL	17.984 s [17.984 s, 17.984 s]	2.804 s (18.5%)
profiling	15.598 s [15.598 s, 15.598 s]	418.0 ms (2.8%)
tracing	15.082 s [15.082 s, 15.082 s]	-98.0 ms (-0.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.04 s [15.04 s, 15.04 s]	-
appsec	14.975 s [14.975 s, 14.975 s]	-65.0 ms (-0.4%)
iast	18.995 s [18.995 s, 18.995 s]	3.955 s (26.3%)
iast_GLOBAL	17.722 s [17.722 s, 17.722 s]	2.682 s (17.8%)
profiling	14.994 s [14.994 s, 14.994 s]	-46.0 ms (-0.3%)
tracing	15.025 s [15.025 s, 15.025 s]	-15.0 ms (-0.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~a6cbc55ef7, baseline=1.45.0-SNAPSHOT~ab205f6a56
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.343 ms) : 2301, 2385
.   : milestone, 2343,
iast (2.086 ms) : 2033, 2139
.   : milestone, 2086,
iast_GLOBAL (2.126 ms) : 2073, 2179
.   : milestone, 2126,
profiling (1.94 ms) : 1898, 1982
.   : milestone, 1940,
tracing (1.922 ms) : 1882, 1963
.   : milestone, 1922,
section candidate
no_agent (1.469 ms) : 1458, 1480
.   : milestone, 1469,
appsec (2.325 ms) : 2283, 2366
.   : milestone, 2325,
iast (2.081 ms) : 2029, 2134
.   : milestone, 2081,
iast_GLOBAL (2.125 ms) : 2072, 2178
.   : milestone, 2125,
profiling (1.935 ms) : 1894, 1977
.   : milestone, 1935,
tracing (1.925 ms) : 1885, 1965
.   : milestone, 1925,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.468 ms [1.457 ms, 1.48 ms]	-
appsec	2.343 ms [2.301 ms, 2.385 ms]	875.246 µs (59.6%)
iast	2.086 ms [2.033 ms, 2.139 ms]	617.784 µs (42.1%)
iast_GLOBAL	2.126 ms [2.073 ms, 2.179 ms]	657.919 µs (44.8%)
profiling	1.94 ms [1.898 ms, 1.982 ms]	471.868 µs (32.1%)
tracing	1.922 ms [1.882 ms, 1.963 ms]	453.976 µs (30.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.48 ms]	-
appsec	2.325 ms [2.283 ms, 2.366 ms]	855.651 µs (58.2%)
iast	2.081 ms [2.029 ms, 2.134 ms]	612.254 µs (41.7%)
iast_GLOBAL	2.125 ms [2.072 ms, 2.178 ms]	656.049 µs (44.7%)
profiling	1.935 ms [1.894 ms, 1.977 ms]	466.483 µs (31.8%)
tracing	1.925 ms [1.885 ms, 1.965 ms]	455.906 µs (31.0%)

pr-commenter · 2024-12-20T01:41:13Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1734689406	1734689783
end_time	2024-12-20T10:11:23	2024-12-20T10:17:40
git_branch	master	mcculls/move-discovery-off-app-thread
git_commit_sha	`ab205f6`	`a6cbc55`
start_time	2024-12-20T10:10:07	2024-12-20T10:16:23

See matching parameters

	Baseline	Candidate
ci_job_id	746800319	746800319
ci_pipeline_id	51642656	51642656
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1734688708	1734688708

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 5 metrics, 10 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-31.724µs; +87.304µs] or [-11.895%; +32.734%]	unstable [-37.958µs; +94.056µs] or [-12.359%; +30.624%]	unstable [-53.091µs; +122.641µs] or [-16.450%; +37.999%]	unstable [-406.342µs; +1175.665µs] or [-65.358%; +189.100%]	unstable [-173.310op/s; +106.710op/s] or [-6.672%; +4.108%]
scenario:basic	unstable [-8.714µs; +20.090µs] or [-3.155%; +7.275%]	unstable [-11.892µs; +32.019µs] or [-3.879%; +10.444%]	unstable [-17.599µs; +46.460µs] or [-5.535%; +14.612%]	unstable [-313.930µs; +901.093µs] or [-57.934%; +166.292%]	unstable [-210.100op/s; +210.100op/s] or [-8.404%; +8.404%]
scenario:loop	same	same	same	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (307.135 µs) : 272, 342
.   : milestone, 307,
basic (306.575 µs) : 295, 318
.   : milestone, 307,
loop (10.358 ms) : 10325, 10392
.   : milestone, 10358,
section candidate
noprobe (335.184 µs) : 253, 417
.   : milestone, 335,
basic (316.639 µs) : 288, 345
.   : milestone, 317,
loop (10.353 ms) : 10319, 10387
.   : milestone, 10353,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	307.135 µs [272.232 µs, 342.038 µs]
basic	306.575 µs [295.445 µs, 317.705 µs]
loop	10.358 ms [10.325 ms, 10.392 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	335.184 µs [253.363 µs, 417.004 µs]
basic	316.639 µs [288.421 µs, 344.857 µs]
loop	10.353 ms [10.319 ms, 10.387 ms]

…ain at Inbox start to merge agent config

…#8120)" This reverts commit 8d5f5ac.

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.50.0` -> `2.50.1` | | [com.google.cloud:google-cloud-core-http](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.49.0` -> `2.49.1` | | [com.google.cloud:google-cloud-core](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.49.0` -> `2.49.1` | | [com.google.api:gax](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.59.0` -> `2.59.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.44.1` -> `1.45.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.44.1` -> `1.45.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.47` -> `2.29.48` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.45.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.45.0): 1.45.0 ##### Breaking changes > \[!WARNING]\ > Support for custom scope manager using OpenTelemetry tracer artifact (`dd-trace-ot`) is dropped. > Tracing with OpenTracing API and custom scope manager will continue to work on 1.44.x releases. ##### Components ##### Application Security Management (IAST) - ✨ Add propagation to URI#toURL method ([#8146](DataDog/dd-trace-java#8146) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Increase IAST propagation to StringBuilder setLength ([#8119](DataDog/dd-trace-java#8119) - [@Mariovido](https://github.com/Mariovido)) - ✨ Increase IAST propagation to StringBuffer append ([#8082](DataDog/dd-trace-java#8082) - [@Mariovido](https://github.com/Mariovido)) - ✨ Handle IAST security controls custom validation and sanitization methods ([#7997](DataDog/dd-trace-java#7997) - [@jandro996](https://github.com/jandro996)) ##### Application Security Management (WAF) - ✨ Update user lifecycle tracking to V3 ([#8108](DataDog/dd-trace-java#8108) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Exploit prevention for Shell Injection / Command Injection ([#7615](DataDog/dd-trace-java#7615) - [@jandro996](https://github.com/jandro996)) ##### Build & Tooling - 💡 Support instrumentation of repackaged libraries ([#8153](DataDog/dd-trace-java#8153) - [@mcculls](https://github.com/mcculls)) - ✨ Configure native image build setting for JDK-22 based GraalVM ([#8092](DataDog/dd-trace-java#8092) - [@MattAlp](https://github.com/MattAlp)) ##### Database Monitoring - ✨ Add full APM/DBM mode for Oracle ([#8090](DataDog/dd-trace-java#8090) - [@nenadnoveljic](https://github.com/nenadnoveljic)) ##### Dynamic Instrumentation - 🐛 make local var hoisting disabled by default ([#8158](DataDog/dd-trace-java#8158) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix var hoisting issue when no previous store ([#8122](DataDog/dd-trace-java#8122) - [@jpbempel](https://github.com/jpbempel)) - ✨ Only decorate spans without code origin information ([#8105](DataDog/dd-trace-java#8105) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Fix suspend Kotlin methods instrumentation ([#8080](DataDog/dd-trace-java#8080) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix class file version detection ([#8057](DataDog/dd-trace-java#8057) - [@jpbempel](https://github.com/jpbempel)) ##### GraalVM native-image - ✨ Configure native image build setting for JDK-22 based GraalVM ([#8092](DataDog/dd-trace-java#8092) - [@MattAlp](https://github.com/MattAlp)) ##### ML Observability (LLMObs) - ✨🧪 Add LLMObs configuration ([#8076](DataDog/dd-trace-java#8076) - [@gary-huang](https://github.com/gary-huang)) ##### Metrics - Bump integrations-core submodule to 7.60.0 ([#8098](DataDog/dd-trace-java#8098) - [@mcculls](https://github.com/mcculls)) - Upgrade to java-dogstatsd-client v4.4.3 ([#8096](DataDog/dd-trace-java#8096) - [@mcculls](https://github.com/mcculls)) ##### OpenTracing - ⚠️🧹 Remove custom scope manager support ([#8164](DataDog/dd-trace-java#8164) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Telemetry - ✨ Retry telemetry requests if CI Visibility is enabled ([#8147](DataDog/dd-trace-java#8147) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add configurable Dependency service resolution period ([#8079](DataDog/dd-trace-java#8079) - [@jandro996](https://github.com/jandro996)) ##### Testing - 🐛 Remove restriction to not run vertx4 latest tests on java 17 ([#8133](DataDog/dd-trace-java#8133) - [@vandonr](https://github.com/vandonr)) ##### Tracer core - ✨ Defer remote components to avoid OkHttp class-loading side-effects ([#8131](DataDog/dd-trace-java#8131) - [@mcculls](https://github.com/mcculls)) - ✨ Improve Context API null handling and Javadoc ([#8129](DataDog/dd-trace-java#8129) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - 🐛⚡ Avoid performing blocking I/O operation on application thread ([#8120](DataDog/dd-trace-java#8120) - [@mcculls](https://github.com/mcculls)) - 💡 Introduce a shared context component, independent of tracing ([#8117](DataDog/dd-trace-java#8117) - [@mcculls](https://github.com/mcculls)) - ✨ Improves ServiceNameCollector ([#8109](DataDog/dd-trace-java#8109) - [@amarziali](https://github.com/amarziali)) - Upgrade to ASM 9.7.1 (adds new constant for Java 24) ([#8097](DataDog/dd-trace-java#8097) - [@mcculls](https://github.com/mcculls)) - 🐛 Dynamically evaluate service name for message consumers ([#8088](DataDog/dd-trace-java#8088) - [@amarziali](https://github.com/amarziali)) ##### Serverless - 🐛 Add avoid double instrumenting lambda non-streaming handlers. ([#8073](DataDog/dd-trace-java#8073) - [@purple4reina](https://github.com/purple4reina)) ##### Instrumentations ##### AWS SDK instrumentation - 💡 Instrument EMR's relocated AWS SDK ([#8157](DataDog/dd-trace-java#8157) - [@mcculls](https://github.com/mcculls)) ##### Eclipse Vert.x instrumentation - 🐛 Remove restriction to not run vertx4 latest tests on java 17 ([#8133](DataDog/dd-trace-java#8133) - [@vandonr](https://github.com/vandonr)) ##### JDBC instrumentation - ✨ Add full APM/DBM mode for Oracle ([#8090](DataDog/dd-trace-java#8090) - [@nenadnoveljic](https://github.com/nenadnoveljic)) ##### Jetty instrumentation - 🐛 Ensure jetty 12 has servlet.path starting with / ([#8093](DataDog/dd-trace-java#8093) - [@github-actions](https://github.com/github-actions)\[bot]) ##### JMS instrumentation - 🧹 Re-use `javax` JMS module for `jakarta` namespace ([#8155](DataDog/dd-trace-java#8155) - [@mcculls](https://github.com/mcculls)) - 🧹 Group `javax.jms` instrumentations under a single module ([#8154](DataDog/dd-trace-java#8154) - [@mcculls](https://github.com/mcculls)) ##### Reactor instrumentation - 🐛 Reactor: early propagate span in context when subscribing ([#8166](DataDog/dd-trace-java#8166) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: ba2355aa4e2e39ab1fee27319cc4176238efd90b

mcculls added type: bug comp: core Tracer core labels Dec 20, 2024

mcculls requested a review from a team as a code owner December 20, 2024 00:44

mcculls requested a review from nayeem-kamal December 20, 2024 00:44

mcculls marked this pull request as draft December 20, 2024 01:04

mcculls force-pushed the mcculls/move-discovery-off-app-thread branch from a457acb to 8e8639a Compare December 20, 2024 03:35

mcculls marked this pull request as ready for review December 20, 2024 03:38

mcculls requested a review from a team as a code owner December 20, 2024 03:38

mcculls requested review from jpbempel and removed request for a team December 20, 2024 03:38

jpbempel approved these changes Dec 20, 2024

View reviewed changes

amarziali approved these changes Dec 20, 2024

View reviewed changes

mcculls force-pushed the mcculls/move-discovery-off-app-thread branch from 8e8639a to dfab04e Compare December 20, 2024 09:46

mcculls added 4 commits December 20, 2024 09:58

Avoid performing blocking I/O operation on application thread

3cb466b

Make test more tolerant of parallel discover requests

e6224b2

Cleanup test side-effects

5fc9c69

Call checkDynamicConfig() at DSM start to load tracer config, then ag…

a6cbc55

…ain at Inbox start to merge agent config

mcculls force-pushed the mcculls/move-discovery-off-app-thread branch from dfab04e to a6cbc55 Compare December 20, 2024 09:59

mcculls merged commit 8d5f5ac into master Dec 20, 2024
149 checks passed

mcculls deleted the mcculls/move-discovery-off-app-thread branch December 20, 2024 10:47

github-actions bot added this to the 1.45.0 milestone Dec 20, 2024

mcculls mentioned this pull request Dec 20, 2024

Unable to disable tracing via env var, and connection times out after 30 seconds rather than default #7599

Open

smola added a commit that referenced this pull request Dec 20, 2024

Revert "Avoid performing blocking I/O operation on application thread (…

324f9f2

…#8120)" This reverts commit 8d5f5ac.

smola mentioned this pull request Dec 20, 2024

Revert "Avoid performing blocking I/O operation on application thread (#8120)" #8124

Closed

bm1549 added the tag: performance Performance related changes label Dec 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid performing blocking I/O operation on application thread #8120

Avoid performing blocking I/O operation on application thread #8120

mcculls commented Dec 20, 2024 •

edited by jira bot

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading

Avoid performing blocking I/O operation on application thread #8120

Avoid performing blocking I/O operation on application thread #8120

Conversation

mcculls commented Dec 20, 2024 • edited by jira bot Loading

Motivation

Contributor Checklist

pr-commenter bot commented Dec 20, 2024 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

pr-commenter bot commented Dec 20, 2024 • edited Loading

Debugger benchmarks

Parameters

Summary

mcculls commented Dec 20, 2024 •

edited by jira bot

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading

pr-commenter bot commented Dec 20, 2024 •

edited

Loading