Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

instrument Exception and Error classes to avoid instrumenting generic throwables used for control flow #6128

Merged
merged 1 commit into from
Nov 2, 2023
Merged

instrument Exception and Error classes to avoid instrumenting generic throwables used for control flow #6128

merged 1 commit into from
Nov 2, 2023

Conversation

richardstartin
Copy link
Member

@richardstartin richardstartin commented Nov 1, 2023 •
edited by jira bot
Loading

What Does This Do

This instruments the Exception and Error classes in the java.lang and kotlin packages, instead of Throwable which avoids several throwable types used for control flow, such as Scala's ControlThrowable. This also avoids resource tracking mechanisms such as Netty's ResourceLeakDetector which we were filtering out.

Avoiding the instrumentation altogether in these non-exceptional cases is important because the instrumentation interferes with escape analysis and ensuing optimisations which can reduce the cost of these exception types to practically nothing.

Motivation

Additional Notes

Jira ticket: PROF-8556

@pr-commenter
Copy link

pr-commenter bot commented Nov 1, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~0f5ae07427 1.23.0-SNAPSHOT~1f99550fee
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.033 s) : 0, 1033212
Total [baseline] (8.806 s) : 0, 8806201
Agent [candidate] (1.035 s) : 0, 1034991
Total [candidate] (8.755 s) : 0, 8754893
section iast
Agent [baseline] (1.152 s) : 0, 1151575
Total [baseline] (9.291 s) : 0, 9290639
Agent [candidate] (1.153 s) : 0, 1153446
Total [candidate] (9.322 s) : 0, 9322032
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145280
Total [baseline] (9.282 s) : 0, 9281825
Agent [candidate] (1.146 s) : 0, 1146077
Total [candidate] (9.308 s) : 0, 9308088
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.033 s -
Agent iast 1.152 s 118.363 ms (11.5%)
Agent iast_TELEMETRY_OFF 1.145 s 112.068 ms (10.8%)
Total tracing 8.806 s -
Total iast 9.291 s 484.439 ms (5.5%)
Total iast_TELEMETRY_OFF 9.282 s 475.625 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.035 s -
Agent iast 1.153 s 118.456 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.146 s 111.086 ms (10.7%)
Total tracing 8.755 s -
Total iast 9.322 s 567.139 ms (6.5%)
Total iast_TELEMETRY_OFF 9.308 s 553.195 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (644.665 ms) : 0, 644665
BytebuddyAgent [candidate] (645.816 ms) : 0, 645816
GlobalTracer [baseline] (293.045 ms) : 0, 293045
GlobalTracer [candidate] (293.715 ms) : 0, 293715
AppSec [baseline] (49.163 ms) : 0, 49163
AppSec [candidate] (49.016 ms) : 0, 49016
Remote Config [baseline] (676.03 µs) : 0, 676
Remote Config [candidate] (693.183 µs) : 0, 693
Telemetry [baseline] (11.314 ms) : 0, 11314
Telemetry [candidate] (11.308 ms) : 0, 11308
section iast
BytebuddyAgent [baseline] (763.592 ms) : 0, 763592
BytebuddyAgent [candidate] (766.561 ms) : 0, 766561
GlobalTracer [baseline] (274.741 ms) : 0, 274741
GlobalTracer [candidate] (274.853 ms) : 0, 274853
AppSec [baseline] (46.532 ms) : 0, 46532
AppSec [candidate] (46.772 ms) : 0, 46772
IAST [baseline] (18.493 ms) : 0, 18493
IAST [candidate] (17.669 ms) : 0, 17669
Remote Config [baseline] (556.371 µs) : 0, 556
Remote Config [candidate] (555.601 µs) : 0, 556
Telemetry [baseline] (13.196 ms) : 0, 13196
Telemetry [candidate] (12.475 ms) : 0, 12475
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (758.459 ms) : 0, 758459
BytebuddyAgent [candidate] (758.777 ms) : 0, 758777
GlobalTracer [baseline] (274.95 ms) : 0, 274950
GlobalTracer [candidate] (275.245 ms) : 0, 275245
AppSec [baseline] (46.731 ms) : 0, 46731
AppSec [candidate] (46.846 ms) : 0, 46846
IAST [baseline] (18.371 ms) : 0, 18371
IAST [candidate] (18.474 ms) : 0, 18474
Remote Config [baseline] (549.189 µs) : 0, 549
Remote Config [candidate] (566.349 µs) : 0, 566
Telemetry [baseline] (11.828 ms) : 0, 11828
Telemetry [candidate] (11.835 ms) : 0, 11835
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1034284
Total [baseline] (9.308 s) : 0, 9308320
Agent [candidate] (1.036 s) : 0, 1035857
Total [candidate] (9.332 s) : 0, 9332417
section appsec
Agent [baseline] (1.125 s) : 0, 1124905
Total [baseline] (9.45 s) : 0, 9449769
Agent [candidate] (1.122 s) : 0, 1121602
Total [candidate] (9.418 s) : 0, 9417844
section iast
Agent [baseline] (1.151 s) : 0, 1150558
Total [baseline] (9.57 s) : 0, 9569596
Agent [candidate] (1.166 s) : 0, 1166245
Total [candidate] (9.609 s) : 0, 9608634
section profiling
Agent [baseline] (1.211 s) : 0, 1211126
Total [baseline] (9.582 s) : 0, 9581519
Agent [candidate] (1.217 s) : 0, 1216616
Total [candidate] (9.546 s) : 0, 9545718
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.034 s -
Agent appsec 1.125 s 90.621 ms (8.8%)
Agent iast 1.151 s 116.274 ms (11.2%)
Agent profiling 1.211 s 176.842 ms (17.1%)
Total tracing 9.308 s -
Total appsec 9.45 s 141.449 ms (1.5%)
Total iast 9.57 s 261.276 ms (2.8%)
Total profiling 9.582 s 273.199 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.122 s 85.745 ms (8.3%)
Agent iast 1.166 s 130.388 ms (12.6%)
Agent profiling 1.217 s 180.759 ms (17.5%)
Total tracing 9.332 s -
Total appsec 9.418 s 85.427 ms (0.9%)
Total iast 9.609 s 276.217 ms (3.0%)
Total profiling 9.546 s 213.301 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.206 ms) : 0, 645206
BytebuddyAgent [candidate] (646.153 ms) : 0, 646153
GlobalTracer [baseline] (293.869 ms) : 0, 293869
GlobalTracer [candidate] (294.43 ms) : 0, 294430
AppSec [baseline] (48.909 ms) : 0, 48909
AppSec [candidate] (48.781 ms) : 0, 48781
Remote Config [baseline] (688.182 µs) : 0, 688
Remote Config [candidate] (694.031 µs) : 0, 694
Telemetry [baseline] (11.278 ms) : 0, 11278
Telemetry [candidate] (11.31 ms) : 0, 11310
section appsec
BytebuddyAgent [baseline] (648.486 ms) : 0, 648486
BytebuddyAgent [candidate] (646.788 ms) : 0, 646788
GlobalTracer [baseline] (294.932 ms) : 0, 294932
GlobalTracer [candidate] (294.6 ms) : 0, 294600
AppSec [baseline] (139.524 ms) : 0, 139524
AppSec [candidate] (138.266 ms) : 0, 138266
Remote Config [baseline] (652.934 µs) : 0, 653
Remote Config [candidate] (641.511 µs) : 0, 642
Telemetry [baseline] (6.92 ms) : 0, 6920
Telemetry [candidate] (6.888 ms) : 0, 6888
section iast
BytebuddyAgent [baseline] (763.96 ms) : 0, 763960
BytebuddyAgent [candidate] (777.833 ms) : 0, 777833
GlobalTracer [baseline] (274.931 ms) : 0, 274931
GlobalTracer [candidate] (278.117 ms) : 0, 278117
AppSec [baseline] (46.383 ms) : 0, 46383
AppSec [candidate] (47.554 ms) : 0, 47554
IAST [baseline] (17.788 ms) : 0, 17788
IAST [candidate] (15.829 ms) : 0, 15829
Remote Config [baseline] (549.622 µs) : 0, 550
Remote Config [candidate] (553.596 µs) : 0, 554
Telemetry [baseline] (12.505 ms) : 0, 12505
Telemetry [candidate] (11.406 ms) : 0, 11406
section profiling
BytebuddyAgent [baseline] (655.027 ms) : 0, 655027
BytebuddyAgent [candidate] (660.679 ms) : 0, 660679
GlobalTracer [baseline] (359.185 ms) : 0, 359185
GlobalTracer [candidate] (358.956 ms) : 0, 358956
AppSec [baseline] (49.655 ms) : 0, 49655
AppSec [candidate] (49.358 ms) : 0, 49358
Remote Config [baseline] (642.422 µs) : 0, 642
Remote Config [candidate] (638.798 µs) : 0, 639
Telemetry [baseline] (11.319 ms) : 0, 11319
Telemetry [candidate] (11.33 ms) : 0, 11330
ProfilingAgent [baseline] (80.971 ms) : 0, 80971
ProfilingAgent [candidate] (80.81 ms) : 0, 80810
Profiling [baseline] (80.995 ms) : 0, 80995
Profiling [candidate] (80.835 ms) : 0, 80835
Loading

Load

Parameters

Baseline Candidate
commit 1.23.0-SNAPSHOT~0f5ae07427 1.23.0-SNAPSHOT~1f99550fee
config baseline candidate
end_time 2023-11-01T16:06:40 2023-11-01T16:23:10
start_time 2023-11-01T16:06:28 2023-11-01T16:22:57
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.587 µs) : 343, 384
.   : milestone, 364,
iast (460.009 µs) : 439, 481
.   : milestone, 460,
iast_FULL (524.628 µs) : 504, 545
.   : milestone, 525,
iast_INACTIVE (438.482 µs) : 418, 459
.   : milestone, 438,
iast_TELEMETRY_OFF (466.699 µs) : 446, 488
.   : milestone, 467,
tracing (434.195 µs) : 412, 456
.   : milestone, 434,
section candidate
no_agent (362.012 µs) : 342, 382
.   : milestone, 362,
iast (464.798 µs) : 444, 486
.   : milestone, 465,
iast_FULL (530.559 µs) : 510, 551
.   : milestone, 531,
iast_INACTIVE (432.524 µs) : 411, 454
.   : milestone, 433,
iast_TELEMETRY_OFF (461.705 µs) : 441, 483
.   : milestone, 462,
tracing (429.09 µs) : 408, 450
.   : milestone, 429,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.587 µs [343.16 µs, 384.014 µs] -
iast 460.009 µs [439.114 µs, 480.904 µs] 96.422 µs (26.5%)
iast_FULL 524.628 µs [503.865 µs, 545.392 µs] 161.041 µs (44.3%)
iast_INACTIVE 438.482 µs [417.595 µs, 459.368 µs] 74.894 µs (20.6%)
iast_TELEMETRY_OFF 466.699 µs [445.504 µs, 487.894 µs] 103.112 µs (28.4%)
tracing 434.195 µs [412.078 µs, 456.312 µs] 70.608 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.012 µs [341.676 µs, 382.349 µs] -
iast 464.798 µs [443.942 µs, 485.653 µs] 102.786 µs (28.4%)
iast_FULL 530.559 µs [509.843 µs, 551.274 µs] 168.546 µs (46.6%)
iast_INACTIVE 432.524 µs [410.93 µs, 454.117 µs] 70.511 µs (19.5%)
iast_TELEMETRY_OFF 461.705 µs [440.617 µs, 482.793 µs] 99.692 µs (27.5%)
tracing 429.09 µs [407.903 µs, 450.277 µs] 67.078 µs (18.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~1f99550fee, baseline=1.23.0-SNAPSHOT~0f5ae07427
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.351 ms) : 1332, 1371
.   : milestone, 1351,
appsec (1.739 ms) : 1715, 1764
.   : milestone, 1739,
iast (1.483 ms) : 1459, 1507
.   : milestone, 1483,
profiling (1.487 ms) : 1462, 1512
.   : milestone, 1487,
tracing (1.474 ms) : 1449, 1499
.   : milestone, 1474,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.718 ms) : 1693, 1742
.   : milestone, 1718,
iast (1.506 ms) : 1482, 1530
.   : milestone, 1506,
profiling (1.481 ms) : 1454, 1507
.   : milestone, 1481,
tracing (1.479 ms) : 1455, 1503
.   : milestone, 1479,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.351 ms [1.332 ms, 1.371 ms] -
appsec 1.739 ms [1.715 ms, 1.764 ms] 388.221 µs (28.7%)
iast 1.483 ms [1.459 ms, 1.507 ms] 131.786 µs (9.8%)
profiling 1.487 ms [1.462 ms, 1.512 ms] 135.713 µs (10.0%)
tracing 1.474 ms [1.449 ms, 1.499 ms] 122.642 µs (9.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.33 ms, 1.368 ms] -
appsec 1.718 ms [1.693 ms, 1.742 ms] 368.371 µs (27.3%)
iast 1.506 ms [1.482 ms, 1.53 ms] 156.89 µs (11.6%)
profiling 1.481 ms [1.454 ms, 1.507 ms] 131.403 µs (9.7%)
tracing 1.479 ms [1.455 ms, 1.503 ms] 129.683 µs (9.6%)

@richardstartin richardstartin force-pushed the rgs/refactor-exception-profiling branch 2 times, most recently from 735ada3 to 06b6a79 Compare November 1, 2023 13:48
@richardstartin richardstartin changed the title instrument exception and error to avoid instrumenting generic throwables used for control flow instrument Exception and Error classes to avoid instrumenting generic throwables used for control flow Nov 1, 2023
@richardstartin richardstartin force-pushed the rgs/refactor-exception-profiling branch from 06b6a79 to 1f99550 Compare November 1, 2023 15:42
@richardstartin richardstartin marked this pull request as ready for review November 1, 2023 15:49
@richardstartin richardstartin requested review from a team as code owners November 1, 2023 15:49
@richardstartin richardstartin merged commit 047c47c into master Nov 2, 2023
10 of 11 checks passed
@richardstartin richardstartin deleted the rgs/refactor-exception-profiling branch November 2, 2023 09:16
@github-actions github-actions bot added this to the 1.23.0 milestone Nov 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbachorik jbachorik jbachorik approved these changes

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

@bantonsson bantonsson Awaiting requested review from bantonsson

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.23.0
Development

Successfully merging this pull request may close these issues.
2 participants
@richardstartin @jbachorik