Add NullAway to IAST module and fix errors #6106

manuel-alvarez-alvarez · 2023-10-26T15:40:33Z

What Does This Do

Add NullAway as build time tool to prevent NPE in IAST code
Force compilation with JDK11, targeting Java 8.

Motivation

Additional Notes

Jira ticket: APPSEC-11860

pr-commenter · 2023-10-26T16:25:33Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~047c47ca69	1.23.0-SNAPSHOT~07ca0e595a
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042973
Total [baseline] (8.781 s) : 0, 8781123
Agent [candidate] (1.036 s) : 0, 1035544
Total [candidate] (8.79 s) : 0, 8789523
section iast
Agent [baseline] (1.149 s) : 0, 1149404
Total [baseline] (9.358 s) : 0, 9357537
Agent [candidate] (1.151 s) : 0, 1150621
Total [candidate] (9.321 s) : 0, 9320750
section iast_TELEMETRY_OFF
Agent [baseline] (1.152 s) : 0, 1151633
Total [baseline] (9.279 s) : 0, 9279233
Agent [candidate] (1.154 s) : 0, 1153737
Total [candidate] (9.351 s) : 0, 9350716

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	iast	1.149 s	106.431 ms (10.2%)
Agent	iast_TELEMETRY_OFF	1.152 s	108.66 ms (10.4%)
Total	tracing	8.781 s	-
Total	iast	9.358 s	576.413 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.279 s	498.11 ms (5.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	iast	1.151 s	115.077 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.154 s	118.193 ms (11.4%)
Total	tracing	8.79 s	-
Total	iast	9.321 s	531.227 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.351 s	561.193 ms (6.4%)

gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (650.5 ms) : 0, 650500
BytebuddyAgent [candidate] (646.031 ms) : 0, 646031
GlobalTracer [baseline] (296.379 ms) : 0, 296379
GlobalTracer [candidate] (294.349 ms) : 0, 294349
AppSec [baseline] (49.289 ms) : 0, 49289
AppSec [candidate] (48.777 ms) : 0, 48777
Remote Config [baseline] (709.841 µs) : 0, 710
Remote Config [candidate] (700.168 µs) : 0, 700
Telemetry [baseline] (11.383 ms) : 0, 11383
Telemetry [candidate] (11.171 ms) : 0, 11171
section iast
BytebuddyAgent [baseline] (765.11 ms) : 0, 765110
BytebuddyAgent [candidate] (764.932 ms) : 0, 764932
GlobalTracer [baseline] (273.741 ms) : 0, 273741
GlobalTracer [candidate] (273.754 ms) : 0, 273754
AppSec [baseline] (46.788 ms) : 0, 46788
AppSec [candidate] (46.747 ms) : 0, 46747
IAST [baseline] (17.266 ms) : 0, 17266
IAST [candidate] (19.229 ms) : 0, 19229
Remote Config [baseline] (573.161 µs) : 0, 573
Remote Config [candidate] (567.181 µs) : 0, 567
Telemetry [baseline] (11.365 ms) : 0, 11365
Telemetry [candidate] (11.074 ms) : 0, 11074
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.991 ms) : 0, 763991
BytebuddyAgent [candidate] (765.551 ms) : 0, 765551
GlobalTracer [baseline] (274.727 ms) : 0, 274727
GlobalTracer [candidate] (276.677 ms) : 0, 276677
AppSec [baseline] (46.695 ms) : 0, 46695
AppSec [candidate] (47.102 ms) : 0, 47102
IAST [baseline] (18.265 ms) : 0, 18265
IAST [candidate] (17.926 ms) : 0, 17926
Remote Config [baseline] (549.912 µs) : 0, 550
Remote Config [candidate] (569.572 µs) : 0, 570
Telemetry [baseline] (12.608 ms) : 0, 12608
Telemetry [candidate] (11.266 ms) : 0, 11266

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051238
Total [baseline] (9.368 s) : 0, 9367934
Agent [candidate] (1.046 s) : 0, 1046098
Total [candidate] (9.323 s) : 0, 9323487
section appsec
Agent [baseline] (1.122 s) : 0, 1121872
Total [baseline] (9.389 s) : 0, 9389143
Agent [candidate] (1.123 s) : 0, 1122537
Total [candidate] (9.472 s) : 0, 9472111
section iast
Agent [baseline] (1.149 s) : 0, 1149151
Total [baseline] (9.475 s) : 0, 9475029
Agent [candidate] (1.153 s) : 0, 1152721
Total [candidate] (9.555 s) : 0, 9555047
section profiling
Agent [baseline] (1.222 s) : 0, 1222055
Total [baseline] (9.571 s) : 0, 9570906
Agent [candidate] (1.215 s) : 0, 1214698
Total [candidate] (9.527 s) : 0, 9526925

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	appsec	1.122 s	70.634 ms (6.7%)
Agent	iast	1.149 s	97.912 ms (9.3%)
Agent	profiling	1.222 s	170.816 ms (16.2%)
Total	tracing	9.368 s	-
Total	appsec	9.389 s	21.209 ms (0.2%)
Total	iast	9.475 s	107.095 ms (1.1%)
Total	profiling	9.571 s	202.972 ms (2.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.123 s	76.439 ms (7.3%)
Agent	iast	1.153 s	106.622 ms (10.2%)
Agent	profiling	1.215 s	168.599 ms (16.1%)
Total	tracing	9.323 s	-
Total	appsec	9.472 s	148.624 ms (1.6%)
Total	iast	9.555 s	231.56 ms (2.5%)
Total	profiling	9.527 s	203.437 ms (2.2%)

gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (655.294 ms) : 0, 655294
BytebuddyAgent [candidate] (652.644 ms) : 0, 652644
GlobalTracer [baseline] (298.603 ms) : 0, 298603
GlobalTracer [candidate] (297.207 ms) : 0, 297207
AppSec [baseline] (49.996 ms) : 0, 49996
AppSec [candidate] (49.317 ms) : 0, 49317
Remote Config [baseline] (713.839 µs) : 0, 714
Remote Config [candidate] (705.948 µs) : 0, 706
Telemetry [baseline] (11.582 ms) : 0, 11582
Telemetry [candidate] (11.413 ms) : 0, 11413
section appsec
BytebuddyAgent [baseline] (646.972 ms) : 0, 646972
BytebuddyAgent [candidate] (646.698 ms) : 0, 646698
GlobalTracer [baseline] (294.776 ms) : 0, 294776
GlobalTracer [candidate] (295.102 ms) : 0, 295102
AppSec [baseline] (138.103 ms) : 0, 138103
AppSec [candidate] (138.705 ms) : 0, 138705
Remote Config [baseline] (646.203 µs) : 0, 646
Remote Config [candidate] (649.257 µs) : 0, 649
Telemetry [baseline] (6.889 ms) : 0, 6889
Telemetry [candidate] (6.911 ms) : 0, 6911
section iast
BytebuddyAgent [baseline] (765.109 ms) : 0, 765109
BytebuddyAgent [candidate] (766.795 ms) : 0, 766795
GlobalTracer [baseline] (273.78 ms) : 0, 273780
GlobalTracer [candidate] (274.822 ms) : 0, 274822
AppSec [baseline] (46.761 ms) : 0, 46761
AppSec [candidate] (46.753 ms) : 0, 46753
Remote Config [baseline] (564.278 µs) : 0, 564
Remote Config [candidate] (566.616 µs) : 0, 567
Telemetry [baseline] (11.285 ms) : 0, 11285
Telemetry [candidate] (11.841 ms) : 0, 11841
IAST [baseline] (17.131 ms) : 0, 17131
IAST [candidate] (17.371 ms) : 0, 17371
section profiling
ProfilingAgent [baseline] (81.816 ms) : 0, 81816
ProfilingAgent [candidate] (81.487 ms) : 0, 81487
BytebuddyAgent [baseline] (661.19 ms) : 0, 661190
BytebuddyAgent [candidate] (657.968 ms) : 0, 657968
GlobalTracer [baseline] (362.146 ms) : 0, 362146
GlobalTracer [candidate] (359.613 ms) : 0, 359613
AppSec [baseline] (50.147 ms) : 0, 50147
AppSec [candidate] (49.232 ms) : 0, 49232
Remote Config [baseline] (646.27 µs) : 0, 646
Remote Config [candidate] (663.015 µs) : 0, 663
Telemetry [baseline] (11.412 ms) : 0, 11412
Telemetry [candidate] (11.425 ms) : 0, 11425
Profiling [baseline] (81.84 ms) : 0, 81840
Profiling [candidate] (81.512 ms) : 0, 81512

Load

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~047c47ca69	1.23.0-SNAPSHOT~07ca0e595a
config	baseline	candidate
end_time	2023-11-02T13:17:16	2023-11-02T13:33:44
start_time	2023-11-02T13:17:03	2023-11-02T13:33:31

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.823 µs) : 342, 382
.   : milestone, 362,
iast (455.312 µs) : 434, 476
.   : milestone, 455,
iast_FULL (524.125 µs) : 504, 545
.   : milestone, 524,
iast_INACTIVE (430.62 µs) : 410, 452
.   : milestone, 431,
iast_TELEMETRY_OFF (459.251 µs) : 438, 481
.   : milestone, 459,
tracing (433.577 µs) : 412, 455
.   : milestone, 434,
section candidate
no_agent (358.763 µs) : 339, 379
.   : milestone, 359,
iast (459.897 µs) : 439, 481
.   : milestone, 460,
iast_FULL (521.807 µs) : 501, 542
.   : milestone, 522,
iast_INACTIVE (433.802 µs) : 413, 455
.   : milestone, 434,
iast_TELEMETRY_OFF (462.026 µs) : 441, 483
.   : milestone, 462,
tracing (432.261 µs) : 411, 453
.   : milestone, 432,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	361.823 µs [341.977 µs, 381.669 µs]	-
iast	455.312 µs [434.46 µs, 476.164 µs]	93.489 µs (25.8%)
iast_FULL	524.125 µs [503.572 µs, 544.677 µs]	162.302 µs (44.9%)
iast_INACTIVE	430.62 µs [409.514 µs, 451.726 µs]	68.797 µs (19.0%)
iast_TELEMETRY_OFF	459.251 µs [437.527 µs, 480.976 µs]	97.428 µs (26.9%)
tracing	433.577 µs [412.121 µs, 455.033 µs]	71.754 µs (19.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	358.763 µs [338.565 µs, 378.962 µs]	-
iast	459.897 µs [439.018 µs, 480.775 µs]	101.133 µs (28.2%)
iast_FULL	521.807 µs [501.386 µs, 542.229 µs]	163.044 µs (45.4%)
iast_INACTIVE	433.802 µs [413.087 µs, 454.517 µs]	75.038 µs (20.9%)
iast_TELEMETRY_OFF	462.026 µs [440.94 µs, 483.112 µs]	103.263 µs (28.8%)
tracing	432.261 µs [411.169 µs, 453.353 µs]	73.498 µs (20.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~07ca0e595a, baseline=1.23.0-SNAPSHOT~047c47ca69
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.359 ms) : 1339, 1379
.   : milestone, 1359,
appsec (1.68 ms) : 1655, 1705
.   : milestone, 1680,
iast (1.456 ms) : 1433, 1480
.   : milestone, 1456,
profiling (1.48 ms) : 1455, 1505
.   : milestone, 1480,
tracing (1.459 ms) : 1435, 1483
.   : milestone, 1459,
section candidate
no_agent (1.342 ms) : 1322, 1361
.   : milestone, 1342,
appsec (1.692 ms) : 1667, 1717
.   : milestone, 1692,
iast (1.469 ms) : 1445, 1493
.   : milestone, 1469,
profiling (1.487 ms) : 1462, 1513
.   : milestone, 1487,
tracing (1.455 ms) : 1431, 1480
.   : milestone, 1455,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.359 ms [1.339 ms, 1.379 ms]	-
appsec	1.68 ms [1.655 ms, 1.705 ms]	320.753 µs (23.6%)
iast	1.456 ms [1.433 ms, 1.48 ms]	97.593 µs (7.2%)
profiling	1.48 ms [1.455 ms, 1.505 ms]	121.301 µs (8.9%)
tracing	1.459 ms [1.435 ms, 1.483 ms]	100.254 µs (7.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.342 ms [1.322 ms, 1.361 ms]	-
appsec	1.692 ms [1.667 ms, 1.717 ms]	350.226 µs (26.1%)
iast	1.469 ms [1.445 ms, 1.493 ms]	127.604 µs (9.5%)
profiling	1.487 ms [1.462 ms, 1.513 ms]	145.449 µs (10.8%)
tracing	1.455 ms [1.431 ms, 1.48 ms]	113.517 µs (8.5%)

dd-java-agent/agent-iast/src/main/java/com/datadog/iast/IastSystem.java

manuel-alvarez-alvarez added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling comp: asm iast Application Security Management (IAST) labels Oct 26, 2023

smola approved these changes Oct 27, 2023

View reviewed changes

dd-java-agent/agent-iast/src/main/java/com/datadog/iast/IastSystem.java Show resolved Hide resolved

manuel-alvarez-alvarez force-pushed the malvarez/iast-enable-nullaway branch from 323d21d to 47bbfd7 Compare October 30, 2023 09:25

manuel-alvarez-alvarez marked this pull request as ready for review October 30, 2023 09:26

manuel-alvarez-alvarez requested a review from a team as a code owner October 30, 2023 09:26

manuel-alvarez-alvarez requested a review from anderruiz October 30, 2023 09:26

manuel-alvarez-alvarez force-pushed the malvarez/iast-enable-nullaway branch 3 times, most recently from c5e0d75 to 12aec57 Compare November 1, 2023 10:17

smola approved these changes Nov 2, 2023

View reviewed changes

Add NullAway to IAST module and fix errors

07ca0e5

manuel-alvarez-alvarez force-pushed the malvarez/iast-enable-nullaway branch from 12aec57 to 07ca0e5 Compare November 2, 2023 12:56

manuel-alvarez-alvarez merged commit ee3049b into master Nov 2, 2023
10 of 11 checks passed

manuel-alvarez-alvarez deleted the malvarez/iast-enable-nullaway branch November 2, 2023 14:01

github-actions bot added this to the 1.23.0 milestone Nov 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NullAway to IAST module and fix errors #6106

Add NullAway to IAST module and fix errors #6106

manuel-alvarez-alvarez commented Oct 26, 2023 •

edited by smola

Loading

pr-commenter bot commented Oct 26, 2023 •

edited

Loading

Add NullAway to IAST module and fix errors #6106

Add NullAway to IAST module and fix errors #6106

Conversation

manuel-alvarez-alvarez commented Oct 26, 2023 • edited by smola Loading

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Oct 26, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

manuel-alvarez-alvarez commented Oct 26, 2023 •

edited by smola

Loading

pr-commenter bot commented Oct 26, 2023 •

edited

Loading