Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

nikita-tkachenko-datadog · 2023-10-31T18:43:07Z

What Does This Do

Fixes NullPointerException when JUnit 4 tracing listener receives events from a named parameterised test case.

Motivation

When a named parameterised test case is executed, there is a "test suite started" notification with a description that corresponds to a set of parameters.
The tracing listener is not interested in this event, but it should discard it rather than fail with exception.

Additional Notes

Jira ticket: CIVIS-7864

…tation

pr-commenter · 2023-10-31T19:25:03Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~4ad36f5440	1.23.0-SNAPSHOT~73f770884a
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1029013
Total [baseline] (8.771 s) : 0, 8770699
Agent [candidate] (1.027 s) : 0, 1027263
Total [candidate] (8.736 s) : 0, 8736169
section iast
Agent [baseline] (1.143 s) : 0, 1142746
Total [baseline] (9.267 s) : 0, 9266555
Agent [candidate] (1.15 s) : 0, 1150334
Total [candidate] (9.242 s) : 0, 9241899
section iast_TELEMETRY_OFF
Agent [baseline] (1.136 s) : 0, 1135522
Total [baseline] (9.243 s) : 0, 9242561
Agent [candidate] (1.147 s) : 0, 1146727
Total [candidate] (9.244 s) : 0, 9244228

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.029 s	-
Agent	iast	1.143 s	113.733 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.136 s	106.508 ms (10.4%)
Total	tracing	8.771 s	-
Total	iast	9.267 s	495.856 ms (5.7%)
Total	iast_TELEMETRY_OFF	9.243 s	471.862 ms (5.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	iast	1.15 s	123.071 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.147 s	119.464 ms (11.6%)
Total	tracing	8.736 s	-
Total	iast	9.242 s	505.73 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.244 s	508.059 ms (5.8%)

gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.314 ms) : 0, 642314
BytebuddyAgent [candidate] (642.278 ms) : 0, 642278
GlobalTracer [baseline] (295.576 ms) : 0, 295576
GlobalTracer [candidate] (294.434 ms) : 0, 294434
AppSec [baseline] (49.77 ms) : 0, 49770
AppSec [candidate] (49.201 ms) : 0, 49201
Remote Config [baseline] (688.394 µs) : 0, 688
Remote Config [candidate] (686.81 µs) : 0, 687
Telemetry [baseline] (6.156 ms) : 0, 6156
Telemetry [candidate] (6.118 ms) : 0, 6118
section iast
BytebuddyAgent [baseline] (761.374 ms) : 0, 761374
BytebuddyAgent [candidate] (769.679 ms) : 0, 769679
GlobalTracer [baseline] (274.61 ms) : 0, 274610
GlobalTracer [candidate] (273.638 ms) : 0, 273638
AppSec [baseline] (46.552 ms) : 0, 46552
AppSec [candidate] (47.596 ms) : 0, 47596
Remote Config [baseline] (558.31 µs) : 0, 558
Remote Config [candidate] (586.446 µs) : 0, 586
Telemetry [baseline] (7.477 ms) : 0, 7477
Telemetry [candidate] (8.187 ms) : 0, 8187
IAST [baseline] (17.627 ms) : 0, 17627
IAST [candidate] (16.329 ms) : 0, 16329
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (755.315 ms) : 0, 755315
BytebuddyAgent [candidate] (766.638 ms) : 0, 766638
GlobalTracer [baseline] (274.642 ms) : 0, 274642
GlobalTracer [candidate] (274.434 ms) : 0, 274434
AppSec [baseline] (46.767 ms) : 0, 46767
AppSec [candidate] (46.502 ms) : 0, 46502
Remote Config [baseline] (551.729 µs) : 0, 552
Remote Config [candidate] (615.811 µs) : 0, 616
Telemetry [baseline] (5.377 ms) : 0, 5377
Telemetry [candidate] (6.8 ms) : 0, 6800
IAST [baseline] (18.369 ms) : 0, 18369
IAST [candidate] (17.261 ms) : 0, 17261

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1026722
Total [baseline] (9.31 s) : 0, 9310049
Agent [candidate] (1.024 s) : 0, 1024128
Total [candidate] (9.266 s) : 0, 9265931
section appsec
Agent [baseline] (1.115 s) : 0, 1115253
Total [baseline] (9.348 s) : 0, 9347884
Agent [candidate] (1.125 s) : 0, 1125284
Total [candidate] (9.401 s) : 0, 9400718
section iast
Agent [baseline] (1.14 s) : 0, 1139743
Total [baseline] (9.418 s) : 0, 9418164
Agent [candidate] (1.153 s) : 0, 1153146
Total [candidate] (9.5 s) : 0, 9500017
section profiling
Agent [baseline] (1.202 s) : 0, 1202012
Total [baseline] (9.551 s) : 0, 9551040
Agent [candidate] (1.201 s) : 0, 1200974
Total [candidate] (9.489 s) : 0, 9489377

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	appsec	1.115 s	88.531 ms (8.6%)
Agent	iast	1.14 s	113.021 ms (11.0%)
Agent	profiling	1.202 s	175.29 ms (17.1%)
Total	tracing	9.31 s	-
Total	appsec	9.348 s	37.835 ms (0.4%)
Total	iast	9.418 s	108.114 ms (1.2%)
Total	profiling	9.551 s	240.991 ms (2.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.024 s	-
Agent	appsec	1.125 s	101.156 ms (9.9%)
Agent	iast	1.153 s	129.017 ms (12.6%)
Agent	profiling	1.201 s	176.846 ms (17.3%)
Total	tracing	9.266 s	-
Total	appsec	9.401 s	134.787 ms (1.5%)
Total	iast	9.5 s	234.086 ms (2.5%)
Total	profiling	9.489 s	223.446 ms (2.4%)

gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.122 ms) : 0, 642122
BytebuddyAgent [candidate] (640.542 ms) : 0, 640542
GlobalTracer [baseline] (294.581 ms) : 0, 294581
GlobalTracer [candidate] (293.326 ms) : 0, 293326
AppSec [baseline] (48.876 ms) : 0, 48876
AppSec [candidate] (49.071 ms) : 0, 49071
Remote Config [baseline] (678.899 µs) : 0, 679
Remote Config [candidate] (675.121 µs) : 0, 675
Telemetry [baseline] (6.079 ms) : 0, 6079
Telemetry [candidate] (6.097 ms) : 0, 6097
section appsec
BytebuddyAgent [baseline] (643.237 ms) : 0, 643237
BytebuddyAgent [candidate] (648.445 ms) : 0, 648445
GlobalTracer [baseline] (292.718 ms) : 0, 292718
GlobalTracer [candidate] (296.498 ms) : 0, 296498
AppSec [baseline] (138.558 ms) : 0, 138558
AppSec [candidate] (139.142 ms) : 0, 139142
Remote Config [baseline] (642.772 µs) : 0, 643
Remote Config [candidate] (649.549 µs) : 0, 650
Telemetry [baseline] (5.7 ms) : 0, 5700
Telemetry [candidate] (5.781 ms) : 0, 5781
section iast
BytebuddyAgent [baseline] (760.149 ms) : 0, 760149
BytebuddyAgent [candidate] (772.382 ms) : 0, 772382
GlobalTracer [baseline] (273.87 ms) : 0, 273870
GlobalTracer [candidate] (274.075 ms) : 0, 274075
AppSec [baseline] (46.558 ms) : 0, 46558
AppSec [candidate] (47.871 ms) : 0, 47871
Remote Config [baseline] (546.175 µs) : 0, 546
Remote Config [candidate] (1.275 ms) : 0, 1275
Telemetry [baseline] (6.096 ms) : 0, 6096
Telemetry [candidate] (6.836 ms) : 0, 6836
IAST [baseline] (18.191 ms) : 0, 18191
IAST [candidate] (16.214 ms) : 0, 16214
section profiling
BytebuddyAgent [baseline] (653.056 ms) : 0, 653056
BytebuddyAgent [candidate] (652.219 ms) : 0, 652219
GlobalTracer [baseline] (359.016 ms) : 0, 359016
GlobalTracer [candidate] (358.465 ms) : 0, 358465
AppSec [baseline] (49.371 ms) : 0, 49371
AppSec [candidate] (49.615 ms) : 0, 49615
Remote Config [baseline] (643.275 µs) : 0, 643
Remote Config [candidate] (644.695 µs) : 0, 645
Telemetry [baseline] (6.029 ms) : 0, 6029
Telemetry [candidate] (6.025 ms) : 0, 6025
ProfilingAgent [baseline] (80.573 ms) : 0, 80573
ProfilingAgent [candidate] (80.604 ms) : 0, 80604
Profiling [baseline] (80.597 ms) : 0, 80597
Profiling [candidate] (80.628 ms) : 0, 80628

Load

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~4ad36f5440	1.23.0-SNAPSHOT~73f770884a
config	baseline	candidate
end_time	2023-10-31T19:03:29	2023-10-31T19:19:49
start_time	2023-10-31T19:03:17	2023-10-31T19:19:36

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.047 µs) : 345, 385
.   : milestone, 365,
iast (457.26 µs) : 437, 478
.   : milestone, 457,
iast_FULL (522.281 µs) : 502, 543
.   : milestone, 522,
iast_INACTIVE (429.793 µs) : 409, 450
.   : milestone, 430,
iast_TELEMETRY_OFF (464.11 µs) : 442, 486
.   : milestone, 464,
tracing (422.081 µs) : 401, 443
.   : milestone, 422,
section candidate
no_agent (361.344 µs) : 341, 381
.   : milestone, 361,
iast (457.162 µs) : 436, 478
.   : milestone, 457,
iast_FULL (520.79 µs) : 500, 541
.   : milestone, 521,
iast_INACTIVE (430.716 µs) : 410, 452
.   : milestone, 431,
iast_TELEMETRY_OFF (449.57 µs) : 429, 470
.   : milestone, 450,
tracing (428.218 µs) : 407, 450
.   : milestone, 428,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.047 µs [344.786 µs, 385.307 µs]	-
iast	457.26 µs [436.635 µs, 477.884 µs]	92.213 µs (25.3%)
iast_FULL	522.281 µs [501.678 µs, 542.883 µs]	157.234 µs (43.1%)
iast_INACTIVE	429.793 µs [409.264 µs, 450.323 µs]	64.746 µs (17.7%)
iast_TELEMETRY_OFF	464.11 µs [442.416 µs, 485.804 µs]	99.063 µs (27.1%)
tracing	422.081 µs [401.452 µs, 442.709 µs]	57.034 µs (15.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	361.344 µs [341.401 µs, 381.286 µs]	-
iast	457.162 µs [436.123 µs, 478.2 µs]	95.818 µs (26.5%)
iast_FULL	520.79 µs [500.365 µs, 541.215 µs]	159.446 µs (44.1%)
iast_INACTIVE	430.716 µs [409.669 µs, 451.764 µs]	69.373 µs (19.2%)
iast_TELEMETRY_OFF	449.57 µs [428.848 µs, 470.291 µs]	88.226 µs (24.4%)
tracing	428.218 µs [406.914 µs, 449.521 µs]	66.874 µs (18.5%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~73f770884a, baseline=1.23.0-SNAPSHOT~4ad36f5440
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.337 ms) : 1318, 1357
.   : milestone, 1337,
appsec (1.729 ms) : 1705, 1754
.   : milestone, 1729,
iast (1.46 ms) : 1436, 1485
.   : milestone, 1460,
profiling (1.488 ms) : 1463, 1513
.   : milestone, 1488,
tracing (1.448 ms) : 1424, 1471
.   : milestone, 1448,
section candidate
no_agent (1.348 ms) : 1329, 1368
.   : milestone, 1348,
appsec (1.704 ms) : 1679, 1728
.   : milestone, 1704,
iast (1.467 ms) : 1443, 1491
.   : milestone, 1467,
profiling (1.501 ms) : 1475, 1527
.   : milestone, 1501,
tracing (1.445 ms) : 1420, 1470
.   : milestone, 1445,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.337 ms [1.318 ms, 1.357 ms]	-
appsec	1.729 ms [1.705 ms, 1.754 ms]	391.868 µs (29.3%)
iast	1.46 ms [1.436 ms, 1.485 ms]	122.891 µs (9.2%)
profiling	1.488 ms [1.463 ms, 1.513 ms]	150.468 µs (11.3%)
tracing	1.448 ms [1.424 ms, 1.471 ms]	110.062 µs (8.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.348 ms [1.329 ms, 1.368 ms]	-
appsec	1.704 ms [1.679 ms, 1.728 ms]	355.308 µs (26.4%)
iast	1.467 ms [1.443 ms, 1.491 ms]	118.824 µs (8.8%)
profiling	1.501 ms [1.475 ms, 1.527 ms]	152.417 µs (11.3%)
tracing	1.445 ms [1.42 ms, 1.47 ms]	96.75 µs (7.2%)

Fix NPE in when tracing named parametrized tests in JUnit 4 instrumen…

73f7708

…tation

nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Oct 31, 2023

nikita-tkachenko-datadog marked this pull request as ready for review November 3, 2023 14:30

nikita-tkachenko-datadog requested a review from a team as a code owner November 3, 2023 14:30

nikita-tkachenko-datadog requested a review from tonyredondo November 3, 2023 14:30

romainkomorn-exdatadog approved these changes Nov 3, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit 96bc2b9 into master Nov 3, 2023
12 of 13 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/junit4-npe branch November 3, 2023 19:16

github-actions bot added this to the 1.23.0 milestone Nov 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

nikita-tkachenko-datadog commented Oct 31, 2023 •

edited by jira bot

Loading

pr-commenter bot commented Oct 31, 2023

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

Fix NPE when tracing named parameterised tests in JUnit 4 instrumentation #6125

Conversation

nikita-tkachenko-datadog commented Oct 31, 2023 • edited by jira bot Loading

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Oct 31, 2023

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

nikita-tkachenko-datadog commented Oct 31, 2023 •

edited by jira bot

Loading