Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f #6199

smola · 2023-11-13T07:09:00Z

What Does This Do

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

pr-commenter · 2023-11-13T07:53:55Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~768bfbd86b	1.24.0-SNAPSHOT~c41141a46b
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1036060
Total [baseline] (9.326 s) : 0, 9326141
Agent [candidate] (1.051 s) : 0, 1051128
Total [candidate] (9.352 s) : 0, 9351598
section appsec
Agent [baseline] (1.119 s) : 0, 1119487
Total [baseline] (9.393 s) : 0, 9393437
Agent [candidate] (1.132 s) : 0, 1131905
Total [candidate] (9.554 s) : 0, 9553995
section iast
Agent [baseline] (1.149 s) : 0, 1148988
Total [baseline] (9.432 s) : 0, 9431902
Agent [candidate] (1.151 s) : 0, 1151065
Total [candidate] (9.447 s) : 0, 9446781
section profiling
Agent [baseline] (1.218 s) : 0, 1217824
Total [baseline] (9.503 s) : 0, 9503500
Agent [candidate] (1.22 s) : 0, 1219543
Total [candidate] (9.499 s) : 0, 9498655

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	appsec	1.119 s	83.427 ms (8.1%)
Agent	iast	1.149 s	112.928 ms (10.9%)
Agent	profiling	1.218 s	181.764 ms (17.5%)
Total	tracing	9.326 s	-
Total	appsec	9.393 s	67.296 ms (0.7%)
Total	iast	9.432 s	105.761 ms (1.1%)
Total	profiling	9.503 s	177.359 ms (1.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.051 s	-
Agent	appsec	1.132 s	80.777 ms (7.7%)
Agent	iast	1.151 s	99.936 ms (9.5%)
Agent	profiling	1.22 s	168.415 ms (16.0%)
Total	tracing	9.352 s	-
Total	appsec	9.554 s	202.397 ms (2.2%)
Total	iast	9.447 s	95.183 ms (1.0%)
Total	profiling	9.499 s	147.057 ms (1.6%)

gantt
    title petclinic - break down per module: candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (647.008 ms) : 0, 647008
BytebuddyAgent [candidate] (656.377 ms) : 0, 656377
GlobalTracer [baseline] (293.943 ms) : 0, 293943
GlobalTracer [candidate] (298.472 ms) : 0, 298472
AppSec [baseline] (48.56 ms) : 0, 48560
AppSec [candidate] (49.272 ms) : 0, 49272
Remote Config [baseline] (695.581 µs) : 0, 696
Remote Config [candidate] (716.736 µs) : 0, 717
Telemetry [baseline] (11.255 ms) : 0, 11255
Telemetry [candidate] (11.389 ms) : 0, 11389
section appsec
BytebuddyAgent [baseline] (644.946 ms) : 0, 644946
BytebuddyAgent [candidate] (652.017 ms) : 0, 652017
GlobalTracer [baseline] (293.689 ms) : 0, 293689
GlobalTracer [candidate] (296.834 ms) : 0, 296834
AppSec [baseline] (138.783 ms) : 0, 138783
AppSec [candidate] (140.646 ms) : 0, 140646
Remote Config [baseline] (649.002 µs) : 0, 649
Remote Config [candidate] (651.057 µs) : 0, 651
Telemetry [baseline] (6.856 ms) : 0, 6856
Telemetry [candidate] (6.961 ms) : 0, 6961
section iast
BytebuddyAgent [baseline] (763.927 ms) : 0, 763927
BytebuddyAgent [candidate] (765.802 ms) : 0, 765802
GlobalTracer [baseline] (274.291 ms) : 0, 274291
GlobalTracer [candidate] (274.232 ms) : 0, 274232
AppSec [baseline] (46.402 ms) : 0, 46402
AppSec [candidate] (46.867 ms) : 0, 46867
Remote Config [baseline] (572.41 µs) : 0, 572
Remote Config [candidate] (581.342 µs) : 0, 581
Telemetry [baseline] (12.562 ms) : 0, 12562
Telemetry [candidate] (14.019 ms) : 0, 14019
IAST [baseline] (16.783 ms) : 0, 16783
IAST [candidate] (15.043 ms) : 0, 15043
section profiling
BytebuddyAgent [baseline] (656.182 ms) : 0, 656182
BytebuddyAgent [candidate] (656.927 ms) : 0, 656927
GlobalTracer [baseline] (358.556 ms) : 0, 358556
GlobalTracer [candidate] (358.27 ms) : 0, 358270
AppSec [baseline] (49.233 ms) : 0, 49233
AppSec [candidate] (49.046 ms) : 0, 49046
Remote Config [baseline] (644.45 µs) : 0, 644
Remote Config [candidate] (643.293 µs) : 0, 643
Telemetry [baseline] (11.276 ms) : 0, 11276
Telemetry [candidate] (11.385 ms) : 0, 11385
ProfilingAgent [baseline] (87.573 ms) : 0, 87573
ProfilingAgent [candidate] (89.013 ms) : 0, 89013
Profiling [baseline] (87.597 ms) : 0, 87597
Profiling [candidate] (89.037 ms) : 0, 89037

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.035 s) : 0, 1034805
Total [baseline] (8.782 s) : 0, 8781949
Agent [candidate] (1.044 s) : 0, 1043727
Total [candidate] (8.789 s) : 0, 8788716
section iast
Agent [baseline] (1.148 s) : 0, 1148489
Total [baseline] (9.323 s) : 0, 9322533
Agent [candidate] (1.151 s) : 0, 1150528
Total [candidate] (9.29 s) : 0, 9290375
section iast_TELEMETRY_OFF
Agent [baseline] (1.142 s) : 0, 1142350
Total [baseline] (9.333 s) : 0, 9333258
Agent [candidate] (1.145 s) : 0, 1144854
Total [candidate] (9.321 s) : 0, 9321374

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.035 s	-
Agent	iast	1.148 s	113.684 ms (11.0%)
Agent	iast_TELEMETRY_OFF	1.142 s	107.545 ms (10.4%)
Total	tracing	8.782 s	-
Total	iast	9.323 s	540.584 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.333 s	551.309 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.151 s	106.801 ms (10.2%)
Agent	iast_TELEMETRY_OFF	1.145 s	101.127 ms (9.7%)
Total	tracing	8.789 s	-
Total	iast	9.29 s	501.659 ms (5.7%)
Total	iast_TELEMETRY_OFF	9.321 s	532.658 ms (6.1%)

gantt
    title insecure-bank - break down per module: candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.333 ms) : 0, 645333
BytebuddyAgent [candidate] (650.727 ms) : 0, 650727
GlobalTracer [baseline] (293.949 ms) : 0, 293949
GlobalTracer [candidate] (296.714 ms) : 0, 296714
AppSec [baseline] (48.87 ms) : 0, 48870
AppSec [candidate] (49.344 ms) : 0, 49344
Remote Config [baseline] (700.704 µs) : 0, 701
Remote Config [candidate] (709.592 µs) : 0, 710
Telemetry [baseline] (11.421 ms) : 0, 11421
Telemetry [candidate] (11.451 ms) : 0, 11451
section iast
BytebuddyAgent [baseline] (762.857 ms) : 0, 762857
BytebuddyAgent [candidate] (765.222 ms) : 0, 765222
GlobalTracer [baseline] (273.241 ms) : 0, 273241
GlobalTracer [candidate] (274.11 ms) : 0, 274110
AppSec [baseline] (46.633 ms) : 0, 46633
AppSec [candidate] (46.573 ms) : 0, 46573
IAST [baseline] (15.464 ms) : 0, 15464
IAST [candidate] (15.486 ms) : 0, 15486
Remote Config [baseline] (567.901 µs) : 0, 568
Remote Config [candidate] (564.068 µs) : 0, 564
Telemetry [baseline] (15.422 ms) : 0, 15422
Telemetry [candidate] (14.118 ms) : 0, 14118
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (757.957 ms) : 0, 757957
BytebuddyAgent [candidate] (759.966 ms) : 0, 759966
GlobalTracer [baseline] (273.868 ms) : 0, 273868
GlobalTracer [candidate] (274.75 ms) : 0, 274750
AppSec [baseline] (46.29 ms) : 0, 46290
AppSec [candidate] (46.49 ms) : 0, 46490
IAST [baseline] (14.584 ms) : 0, 14584
IAST [candidate] (15.3 ms) : 0, 15300
Remote Config [baseline] (554.102 µs) : 0, 554
Remote Config [candidate] (565.824 µs) : 0, 566
Telemetry [baseline] (14.677 ms) : 0, 14677
Telemetry [candidate] (13.331 ms) : 0, 13331

Load

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~768bfbd86b	1.24.0-SNAPSHOT~c41141a46b
config	baseline	candidate
end_time	2023-11-13T07:32:12	2023-11-13T07:48:40
start_time	2023-11-13T07:31:59	2023-11-13T07:48:28

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.35 ms) : 1331, 1369
.   : milestone, 1350,
appsec (1.691 ms) : 1667, 1716
.   : milestone, 1691,
iast (1.477 ms) : 1453, 1500
.   : milestone, 1477,
profiling (1.492 ms) : 1467, 1518
.   : milestone, 1492,
tracing (1.464 ms) : 1440, 1489
.   : milestone, 1464,
section candidate
no_agent (1.343 ms) : 1323, 1363
.   : milestone, 1343,
appsec (1.738 ms) : 1713, 1764
.   : milestone, 1738,
iast (1.473 ms) : 1449, 1497
.   : milestone, 1473,
profiling (1.48 ms) : 1454, 1506
.   : milestone, 1480,
tracing (1.456 ms) : 1431, 1482
.   : milestone, 1456,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.35 ms [1.331 ms, 1.369 ms]	-
appsec	1.691 ms [1.667 ms, 1.716 ms]	341.861 µs (25.3%)
iast	1.477 ms [1.453 ms, 1.5 ms]	126.904 µs (9.4%)
profiling	1.492 ms [1.467 ms, 1.518 ms]	142.854 µs (10.6%)
tracing	1.464 ms [1.44 ms, 1.489 ms]	114.868 µs (8.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.343 ms [1.323 ms, 1.363 ms]	-
appsec	1.738 ms [1.713 ms, 1.764 ms]	395.832 µs (29.5%)
iast	1.473 ms [1.449 ms, 1.497 ms]	130.384 µs (9.7%)
profiling	1.48 ms [1.454 ms, 1.506 ms]	137.418 µs (10.2%)
tracing	1.456 ms [1.431 ms, 1.482 ms]	113.701 µs (8.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~c41141a46b, baseline=1.24.0-SNAPSHOT~768bfbd86b
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.478 µs) : 342, 385
.   : milestone, 363,
iast (458.564 µs) : 438, 479
.   : milestone, 459,
iast_FULL (522.943 µs) : 502, 544
.   : milestone, 523,
iast_INACTIVE (435.554 µs) : 415, 456
.   : milestone, 436,
iast_TELEMETRY_OFF (455.582 µs) : 434, 477
.   : milestone, 456,
tracing (432.681 µs) : 411, 454
.   : milestone, 433,
section candidate
no_agent (362.337 µs) : 342, 382
.   : milestone, 362,
iast (459.399 µs) : 439, 480
.   : milestone, 459,
iast_FULL (518.797 µs) : 498, 539
.   : milestone, 519,
iast_INACTIVE (440.728 µs) : 419, 462
.   : milestone, 441,
iast_TELEMETRY_OFF (455.125 µs) : 434, 476
.   : milestone, 455,
tracing (439.817 µs) : 418, 461
.   : milestone, 440,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.478 µs [341.921 µs, 385.035 µs]	-
iast	458.564 µs [437.782 µs, 479.346 µs]	95.086 µs (26.2%)
iast_FULL	522.943 µs [502.286 µs, 543.6 µs]	159.464 µs (43.9%)
iast_INACTIVE	435.554 µs [414.827 µs, 456.281 µs]	72.076 µs (19.8%)
iast_TELEMETRY_OFF	455.582 µs [434.198 µs, 476.965 µs]	92.103 µs (25.3%)
tracing	432.681 µs [411.022 µs, 454.341 µs]	69.203 µs (19.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	362.337 µs [342.392 µs, 382.281 µs]	-
iast	459.399 µs [438.972 µs, 479.825 µs]	97.062 µs (26.8%)
iast_FULL	518.797 µs [498.221 µs, 539.374 µs]	156.461 µs (43.2%)
iast_INACTIVE	440.728 µs [419.209 µs, 462.247 µs]	78.392 µs (21.6%)
iast_TELEMETRY_OFF	455.125 µs [434.243 µs, 476.006 µs]	92.788 µs (25.6%)
tracing	439.817 µs [418.405 µs, 461.229 µs]	77.48 µs (21.4%)

Fix error when trying to get class stream for Mockito mocks (#6183) Fix error when serializing skippable tests whose names contain non-ASCII characters (#6182) Fail fast when tracer versions do not match in parent and child processes (#6168) initial instrumentation

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f

c41141a

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Nov 13, 2023

smola requested a review from a team as a code owner November 13, 2023 07:09

smola requested review from PerfectSlayer, am312, a team, ValentinZakharov and jandro996 November 13, 2023 07:09

PerfectSlayer approved these changes Nov 13, 2023

View reviewed changes

smola enabled auto-merge (squash) November 13, 2023 08:52

manuel-alvarez-alvarez approved these changes Nov 13, 2023

View reviewed changes

Merge branch 'master' into smola/system-tests-20231113

b758891

smola merged commit 17734ae into master Nov 13, 2023
9 of 10 checks passed

smola deleted the smola/system-tests-20231113 branch November 13, 2023 10:45

github-actions bot added this to the 1.24.0 milestone Nov 13, 2023

jandro996 pushed a commit that referenced this pull request Nov 14, 2023

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f (#6199)

11955c8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f #6199

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f #6199

smola commented Nov 13, 2023

pr-commenter bot commented Nov 13, 2023

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f #6199

Update system-tests to 6998b307ce0e8d206ca6eb99045651dcc8bc853f #6199

Conversation

smola commented Nov 13, 2023

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Nov 13, 2023

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary