Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add APIs for llm obs #8135

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

add APIs for llm obs #8135

wants to merge 1 commit into from

Conversation

gary-huang
Copy link
Contributor

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Dec 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/add-llm-obs-api
git_commit_date 1736853230 1736861311
git_commit_sha 82e3386 62bc9a0
release_version 1.46.0-SNAPSHOT~82e3386fa9 1.46.0-SNAPSHOT~62bc9a024e
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1736863842 1736863842
ci_job_id 765363283 765363283
ci_pipeline_id 52929355 52929355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060368
Total [baseline] (10.422 s) : 0, 10421662
Agent [candidate] (1.062 s) : 0, 1062192
Total [candidate] (10.53 s) : 0, 10530343
section appsec
Agent [baseline] (1.187 s) : 0, 1186963
Total [baseline] (10.682 s) : 0, 10681782
Agent [candidate] (1.196 s) : 0, 1195889
Total [candidate] (10.748 s) : 0, 10747630
section iast
Agent [baseline] (1.193 s) : 0, 1192957
Total [baseline] (11.018 s) : 0, 11018458
Agent [candidate] (1.187 s) : 0, 1186929
Total [candidate] (10.971 s) : 0, 10970595
section profiling
Agent [baseline] (1.274 s) : 0, 1273521
Total [baseline] (11.007 s) : 0, 11006829
Agent [candidate] (1.268 s) : 0, 1268303
Total [candidate] (10.888 s) : 0, 10887682
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.187 s 126.595 ms (11.9%)
Agent iast 1.193 s 132.589 ms (12.5%)
Agent profiling 1.274 s 213.153 ms (20.1%)
Total tracing 10.422 s -
Total appsec 10.682 s 260.121 ms (2.5%)
Total iast 11.018 s 596.797 ms (5.7%)
Total profiling 11.007 s 585.168 ms (5.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.196 s 133.697 ms (12.6%)
Agent iast 1.187 s 124.737 ms (11.7%)
Agent profiling 1.268 s 206.112 ms (19.4%)
Total tracing 10.53 s -
Total appsec 10.748 s 217.287 ms (2.1%)
Total iast 10.971 s 440.252 ms (4.2%)
Total profiling 10.888 s 357.339 ms (3.4%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.516 ms) : 0, 718516
BytebuddyAgent [candidate] (719.812 ms) : 0, 719812
GlobalTracer [baseline] (255.654 ms) : 0, 255654
GlobalTracer [candidate] (257.83 ms) : 0, 257830
AppSec [baseline] (55.247 ms) : 0, 55247
AppSec [candidate] (56.37 ms) : 0, 56370
Remote Config [baseline] (721.216 µs) : 0, 721
Remote Config [candidate] (736.959 µs) : 0, 737
Telemetry [baseline] (15.195 ms) : 0, 15195
Telemetry [candidate] (12.349 ms) : 0, 12349
section appsec
BytebuddyAgent [baseline] (729.839 ms) : 0, 729839
BytebuddyAgent [candidate] (736.013 ms) : 0, 736013
GlobalTracer [baseline] (252.898 ms) : 0, 252898
GlobalTracer [candidate] (254.344 ms) : 0, 254344
AppSec [baseline] (170.276 ms) : 0, 170276
AppSec [candidate] (171.78 ms) : 0, 171780
IAST [baseline] (20.105 ms) : 0, 20105
IAST [candidate] (19.521 ms) : 0, 19521
Remote Config [baseline] (678.646 µs) : 0, 679
Remote Config [candidate] (663.199 µs) : 0, 663
Telemetry [baseline] (8.226 ms) : 0, 8226
Telemetry [candidate] (8.229 ms) : 0, 8229
section iast
BytebuddyAgent [baseline] (838.803 ms) : 0, 838803
BytebuddyAgent [candidate] (834.434 ms) : 0, 834434
GlobalTracer [baseline] (248.971 ms) : 0, 248971
GlobalTracer [candidate] (247.898 ms) : 0, 247898
AppSec [baseline] (58.441 ms) : 0, 58441
AppSec [candidate] (58.084 ms) : 0, 58084
IAST [baseline] (21.902 ms) : 0, 21902
IAST [candidate] (21.761 ms) : 0, 21761
Remote Config [baseline] (684.755 µs) : 0, 685
Remote Config [candidate] (669.538 µs) : 0, 670
Telemetry [baseline] (8.975 ms) : 0, 8975
Telemetry [candidate] (8.861 ms) : 0, 8861
section profiling
BytebuddyAgent [baseline] (702.622 ms) : 0, 702622
BytebuddyAgent [candidate] (702.68 ms) : 0, 702680
GlobalTracer [baseline] (368.104 ms) : 0, 368104
GlobalTracer [candidate] (365.813 ms) : 0, 365813
AppSec [baseline] (54.76 ms) : 0, 54760
AppSec [candidate] (53.956 ms) : 0, 53956
Remote Config [baseline] (698.174 µs) : 0, 698
Remote Config [candidate] (690.557 µs) : 0, 691
Telemetry [baseline] (8.887 ms) : 0, 8887
Telemetry [candidate] (8.796 ms) : 0, 8796
ProfilingAgent [baseline] (96.513 ms) : 0, 96513
ProfilingAgent [candidate] (94.53 ms) : 0, 94530
Profiling [baseline] (96.537 ms) : 0, 96537
Profiling [candidate] (94.554 ms) : 0, 94554
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061199
Total [baseline] (8.641 s) : 0, 8640869
Agent [candidate] (1.06 s) : 0, 1059631
Total [candidate] (8.669 s) : 0, 8669460
section iast
Agent [baseline] (1.184 s) : 0, 1183509
Total [baseline] (9.22 s) : 0, 9219698
Agent [candidate] (1.181 s) : 0, 1181063
Total [candidate] (9.206 s) : 0, 9206042
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.181 s) : 0, 1180501
Total [baseline] (9.197 s) : 0, 9197007
Agent [candidate] (1.19 s) : 0, 1190262
Total [candidate] (9.225 s) : 0, 9225448
section iast_TELEMETRY_OFF
Agent [baseline] (1.183 s) : 0, 1182712
Total [baseline] (9.235 s) : 0, 9235470
Agent [candidate] (1.176 s) : 0, 1176435
Total [candidate] (9.21 s) : 0, 9209611
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.184 s 122.31 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.181 s 119.302 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.183 s 121.513 ms (11.5%)
Total tracing 8.641 s -
Total iast 9.22 s 578.828 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.197 s 556.138 ms (6.4%)
Total iast_TELEMETRY_OFF 9.235 s 594.6 ms (6.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent iast 1.181 s 121.433 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.19 s 130.631 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.176 s 116.804 ms (11.0%)
Total tracing 8.669 s -
Total iast 9.206 s 536.582 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.225 s 555.988 ms (6.4%)
Total iast_TELEMETRY_OFF 9.21 s 540.152 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.221 ms) : 0, 718221
BytebuddyAgent [candidate] (717.318 ms) : 0, 717318
GlobalTracer [baseline] (255.725 ms) : 0, 255725
GlobalTracer [candidate] (256.632 ms) : 0, 256632
AppSec [baseline] (56.098 ms) : 0, 56098
AppSec [candidate] (55.493 ms) : 0, 55493
Remote Config [baseline] (726.158 µs) : 0, 726
Remote Config [candidate] (729.043 µs) : 0, 729
Telemetry [baseline] (15.342 ms) : 0, 15342
Telemetry [candidate] (14.414 ms) : 0, 14414
section iast
BytebuddyAgent [baseline] (832.349 ms) : 0, 832349
BytebuddyAgent [candidate] (830.558 ms) : 0, 830558
GlobalTracer [baseline] (247.098 ms) : 0, 247098
GlobalTracer [candidate] (246.413 ms) : 0, 246413
AppSec [baseline] (58.195 ms) : 0, 58195
AppSec [candidate] (57.914 ms) : 0, 57914
IAST [baseline] (21.449 ms) : 0, 21449
IAST [candidate] (21.553 ms) : 0, 21553
Remote Config [baseline] (670.756 µs) : 0, 671
Remote Config [candidate] (680.586 µs) : 0, 681
Telemetry [baseline] (8.793 ms) : 0, 8793
Telemetry [candidate] (8.837 ms) : 0, 8837
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (829.599 ms) : 0, 829599
BytebuddyAgent [candidate] (837.002 ms) : 0, 837002
GlobalTracer [baseline] (246.482 ms) : 0, 246482
GlobalTracer [candidate] (248.262 ms) : 0, 248262
AppSec [baseline] (58.231 ms) : 0, 58231
AppSec [candidate] (58.652 ms) : 0, 58652
IAST [baseline] (21.675 ms) : 0, 21675
IAST [candidate] (21.658 ms) : 0, 21658
Remote Config [baseline] (678.084 µs) : 0, 678
Remote Config [candidate] (681.265 µs) : 0, 681
Telemetry [baseline] (8.859 ms) : 0, 8859
Telemetry [candidate] (8.935 ms) : 0, 8935
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (831.643 ms) : 0, 831643
BytebuddyAgent [candidate] (827.206 ms) : 0, 827206
GlobalTracer [baseline] (247.785 ms) : 0, 247785
GlobalTracer [candidate] (246.446 ms) : 0, 246446
AppSec [baseline] (57.854 ms) : 0, 57854
AppSec [candidate] (57.674 ms) : 0, 57674
IAST [baseline] (21.012 ms) : 0, 21012
IAST [candidate] (20.723 ms) : 0, 20723
Remote Config [baseline] (665.736 µs) : 0, 666
Remote Config [candidate] (663.99 µs) : 0, 664
Telemetry [baseline] (8.729 ms) : 0, 8729
Telemetry [candidate] (8.685 ms) : 0, 8685
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-14T13:40:48 2025-01-14T13:47:47
git_branch master gary/add-llm-obs-api
git_commit_date 1736853230 1736861311
git_commit_sha 82e3386 62bc9a0
release_version 1.46.0-SNAPSHOT~82e3386fa9 1.46.0-SNAPSHOT~62bc9a024e
start_time 2025-01-14T13:40:35 2025-01-14T13:47:34
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1736862821 1736862821
ci_job_id 765363284 765363284
ci_pipeline_id 52929355 52929355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.367 ms) : 1349, 1386
.   : milestone, 1367,
appsec (1.759 ms) : 1736, 1783
.   : milestone, 1759,
appsec_no_iast (1.752 ms) : 1729, 1775
.   : milestone, 1752,
iast (1.52 ms) : 1497, 1543
.   : milestone, 1520,
profiling (1.529 ms) : 1504, 1554
.   : milestone, 1529,
tracing (1.503 ms) : 1479, 1527
.   : milestone, 1503,
section candidate
no_agent (1.366 ms) : 1347, 1385
.   : milestone, 1366,
appsec (1.739 ms) : 1715, 1763
.   : milestone, 1739,
appsec_no_iast (1.748 ms) : 1723, 1772
.   : milestone, 1748,
iast (1.495 ms) : 1472, 1518
.   : milestone, 1495,
profiling (1.56 ms) : 1535, 1585
.   : milestone, 1560,
tracing (1.505 ms) : 1481, 1529
.   : milestone, 1505,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.367 ms [1.349 ms, 1.386 ms] -
appsec 1.759 ms [1.736 ms, 1.783 ms] 392.111 µs (28.7%)
appsec_no_iast 1.752 ms [1.729 ms, 1.775 ms] 384.295 µs (28.1%)
iast 1.52 ms [1.497 ms, 1.543 ms] 152.935 µs (11.2%)
profiling 1.529 ms [1.504 ms, 1.554 ms] 161.73 µs (11.8%)
tracing 1.503 ms [1.479 ms, 1.527 ms] 136.093 µs (10.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.366 ms [1.347 ms, 1.385 ms] -
appsec 1.739 ms [1.715 ms, 1.763 ms] 373.29 µs (27.3%)
appsec_no_iast 1.748 ms [1.723 ms, 1.772 ms] 381.722 µs (27.9%)
iast 1.495 ms [1.472 ms, 1.518 ms] 129.352 µs (9.5%)
profiling 1.56 ms [1.535 ms, 1.585 ms] 194.282 µs (14.2%)
tracing 1.505 ms [1.481 ms, 1.529 ms] 139.138 µs (10.2%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.508 µs) : 355, 396
.   : milestone, 376,
iast (497.13 µs) : 475, 519
.   : milestone, 497,
iast_FULL (651.586 µs) : 630, 673
.   : milestone, 652,
iast_GLOBAL (520.209 µs) : 498, 543
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (487.72 µs) : 466, 509
.   : milestone, 488,
iast_INACTIVE (454.382 µs) : 433, 476
.   : milestone, 454,
iast_TELEMETRY_OFF (488.212 µs) : 466, 510
.   : milestone, 488,
tracing (446.864 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (380.546 µs) : 361, 400
.   : milestone, 381,
iast (497.465 µs) : 476, 519
.   : milestone, 497,
iast_FULL (655.576 µs) : 634, 677
.   : milestone, 656,
iast_GLOBAL (516.131 µs) : 495, 537
.   : milestone, 516,
iast_HARDCODED_SECRET_DISABLED (488.374 µs) : 467, 510
.   : milestone, 488,
iast_INACTIVE (452.414 µs) : 431, 474
.   : milestone, 452,
iast_TELEMETRY_OFF (494.996 µs) : 471, 519
.   : milestone, 495,
tracing (449.118 µs) : 428, 471
.   : milestone, 449,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.508 µs [355.356 µs, 395.659 µs] -
iast 497.13 µs [475.037 µs, 519.223 µs] 121.622 µs (32.4%)
iast_FULL 651.586 µs [630.086 µs, 673.085 µs] 276.078 µs (73.5%)
iast_GLOBAL 520.209 µs [497.589 µs, 542.829 µs] 144.701 µs (38.5%)
iast_HARDCODED_SECRET_DISABLED 487.72 µs [466.146 µs, 509.294 µs] 112.212 µs (29.9%)
iast_INACTIVE 454.382 µs [432.709 µs, 476.056 µs] 78.874 µs (21.0%)
iast_TELEMETRY_OFF 488.212 µs [466.304 µs, 510.119 µs] 112.704 µs (30.0%)
tracing 446.864 µs [426.191 µs, 467.537 µs] 71.356 µs (19.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.546 µs [360.933 µs, 400.158 µs] -
iast 497.465 µs [475.706 µs, 519.223 µs] 116.919 µs (30.7%)
iast_FULL 655.576 µs [633.881 µs, 677.272 µs] 275.03 µs (72.3%)
iast_GLOBAL 516.131 µs [494.915 µs, 537.348 µs] 135.586 µs (35.6%)
iast_HARDCODED_SECRET_DISABLED 488.374 µs [467.088 µs, 509.661 µs] 107.829 µs (28.3%)
iast_INACTIVE 452.414 µs [431.204 µs, 473.623 µs] 71.868 µs (18.9%)
iast_TELEMETRY_OFF 494.996 µs [471.076 µs, 518.916 µs] 114.45 µs (30.1%)
tracing 449.118 µs [427.619 µs, 470.618 µs] 68.573 µs (18.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/add-llm-obs-api
git_commit_date 1736853230 1736861311
git_commit_sha 82e3386 62bc9a0
release_version 1.46.0-SNAPSHOT~82e3386fa9 1.46.0-SNAPSHOT~62bc9a024e
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1736863401 1736863401
ci_job_id 765363285 765363285
ci_pipeline_id 52929355 52929355
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.894 s) : 14894000, 14894000
.   : milestone, 14894000,
appsec (15.073 s) : 15073000, 15073000
.   : milestone, 15073000,
iast (18.881 s) : 18881000, 18881000
.   : milestone, 18881000,
iast_GLOBAL (17.84 s) : 17840000, 17840000
.   : milestone, 17840000,
profiling (14.952 s) : 14952000, 14952000
.   : milestone, 14952000,
tracing (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
section candidate
no_agent (15.277 s) : 15277000, 15277000
.   : milestone, 15277000,
appsec (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
iast (18.657 s) : 18657000, 18657000
.   : milestone, 18657000,
iast_GLOBAL (17.883 s) : 17883000, 17883000
.   : milestone, 17883000,
profiling (15.121 s) : 15121000, 15121000
.   : milestone, 15121000,
tracing (14.803 s) : 14803000, 14803000
.   : milestone, 14803000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.894 s [14.894 s, 14.894 s] -
appsec 15.073 s [15.073 s, 15.073 s] 179.0 ms (1.2%)
iast 18.881 s [18.881 s, 18.881 s] 3.987 s (26.8%)
iast_GLOBAL 17.84 s [17.84 s, 17.84 s] 2.946 s (19.8%)
profiling 14.952 s [14.952 s, 14.952 s] 58.0 ms (0.4%)
tracing 14.911 s [14.911 s, 14.911 s] 17.0 ms (0.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.277 s [15.277 s, 15.277 s] -
appsec 15.024 s [15.024 s, 15.024 s] -253.0 ms (-1.7%)
iast 18.657 s [18.657 s, 18.657 s] 3.38 s (22.1%)
iast_GLOBAL 17.883 s [17.883 s, 17.883 s] 2.606 s (17.1%)
profiling 15.121 s [15.121 s, 15.121 s] -156.0 ms (-1.0%)
tracing 14.803 s [14.803 s, 14.803 s] -474.0 ms (-3.1%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~62bc9a024e, baseline=1.46.0-SNAPSHOT~82e3386fa9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.35 ms) : 2307, 2393
.   : milestone, 2350,
iast (2.098 ms) : 2044, 2153
.   : milestone, 2098,
iast_GLOBAL (2.14 ms) : 2085, 2194
.   : milestone, 2140,
profiling (1.977 ms) : 1933, 2021
.   : milestone, 1977,
tracing (1.938 ms) : 1897, 1980
.   : milestone, 1938,
section candidate
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.353 ms) : 2310, 2396
.   : milestone, 2353,
iast (2.103 ms) : 2049, 2158
.   : milestone, 2103,
iast_GLOBAL (2.153 ms) : 2098, 2208
.   : milestone, 2153,
profiling (1.984 ms) : 1939, 2029
.   : milestone, 1984,
tracing (1.949 ms) : 1907, 1991
.   : milestone, 1949,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.477 ms] -
appsec 2.35 ms [2.307 ms, 2.393 ms] 884.683 µs (60.4%)
iast 2.098 ms [2.044 ms, 2.153 ms] 632.827 µs (43.2%)
iast_GLOBAL 2.14 ms [2.085 ms, 2.194 ms] 674.152 µs (46.0%)
profiling 1.977 ms [1.933 ms, 2.021 ms] 511.558 µs (34.9%)
tracing 1.938 ms [1.897 ms, 1.98 ms] 472.95 µs (32.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.477 ms] -
appsec 2.353 ms [2.31 ms, 2.396 ms] 888.123 µs (60.6%)
iast 2.103 ms [2.049 ms, 2.158 ms] 637.935 µs (43.5%)
iast_GLOBAL 2.153 ms [2.098 ms, 2.208 ms] 688.237 µs (47.0%)
profiling 1.984 ms [1.939 ms, 2.029 ms] 518.765 µs (35.4%)
tracing 1.949 ms [1.907 ms, 1.991 ms] 483.743 µs (33.0%)

@gary-huang gary-huang force-pushed the gary/add-llm-obs-api branch from 85d0946 to dd637fd Compare January 2, 2025 19:08
mcculls
mcculls reviewed Jan 2, 2025
View reviewed changes
dd-trace-api/src/main/java/datadog/trace/api/llmobs/LLMObs.java Outdated
private static LLMObsSpanFactory SPAN_FACTORY = NoOpLLMObsSpanFactory.INSTANCE;

/**
* This a hook for injecting SpanFactory implementation. It should only be used internally by
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, internal APIs should be added to the internal-api module.

You can keep the same package structure and they will still be visible to the tracer internals.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah gotcha, thanks!

@mcculls mcculls force-pushed the gary/add-llmobs-configs branch from 14f71b3 to 5d90450 Compare January 7, 2025 09:35
Base automatically changed from gary/add-llmobs-configs to master January 7, 2025 15:36
@gary-huang gary-huang force-pushed the gary/add-llm-obs-api branch from dd637fd to 85debfc Compare January 14, 2025 13:24
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

@gary-huang gary-huang force-pushed the gary/add-llm-obs-api branch from 85debfc to 98060a9 Compare January 14, 2025 13:27
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Looks like you updated a Git Submodule.
If this was not intentional please make sure to:

@gary-huang gary-huang force-pushed the gary/add-llm-obs-api branch from 98060a9 to 62bc9a0 Compare January 14, 2025 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mcculls mcculls mcculls left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gary-huang @mcculls