You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Run large batches of new builds (no exact size to trigger the leak has been found, but it's unmissable when build queue size reaches 1000)
Wait for builds to complete
Inspect the jvm_threads metric scraped from the TeamCity server's Prometheus endpoint
Generate & inspect a thread dump of the server in the Administration / Diagnostics page
Expected result
Thread count stays steady over time.
Actual result
Thread count increases sharply while the builds are run, and thread dumps in TeamCity server show thousands of instances of the below stack trace:
"BatchSpanProcessor_WorkerThread-1" daemon group="main" prio=5 tid=935800 nid=935800 waiting
java.lang.Thread.State: TIMED_WAITING
at java.base@17.0.7/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@17.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
at java.base@17.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
at java.base@17.0.7/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:435)
at io.opentelemetry.sdk.trace.export.BatchSpanProcessor$Worker.run(BatchSpanProcessor.java:266)
at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
Examples of the pattern of increased jvm_threads counts is shown below.
Additional information
This was originally reported to JetBrains -- see YouTrack ticket TW-91607 -- who ruled out any usage of OTel SDKs in their code.
Disabling the plugin temporarily while running the reproduction steps also kept jvm_threads from increasing.
The text was updated successfully, but these errors were encountered:
Plugin version: 2.0.728
Steps to reproduce the problem:
jvm_threads
metric scraped from the TeamCity server's Prometheus endpointExpected result
Actual result
Examples of the pattern of increased
jvm_threads
counts is shown below.Additional information
This was originally reported to JetBrains -- see YouTrack ticket TW-91607 -- who ruled out any usage of OTel SDKs in their code.
Disabling the plugin temporarily while running the reproduction steps also kept
jvm_threads
from increasing.The text was updated successfully, but these errors were encountered: