Metrics: Transactions Per Second

[mzabaluev]

Parent topic: #367

Description

Metric family: Transactions per second

Measurement points

Ingress vs. egress: @l-monninger I need conceptual clarity on this and how does this apply to any of the points below.

• transaction pipe ingress (transactions submitted by node clients)
• blob formation for the DA (transactions that survived load shedding and pre-validation)
• execution (transactions from DA)

Method

OpenTelemetry events with timestamps emitted at each of the observation points. The consumer is responsible for applying sliding window analysis or similar to compute transactions per second.

Position in popular monitoring strategies

USE: none applicable
RED: Rate
Four Golden Signals: Traffic

Open issues

Is OpenTelemetry settled upon?
Are there off-the-shelf solutions to analyze event streams?

[mzabaluev]

Should we repurpose the current tracing events for the "movement_timing" target, and send them to OpenTelepathy instead of a JSON log?

[l-monninger]

Yeah, so if there's an easy way to do it, I would like to be able to toggle between the two or turn them on and off non-mutually-exclusively. That way, we can still reuse the notebook without having to write an OTP agent and dump from it locally.

[mzabaluev]

Right, I think it should be possible to plug in alternative tracing subscribers for OpenTelemetry and our homespun JSON log.

One complication is that "movement_timing" partially instruments with spans to collect elapsed time data, while for metrics we'd generally like to emit singular events. I'll need to check if we can introduce another target for filtering telemetry events, but also include them in timing data sets by having them embedded in the timing spans.