Metrics: Transaction Latency

[mzabaluev]

Parent discussion: #367

Description

Latency of transactions at stages of their progression through the system.

Measurement points

Ingress:

• Transaction pipe ingress (submitted by node clients)
• Transactions from DA (sequenced by the network)

Egress:

• Post execution

Method

OpenTelemetry events with timestamps and transaction hashes emitted at each of the observation points for each transaction. The consumer is responsible for creating an aggregated latency metric.

Position in popular monitoring strategies

USE: none applicable
RED: Duration
Four Golden Signals: Latency

Open issues

Is OpenTelemetry settled upon?
Should the node emit events with just timestamps or compute duration statistics internally?

[l-monninger]

If you can link to lines in code, I think this is ready for an issue.