Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for easier network metrics enablement #1013

Merged
merged 4 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/sources/configure/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,7 +835,7 @@ The purpose of this value is to avoid reporting indefinitely finished applicatio

| YAML | Environment variable | Type | Default |
|------------|-------------------------------|-----------------|------------------------------|
| `features` | `BEYLA_OTEL_METRICS_FEATURES` | list of strings | `["application", "network"]` |
| `features` | `BEYLA_OTEL_METRICS_FEATURES` | list of strings | `["application"]` |

A list of metric groups which are allowed to be exported. Each group belongs to a different feature
of Beyla: application-level metrics or network metrics.
Expand All @@ -853,8 +853,8 @@ of Beyla: application-level metrics or network metrics.
the OpenTelemetry service names used in Beyla. In Kubernetes environments, the OpenTelemetry service name set by the service name
discovery is the best choice for service graph metrics.
- If the list contains `network`, the Beyla OpenTelemetry exporter exports network-level
metrics; but only if there is defined an OpenTelemetry endpoint and the
[network metrics are enabled]({{< relref "../network" >}}).
metrics; but only if there is an OpenTelemetry endpoint defined. For network-level metrics options visit the
[network metrics]({{< relref "../network" >}}) configuration documentation.

Usually you do not need to change this configuration option, unless, for example, a Beyla instance
instruments both network and applications, and you want to disable application-level metrics because
Expand Down Expand Up @@ -1214,7 +1214,7 @@ The `buckets` object allows overriding the bucket boundaries of diverse histogra

| YAML | Environment variable | Type | Default |
|------------|-----------------------------|-----------------|------------------------------|
| `features` | `BEYLA_PROMETHEUS_FEATURES` | list of strings | `["application", "network"]` |
| `features` | `BEYLA_PROMETHEUS_FEATURES` | list of strings | `["application"]` |

A list of metric groups that are allowed to be exported. Each group belongs to a different feature
of Beyla: application-level metrics or network metrics.
Expand All @@ -1232,8 +1232,8 @@ of Beyla: application-level metrics or network metrics.
the OpenTelemetry service names used in Beyla. In Kubernetes environments, the OpenTelemetry service name set by the service name
discovery is the best choice for service graph metrics.
- If the list contains `network`, the Beyla Prometheus exporter exports network-level
metrics; but only if the Prometheus `port` property is defined and the
[network metrics are enabled]({{< relref "../network" >}}).
metrics; but only if the Prometheus `port` property is defined. For network-level metrics options visit the
[network metrics]({{< relref "../network" >}}) configuration documentation.

Usually you do not need to change this configuration option, unless, for example, a Beyla instance
instruments both network and applications, and you want to disable application-level metrics because
Expand Down
4 changes: 3 additions & 1 deletion docs/sources/network/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ network metrics (in the previous example, `otel_metrics_export`, but it also acc
| -------- | ----------------------- | ------- | ------- |
| `enable` | `BEYLA_NETWORK_METRICS` | boolean | `false` |

Enables network metrics reporting in Beyla.
Explicitly enables network metrics reporting in Beyla. You can also enable network metrics reporting
by adding `network` to the list of `features` for [otel_metrics_export]({{< relref "../configure/options.md#otel-metrics-exporter" >}}))
or [prometheus_export]({{< relref "../configure/options.md#prometheus-http-endpoint" >}})).
Comment on lines 52 to +57
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to keep this only for backwards-compatibility purposes, I'd even delete it from the documentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I should follow-up with a PR to remove the docs and also remove it from examples and the chart. Perhaps we do this work for 2.0?


| YAML | Environment variable | Type | Default |
| -------------------- | ---------------------------------- | -------- | ------------------- |
Expand Down
14 changes: 11 additions & 3 deletions pkg/beyla/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ var DefaultConfig = Config{
Buckets: otel.DefaultBuckets,
ReportersCacheLen: ReporterLRUSize,
HistogramAggregation: otel.AggregationExplicit,
Features: []string{otel.FeatureNetwork, otel.FeatureApplication},
Features: []string{otel.FeatureApplication},
Instrumentations: []string{
instrumentations.InstrumentationALL,
},
Expand All @@ -85,7 +85,7 @@ var DefaultConfig = Config{
Prometheus: prom.PrometheusConfig{
Path: "/metrics",
Buckets: otel.DefaultBuckets,
Features: []string{otel.FeatureNetwork, otel.FeatureApplication},
Features: []string{otel.FeatureApplication},
Instrumentations: []string{
instrumentations.InstrumentationALL,
},
Expand Down Expand Up @@ -233,11 +233,19 @@ func (c *Config) Validate() error {
return nil
}

func (c *Config) promNetO11yEnabled() bool {
return c.Prometheus.Enabled() && c.Prometheus.NetworkMetricsEnabled()
}

func (c *Config) otelNetO11yEnabled() bool {
return (c.Metrics.Enabled() || c.Grafana.OTLP.MetricsEnabled()) && c.Metrics.NetworkMetricsEnabled()
}

// Enabled checks if a given Beyla feature is enabled according to the global configuration
func (c *Config) Enabled(feature Feature) bool {
switch feature {
case FeatureNetO11y:
return c.NetworkFlows.Enable
return c.NetworkFlows.Enable || c.promNetO11yEnabled() || c.otelNetO11yEnabled()
case FeatureAppO11y:
return c.Port.Len() > 0 || c.Exec.IsSet() || len(c.Discovery.Services) > 0 || c.Discovery.SystemWide
}
Expand Down
24 changes: 22 additions & 2 deletions pkg/beyla/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ network:
DurationHistogram: []float64{0, 1, 2},
RequestSizeHistogram: otel.DefaultBuckets.RequestSizeHistogram,
},
Features: []string{"network", "application"},
Features: []string{"application"},
Instrumentations: []string{
instrumentations.InstrumentationALL,
},
Expand All @@ -140,7 +140,7 @@ network:
},
Prometheus: prom.PrometheusConfig{
Path: "/metrics",
Features: []string{otel.FeatureNetwork, otel.FeatureApplication},
Features: []string{otel.FeatureApplication},
Instrumentations: []string{
instrumentations.InstrumentationALL,
},
Expand Down Expand Up @@ -288,6 +288,26 @@ func TestConfig_OtelGoAutoEnv(t *testing.T) {
assert.True(t, cfg.Exec.IsSet()) // Exec maps to BEYLA_EXECUTABLE_NAME
}

func TestConfig_NetworkImplicit(t *testing.T) {
// OTEL_GO_AUTO_TARGET_EXE is an alias to BEYLA_EXECUTABLE_NAME
// (Compatibility with OpenTelemetry)
require.NoError(t, os.Setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4318"))
require.NoError(t, os.Setenv("BEYLA_OTEL_METRIC_FEATURES", "network"))
cfg, err := LoadConfig(bytes.NewReader(nil))
require.NoError(t, err)
assert.True(t, cfg.Enabled(FeatureNetO11y)) // Net o11y should be on
}

func TestConfig_NetworkImplicitProm(t *testing.T) {
// OTEL_GO_AUTO_TARGET_EXE is an alias to BEYLA_EXECUTABLE_NAME
// (Compatibility with OpenTelemetry)
require.NoError(t, os.Setenv("BEYLA_PROMETHEUS_PORT", "9090"))
require.NoError(t, os.Setenv("BEYLA_PROMETHEUS_FEATURES", "network"))
cfg, err := LoadConfig(bytes.NewReader(nil))
require.NoError(t, err)
assert.True(t, cfg.Enabled(FeatureNetO11y)) // Net o11y should be on
}

func loadConfig(t *testing.T, env map[string]string) *Config {
for k, v := range env {
require.NoError(t, os.Setenv(k, v))
Expand Down
6 changes: 5 additions & 1 deletion pkg/internal/export/otel/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,12 @@ func (m *MetricsConfig) OTelMetricsEnabled() bool {
return slices.Contains(m.Features, FeatureApplication)
}

func (m *MetricsConfig) NetworkMetricsEnabled() bool {
return slices.Contains(m.Features, FeatureNetwork)
}

func (m *MetricsConfig) Enabled() bool {
return m.EndpointEnabled() && (m.OTelMetricsEnabled() || m.SpanMetricsEnabled() || m.ServiceGraphMetricsEnabled())
return m.EndpointEnabled() && (m.OTelMetricsEnabled() || m.SpanMetricsEnabled() || m.ServiceGraphMetricsEnabled() || m.NetworkMetricsEnabled())
}

// MetricsReporter implements the graph node that receives request.Span
Expand Down
4 changes: 2 additions & 2 deletions pkg/internal/export/otel/metrics_net.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"context"
"fmt"
"log/slog"
"slices"
"time"

"github.com/google/uuid"
Expand All @@ -25,10 +24,11 @@ import (
type NetMetricsConfig struct {
Metrics *MetricsConfig
AttributeSelectors attributes.Selection
GloballyEnabled bool
}

func (mc NetMetricsConfig) Enabled() bool {
return mc.Metrics != nil && mc.Metrics.EndpointEnabled() && slices.Contains(mc.Metrics.Features, FeatureNetwork)
return mc.Metrics != nil && mc.Metrics.EndpointEnabled() && (mc.Metrics.NetworkMetricsEnabled() || mc.GloballyEnabled)
}

func nmlog() *slog.Logger {
Expand Down
2 changes: 1 addition & 1 deletion pkg/internal/export/otel/metrics_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -528,6 +528,7 @@ func TestMetricsConfig_Enabled(t *testing.T) {
assert.True(t, (&MetricsConfig{Features: []string{FeatureApplication, FeatureNetwork}, CommonEndpoint: "foo"}).Enabled())
assert.True(t, (&MetricsConfig{Features: []string{FeatureApplication}, MetricsEndpoint: "foo"}).Enabled())
assert.True(t, (&MetricsConfig{Features: []string{FeatureNetwork, FeatureApplication}, Grafana: &GrafanaOTLP{Submit: []string{"traces", "metrics"}, InstanceID: "33221"}}).Enabled())
assert.True(t, (&MetricsConfig{MetricsEndpoint: "foo", Features: []string{FeatureNetwork}}).Enabled())
}

func TestMetricsConfig_Disabled(t *testing.T) {
Expand All @@ -536,7 +537,6 @@ func TestMetricsConfig_Disabled(t *testing.T) {
assert.False(t, (&MetricsConfig{Features: []string{FeatureApplication}, Grafana: &GrafanaOTLP{Submit: []string{"metrics"}}}).Enabled())
// application feature is not enabled
assert.False(t, (&MetricsConfig{CommonEndpoint: "foo"}).Enabled())
assert.False(t, (&MetricsConfig{MetricsEndpoint: "foo", Features: []string{FeatureNetwork}}).Enabled())
assert.False(t, (&MetricsConfig{Grafana: &GrafanaOTLP{Submit: []string{"traces", "metrics"}, InstanceID: "33221"}}).Enabled())
}

Expand Down
6 changes: 5 additions & 1 deletion pkg/internal/export/prom/prom.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,13 +128,17 @@ func (p *PrometheusConfig) ServiceGraphMetricsEnabled() bool {
return slices.Contains(p.Features, otel.FeatureGraph)
}

func (p *PrometheusConfig) NetworkMetricsEnabled() bool {
return slices.Contains(p.Features, otel.FeatureNetwork)
}

func (p *PrometheusConfig) EndpointEnabled() bool {
return p.Port != 0 || p.Registry != nil
}

// nolint:gocritic
func (p *PrometheusConfig) Enabled() bool {
return p.EndpointEnabled() && (p.OTelMetricsEnabled() || p.SpanMetricsEnabled() || p.ServiceGraphMetricsEnabled())
return p.EndpointEnabled() && (p.OTelMetricsEnabled() || p.SpanMetricsEnabled() || p.ServiceGraphMetricsEnabled() || p.NetworkMetricsEnabled())
}

type metricsReporter struct {
Expand Down
5 changes: 2 additions & 3 deletions pkg/internal/export/prom/prom_net.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,13 @@ package prom
import (
"context"
"fmt"
"slices"

"github.com/mariomac/pipes/pipe"
"github.com/prometheus/client_golang/prometheus"

"github.com/grafana/beyla/pkg/internal/connector"
"github.com/grafana/beyla/pkg/internal/export/attributes"
"github.com/grafana/beyla/pkg/internal/export/expire"
"github.com/grafana/beyla/pkg/internal/export/otel"
"github.com/grafana/beyla/pkg/internal/netolly/ebpf"
"github.com/grafana/beyla/pkg/internal/pipe/global"
)
Expand All @@ -22,11 +20,12 @@ import (
type NetPrometheusConfig struct {
Config *PrometheusConfig
AttributeSelectors attributes.Selection
GloballyEnabled bool
}

// nolint:gocritic
func (p NetPrometheusConfig) Enabled() bool {
return p.Config != nil && p.Config.Port != 0 && slices.Contains(p.Config.Features, otel.FeatureNetwork)
return p.Config != nil && p.Config.Port != 0 && (p.Config.NetworkMetricsEnabled() || p.GloballyEnabled)
}

type netMetricsReporter struct {
Expand Down
2 changes: 2 additions & 0 deletions pkg/internal/netolly/agent/pipeline.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,12 +136,14 @@ func (f *Flows) pipelineBuilder(ctx context.Context) (*pipe.Builder[*FlowsPipeli
return otel.NetMetricsExporterProvider(ctx, f.ctxInfo, &otel.NetMetricsConfig{
Metrics: &f.cfg.Metrics,
AttributeSelectors: f.cfg.Attributes.Select,
GloballyEnabled: f.cfg.NetworkFlows.Enable,
})
})
pipe.AddFinalProvider(pb, promExport, func() (pipe.FinalFunc[[]*ebpf.Record], error) {
return prom.NetPrometheusEndpoint(ctx, f.ctxInfo, &prom.NetPrometheusConfig{
Config: &f.cfg.Prometheus,
AttributeSelectors: f.cfg.Attributes.Select,
GloballyEnabled: f.cfg.NetworkFlows.Enable,
})
})
pipe.AddFinalProvider(pb, printer, func() (pipe.FinalFunc[[]*ebpf.Record], error) {
Expand Down
2 changes: 1 addition & 1 deletion test/integration/docker-compose-netolly-direction.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ services:
BEYLA_CONFIG_PATH: /configs/instrumenter-config-netolly${BEYLA_CONFIG_SUFFIX}.yml
GOCOVERDIR: "/coverage"
BEYLA_NETWORK_SOURCE: ${BEYLA_NETWORK_SOURCE}
BEYLA_NETWORK_METRICS: "true"
BEYLA_OTEL_METRICS_FEATURES: "network" # implicitly enabling network metrics without a global enable
BEYLA_NETWORK_PRINT_FLOWS: "true"
BEYLA_NETWORK_DEDUPER: ${BEYLA_NETWORK_DEDUPER}
OTEL_EXPORTER_OTLP_ENDPOINT: http://otelcol:4318
Expand Down
2 changes: 1 addition & 1 deletion test/integration/docker-compose-netolly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ services:
BEYLA_CONFIG_PATH: /configs/instrumenter-config-netolly${BEYLA_CONFIG_SUFFIX}.yml
GOCOVERDIR: "/coverage"
BEYLA_NETWORK_SOURCE: ${BEYLA_NETWORK_SOURCE}
BEYLA_NETWORK_METRICS: "true"
BEYLA_NETWORK_METRICS: "true" # explicitly enabling network metrics
BEYLA_NETWORK_PRINT_FLOWS: "true"
BEYLA_NETWORK_DEDUPER: ${BEYLA_NETWORK_DEDUPER}
OTEL_EXPORTER_OTLP_ENDPOINT: http://otelcol:4318
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ spec:
value: "/testoutput"
- name: BEYLA_CONFIG_PATH
value: /config/beyla-config.yml
- name: BEYLA_NETWORK_METRICS
value: "true"
- name: BEYLA_PROMETHEUS_FEATURES # implicitly enabling network metrics for Prometheus scrape
value: "network"
- name: BEYLA_NETWORK_CACHE_ACTIVE_TIMEOUT
value: "100ms"
- name: BEYLA_NETWORK_CACHE_MAX_FLOWS
Expand Down
Loading