Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

App metrics attribute selectors #789

Merged
merged 28 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8f6737e
Porting attribute filters to OTEL app o11y
mariomac Apr 24, 2024
0c74fc0
start to set common default supported attributes definition
mariomac Apr 24, 2024
c84b9ad
wildcard inclusion/exclusion lists
mariomac Apr 25, 2024
270c581
fixed some compilarion errors
mariomac Apr 25, 2024
cf3813c
fix some imports mess
mariomac Apr 25, 2024
a3ead3e
attributes selector for prometheus
mariomac Apr 25, 2024
05ffa97
almost done. Failing pipeline tests
mariomac Apr 25, 2024
65b6bd6
Merge branch 'main' of github.com:grafana/ebpf-autoinstrument into at…
mariomac Apr 26, 2024
555804d
fixed attribute groups for route metrics
mariomac Apr 26, 2024
f459368
unit tests passing
mariomac Apr 26, 2024
3a61abb
simplified normalizetodot notation
mariomac Apr 26, 2024
4d97324
restored deprecated configuration options
mariomac Apr 26, 2024
80e9c2a
reorganized attributes package and renamed to metric
mariomac Apr 26, 2024
1c2f90d
Merge branch 'main' of github.com:grafana/ebpf-autoinstrument into at…
mariomac Apr 29, 2024
9929d1e
remove super-linter as it does not really lint any helm stuff
mariomac Apr 29, 2024
d3e6452
fixing deduper attributes
mariomac Apr 29, 2024
eb278b5
Fix tests
mariomac Apr 29, 2024
a423bf6
create attr.Name and move attribute definitions to 1.23
mariomac Apr 30, 2024
0a0d4c3
Reworked attribute naming and normalization. Expirer broken
mariomac Apr 30, 2024
e85d014
fixed unit tests and few integration tests
mariomac Apr 30, 2024
8bb3b85
renamed some attributes and fixed integration tests
mariomac Apr 30, 2024
e962cd3
code review until metric package
mariomac Apr 30, 2024
a61546e
Code commented and small reorganizations
mariomac Apr 30, 2024
2e1b0e0
Fixed service namespace in prom exporters
mariomac Apr 30, 2024
c154a98
Merge branch 'main' of github.com:grafana/ebpf-autoinstrument into at…
mariomac Apr 30, 2024
1ba8864
Merge branch 'main' of github.com:grafana/ebpf-autoinstrument into at…
mariomac May 3, 2024
36fdb08
moved network record getters
mariomac May 3, 2024
aed8314
simplified and commented getters
mariomac May 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/sources/network/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,14 +220,15 @@ Beyla only includes a subset of the available attributes to avoid leading to
a [cardinality explosion](/blog/2022/02/15/what-are-cardinality-spikes-and-why-do-they-matter/) in
the metrics storage, especially if some attributes like `src.address` or `dst.address` capture the IP addresses of the external traffic.

The `attributes.allow` YAML subsection makes it possible to select the attributes to report:
The `attributes.select.<metric-name>.include` YAML subsection makes it possible to select the attributes to report:

```yaml
network:
enable: true
attributes:
allow:
beyla.network.flow.bytes:
include:
- k8s.src.owner.name
- k8s.src.namespace
- k8s.dst.owner.name
Expand Down
8 changes: 4 additions & 4 deletions pkg/beyla/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ import (
"gopkg.in/yaml.v3"

ebpfcommon "github.com/grafana/beyla/pkg/internal/ebpf/common"
"github.com/grafana/beyla/pkg/internal/export/attr"
"github.com/grafana/beyla/pkg/internal/export/debug"
"github.com/grafana/beyla/pkg/internal/export/metric"
"github.com/grafana/beyla/pkg/internal/export/otel"
"github.com/grafana/beyla/pkg/internal/export/prom"
"github.com/grafana/beyla/pkg/internal/imetrics"
Expand Down Expand Up @@ -164,9 +164,9 @@ func (t TracesReceiverConfig) Enabled() bool {
// Attributes configures the decoration of some extra attributes that will be
// added to each span
type Attributes struct {
Kubernetes transform.KubernetesDecorator `yaml:"kubernetes"`
InstanceID traces.InstanceIDConfig `yaml:"instance_id"`
Allow attr.AllowedAttributesDefinition `yaml:"allow"`
Kubernetes transform.KubernetesDecorator `yaml:"kubernetes"`
InstanceID traces.InstanceIDConfig `yaml:"instance_id"`
Select metric.Selection `yaml:"select"`
}

type ConfigError string
Expand Down
15 changes: 10 additions & 5 deletions pkg/beyla/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ import (
"github.com/stretchr/testify/require"

ebpfcommon "github.com/grafana/beyla/pkg/internal/ebpf/common"
"github.com/grafana/beyla/pkg/internal/export/metric"
"github.com/grafana/beyla/pkg/internal/export/otel"
"github.com/grafana/beyla/pkg/internal/export/prom"
"github.com/grafana/beyla/pkg/internal/imetrics"
"github.com/grafana/beyla/pkg/internal/metricname"
"github.com/grafana/beyla/pkg/internal/netolly/transform/cidr"
"github.com/grafana/beyla/pkg/internal/traces"
"github.com/grafana/beyla/pkg/transform"
Expand Down Expand Up @@ -43,8 +43,10 @@ attributes:
informers_sync_timeout: 30s
instance_id:
dns: true
allow:
global: ["foo", "bar"]
select:
beyla.network.flow:
include: ["foo", "bar"]
exclude: ["baz", "bae"]
network:
enable: true
cidrs:
Expand Down Expand Up @@ -150,8 +152,11 @@ network:
Enable: transform.EnabledTrue,
InformersSyncTimeout: 30 * time.Second,
},
Allow: map[metricname.Normal][]string{
"global": {"foo", "bar"},
Select: metric.Selection{
metric.BeylaNetworkFlow.Section: metric.InclusionLists{
Include: []string{"foo", "bar"},
Exclude: []string{"baz", "bae"},
},
},
},
Routes: &transform.RoutesConfig{},
Expand Down
28 changes: 28 additions & 0 deletions pkg/components/beyla.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ import (
"github.com/grafana/beyla/pkg/beyla"
"github.com/grafana/beyla/pkg/internal/appolly"
"github.com/grafana/beyla/pkg/internal/connector"
"github.com/grafana/beyla/pkg/internal/export/metric"
"github.com/grafana/beyla/pkg/internal/imetrics"
"github.com/grafana/beyla/pkg/internal/netolly/agent"
"github.com/grafana/beyla/pkg/internal/netolly/flow"
"github.com/grafana/beyla/pkg/internal/pipe/global"
)

Expand Down Expand Up @@ -95,5 +97,31 @@ func buildCommonContextInfo(
slog.Debug("not reporting internal metrics")
ctxInfo.Metrics = imetrics.NoopReporter{}
}

attributeGroups(config, ctxInfo)

return ctxInfo
}

// attributeGroups specifies, based in the provided configuration, which groups of attributes
// need to be enabled by default for the diverse metrics
func attributeGroups(config *beyla.Config, ctxInfo *global.ContextInfo) {
if ctxInfo.K8sEnabled {
ctxInfo.MetricAttributeGroups.Add(metric.GroupKubernetes)
}
if config.Routes != nil {
ctxInfo.MetricAttributeGroups.Add(metric.GroupHTTPRoutes)
}
if config.Metrics.ReportPeerInfo || config.Prometheus.ReportPeerInfo {
ctxInfo.MetricAttributeGroups.Add(metric.GroupPeerInfo)
}
if config.Metrics.ReportTarget || config.Prometheus.ReportTarget {
ctxInfo.MetricAttributeGroups.Add(metric.GroupTarget)
}
if config.NetworkFlows.Deduper == flow.DeduperNone {
ctxInfo.MetricAttributeGroups.Add(metric.GroupNetIfaceDirection)
}
if config.NetworkFlows.CIDRs.Enabled() {
ctxInfo.MetricAttributeGroups.Add(metric.GroupNetCIDR)
}
}
83 changes: 0 additions & 83 deletions pkg/internal/export/attr/allowed.go

This file was deleted.

86 changes: 0 additions & 86 deletions pkg/internal/export/attr/allowed_test.go

This file was deleted.

80 changes: 0 additions & 80 deletions pkg/internal/export/attr/attr.go

This file was deleted.

Loading
Loading