Skip to content

Commit

Permalink
[DOC] Update migrate static for tracing (#1091)
Browse files Browse the repository at this point in the history 
Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com>
  • Loading branch information
@knylander-grafana @clayton-cornell
knylander-grafana and clayton-cornell authored Jun 24, 2024
1 parent d225ad3 commit e09a285
Show file tree
Hide file tree
Showing 4 changed files with 109 additions and 46 deletions.
8 changes: 4 additions & 4 deletions docs/sources/reference/components/otelcol.connector.servicegraph.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ otelcol.receiver.otlp "default" {
}
output {
traces = [otelcol.connector.servicegraph.default.input,otelcol.exporter.otlp.grafana_cloud_tempo.input]
traces = [otelcol.connector.servicegraph.default.input,otelcol.exporter.otlp.grafana_cloud_traces.input]
}
}
Expand All @@ -201,14 +201,14 @@ prometheus.remote_write "mimir" {
}
}
otelcol.exporter.otlp "grafana_cloud_tempo" {
otelcol.exporter.otlp "grafana_cloud_traces" {
client {
endpoint = "https://tempo-xxx.grafana.net/tempo"
auth = otelcol.auth.basic.grafana_cloud_tempo.handler
auth = otelcol.auth.basic.grafana_cloud_traces.handler
}
}
otelcol.auth.basic "grafana_cloud_tempo" {
otelcol.auth.basic "grafana_cloud_traces" {
username = env("TEMPO_USERNAME")
password = env("GRAFANA_CLOUD_API_KEY")
}
Expand Down
6 changes: 3 additions & 3 deletions docs/sources/reference/components/otelcol.exporter.otlp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -205,13 +205,13 @@ otelcol.exporter.otlp "tempo" {
You can create an `otlp` exporter that sends your data to a managed service, for example, Grafana Cloud. The Tempo username and Grafana Cloud API Key are injected in this example through environment variables.

```alloy
otelcol.exporter.otlp "grafana_cloud_tempo" {
otelcol.exporter.otlp "grafana_cloud_traces" {
client {
endpoint = "tempo-xxx.grafana.net/tempo:443"
auth = otelcol.auth.basic.grafana_cloud_tempo.handler
auth = otelcol.auth.basic.grafana_cloud_traces.handler
}
}
otelcol.auth.basic "grafana_cloud_tempo" {
otelcol.auth.basic "grafana_cloud_traces" {
username = env("TEMPO_USERNAME")
password = env("GRAFANA_CLOUD_API_KEY")
}
Expand Down
65 changes: 64 additions & 1 deletion docs/sources/tasks/migrate/from-static.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ This topic describes how to:
* [loki.process][]
* [loki.source.file][]
* [loki.write][]
* [otelcol.receiver.otlp][]
* [otelcol.processor.batch][]
* [otelcol.exporter.otlp][]

## Before you begin

Expand Down Expand Up @@ -169,6 +172,23 @@ logs:
- level
clients:
- url: https://USER_ID:API_KEY@logs-prod3.grafana.net/loki/api/v1/push
traces:
configs:
- name: tempo
receivers:
otlp:
protocols:
grpc:
http:
batch:
send_batch_size: 10000
timeout: 20s
remote_write:
- endpoint: tempo-us-central1.grafana.net:443
basic_auth:
username: USERNAME
password: PASSWORD
```
The convert command takes the YAML file as input and outputs a [{{< param "PRODUCT_NAME" >}} configuration][configuration] file.
Expand Down Expand Up @@ -197,7 +217,7 @@ prometheus.scrape "metrics_test_local_agent" {
prometheus.remote_write "metrics_test" {
endpoint {
name = "test-3a2a1b"
name = "test-4dec64"
url = "https://prometheus-us-central1.grafana.net/api/prom/push"
basic_auth {
Expand Down Expand Up @@ -258,6 +278,46 @@ loki.write "logs_varlogs" {
}
external_labels = {}
}
otelcol.receiver.otlp "default" {
grpc {
include_metadata = true
}
http {
include_metadata = true
}
output {
metrics = []
logs = []
traces = [otelcol.processor.batch.default.input]
}
}
otelcol.processor.batch "default" {
timeout = "20s"
send_batch_size = 10000
output {
metrics = []
logs = []
traces = [otelcol.exporter.otlp.default_0.input]
}
}
otelcol.exporter.otlp "default_0" {
retry_on_failure {
max_elapsed_time = "1m0s"
}
client {
endpoint = "tempo-us-central1.grafana.net:443"
headers = {
authorization = "Basic VVNFUk5BTUU6UEFTU1dPUkQ=",
}
}
}
```
## Integrations Next
Expand Down Expand Up @@ -322,3 +382,6 @@ The following list is specific to the convert command and not {{< param "PRODUCT
[Metrics]: https://grafana.com/docs/agent/latest/static/configuration/metrics-config/
[Logs]: https://grafana.com/docs/agent/latest/static/configuration/logs-config/
[UI]: ../../debug/#alloy-ui
[otelcol.receiver.otlp]: ../../../reference/components/otelcol.receiver.otlp/
[otelcol.processor.batch]: ../../../reference/components/otelcol.processor.batch/
[otelcol.exporter.otlp]:../../../reference/components/otelcol.exporter.otlp/
76 changes: 38 additions & 38 deletions docs/sources/tasks/opentelemetry-to-lgtm-stack.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ Traces: OTel → batch processor → OTel exporter
### Grafana Loki

[Grafana Loki][] is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus.
Similar to Prometheus, to send from OTLP to Loki, you can do a passthrough from the [otelcol.exporter.loki][] component to [loki.write][] component.
Similar to Prometheus, to send from OTLP to Loki, you can configure passthrough from the [otelcol.exporter.loki][] component to [loki.write][] component.

```alloy
otelcol.exporter.loki "default" {
Expand All @@ -133,15 +133,15 @@ loki.write "default" {
}
```

To use Loki with basic-auth, which is required with Grafana Cloud Loki, you must configure the [loki.write][] component.
To use Loki with basic-auth, which is required with Grafana Cloud Logs, you must configure the [loki.write][] component.
You can get the Loki configuration from the Loki **Details** page in the [Grafana Cloud Portal][].

```alloy
otelcol.exporter.loki "grafana_cloud_loki" {
forward_to = [loki.write.grafana_cloud_loki.receiver]
otelcol.exporter.loki "grafana_cloud_logs" {
forward_to = [loki.write.grafana_cloud_logs.receiver]
}
loki.write "grafana_cloud_loki" {
loki.write "grafana_cloud_logs" {
endpoint {
url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"
Expand All @@ -166,18 +166,18 @@ otelcol.exporter.otlp "default" {
}
```

To use Tempo with basic-auth, which is required with Grafana Cloud Tempo, you must use the [otelcol.auth.basic][] component.
To use Tempo with basic-auth, which is required with Grafana Cloud Traces, you must use the [otelcol.auth.basic][] component.
You can get the Tempo configuration from the Tempo **Details** page in the [Grafana Cloud Portal][].

```alloy
otelcol.exporter.otlp "grafana_cloud_tempo" {
otelcol.exporter.otlp "grafana_cloud_traces" {
client {
endpoint = "tempo-us-central1.grafana.net:443"
auth = otelcol.auth.basic.grafana_cloud_tempo.handler
auth = otelcol.auth.basic.grafana_cloud_traces.handler
}
}
otelcol.auth.basic "grafana_cloud_tempo" {
otelcol.auth.basic "grafana_cloud_traces" {
username = 4094
password = env("GRAFANA_CLOUD_API_KEY")
}
Expand All @@ -186,7 +186,7 @@ otelcol.auth.basic "grafana_cloud_tempo" {
### Grafana Mimir or Prometheus Remote Write

[Prometheus Remote Write][] is a popular metrics transmission protocol supported by most metrics systems, including [Grafana Mimir][] and Grafana Cloud.
To send from OTLP to Prometheus, you can do a passthrough from the [otelcol.exporter.prometheus][] to the [prometheus.remote_write][] component.
To send from OTLP to a Prometheus compatible `remote_write` endpoint, you can configure passthrough from the [otelcol.exporter.prometheus][] to the [prometheus.remote_write][] component.
The Prometheus remote write component in {{< param "PRODUCT_NAME" >}} is a robust protocol implementation, including a Write Ahead Log (WAL) for resiliency.

```alloy
Expand All @@ -201,15 +201,15 @@ prometheus.remote_write "default" {
}
```

To use Prometheus with basic-auth, which is required with Grafana Cloud Prometheus, you must configure the [prometheus.remote_write][] component.
To use Prometheus with basic-auth, which is required with Grafana Cloud Metrics, you must configure the [prometheus.remote_write][] component.
You can get the Prometheus configuration from the Prometheus **Details** page in the [Grafana Cloud Portal][].

```alloy
otelcol.exporter.prometheus "grafana_cloud_prometheus" {
forward_to = [prometheus.remote_write.grafana_cloud_prometheus.receiver]
otelcol.exporter.prometheus "grafana_cloud_metrics" {
forward_to = [prometheus.remote_write.grafana_cloud_metrics.receiver]
}
prometheus.remote_write "grafana_cloud_prometheus" {
prometheus.remote_write "grafana_cloud_metrics" {
endpoint {
url = "https://prometheus-us-central1.grafana.net/api/prom/push"
Expand Down Expand Up @@ -245,29 +245,29 @@ otelcol.receiver.otlp "example" {
otelcol.processor.batch "example" {
output {
metrics = [otelcol.exporter.prometheus.grafana_cloud_prometheus.input]
logs = [otelcol.exporter.loki.grafana_cloud_loki.input]
traces = [otelcol.exporter.otlp.grafana_cloud_tempo.input]
metrics = [otelcol.exporter.prometheus.grafana_cloud_metrics.input]
logs = [otelcol.exporter.loki.grafana_cloud_logs.input]
traces = [otelcol.exporter.otlp.grafana_cloud_traces.input]
}
}
otelcol.exporter.otlp "grafana_cloud_tempo" {
otelcol.exporter.otlp "grafana_cloud_traces" {
client {
endpoint = "tempo-us-central1.grafana.net:443"
auth = otelcol.auth.basic.grafana_cloud_tempo.handler
auth = otelcol.auth.basic.grafana_cloud_traces.handler
}
}
otelcol.auth.basic "grafana_cloud_tempo" {
otelcol.auth.basic "grafana_cloud_traces" {
username = 4094
password = env("GRAFANA_CLOUD_API_KEY")
}
otelcol.exporter.prometheus "grafana_cloud_prometheus" {
forward_to = [prometheus.remote_write.grafana_cloud_prometheus.receiver]
otelcol.exporter.prometheus "grafana_cloud_metrics" {
forward_to = [prometheus.remote_write.grafana_cloud_metrics.receiver]
}
prometheus.remote_write "grafana_cloud_prometheus" {
prometheus.remote_write "grafana_cloud_metrics" {
endpoint {
url = "https://prometheus-us-central1.grafana.net/api/prom/push"
Expand All @@ -278,11 +278,11 @@ prometheus.remote_write "grafana_cloud_prometheus" {
}
}
otelcol.exporter.loki "grafana_cloud_loki" {
forward_to = [loki.write.grafana_cloud_loki.receiver]
otelcol.exporter.loki "grafana_cloud_logs" {
forward_to = [loki.write.grafana_cloud_logs.receiver]
}
loki.write "grafana_cloud_loki" {
loki.write "grafana_cloud_logs" {
endpoint {
url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"
Expand All @@ -302,18 +302,18 @@ Running {{< param "PRODUCT_NAME" >}} now will give you the following:
ts=2023-05-09T09:37:15.300959Z level=info msg="running usage stats reporter"
ts=2023-05-09T09:37:15.300958Z level=info msg="now listening for http traffic" addr=127.0.0.1:12345
ts=2023-05-09T09:37:15.301104Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="starting complete graph evaluation"
ts=2023-05-09T09:37:15.301307Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=loki.write.grafana_cloud_loki duration=188.209µs
ts=2023-05-09T09:37:15.301334Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.loki.grafana_cloud_loki duration=18.791µs
ts=2023-05-09T09:37:15.303138Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="replaying WAL, this may take a while" dir=data-alloy/prometheus.remote_write.grafana_cloud_prometheus/wal
ts=2023-05-09T09:37:15.303257Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="WAL segment loaded" segment=0 maxSegment=1
ts=2023-05-09T09:37:15.303302Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="WAL segment loaded" segment=1 maxSegment=1
ts=2023-05-09T09:37:15.303507Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting WAL watcher" queue=7f623a
ts=2023-05-09T09:37:15.303515Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting scraped metadata watcher"
ts=2023-05-09T09:37:15.303522Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=prometheus.remote_write.grafana_cloud_prometheus duration=2.181958ms
ts=2023-05-09T09:37:15.303557Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.prometheus.grafana_cloud_prometheus duration=30.083µs
ts=2023-05-09T09:37:15.303611Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Replaying WAL" queue=7f623a
ts=2023-05-09T09:37:15.303618Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.auth.basic.grafana_cloud_tempo duration=52.5µs
ts=2023-05-09T09:37:15.303694Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.otlp.grafana_cloud_tempo duration=70.375µs
ts=2023-05-09T09:37:15.301307Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=loki.write.grafana_cloud_logs duration=188.209µs
ts=2023-05-09T09:37:15.301334Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.loki.grafana_cloud_logs duration=18.791µs
ts=2023-05-09T09:37:15.303138Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="replaying WAL, this may take a while" dir=data-alloy/prometheus.remote_write.grafana_cloud_metrics/wal
ts=2023-05-09T09:37:15.303257Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="WAL segment loaded" segment=0 maxSegment=1
ts=2023-05-09T09:37:15.303302Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="WAL segment loaded" segment=1 maxSegment=1
ts=2023-05-09T09:37:15.303507Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting WAL watcher" queue=7f623a
ts=2023-05-09T09:37:15.303515Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting scraped metadata watcher"
ts=2023-05-09T09:37:15.303522Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=prometheus.remote_write.grafana_cloud_metrics duration=2.181958ms
ts=2023-05-09T09:37:15.303557Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.prometheus.grafana_cloud_metrics duration=30.083µs
ts=2023-05-09T09:37:15.303611Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Replaying WAL" queue=7f623a
ts=2023-05-09T09:37:15.303618Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.auth.basic.grafana_cloud_traces duration=52.5µs
ts=2023-05-09T09:37:15.303694Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.otlp.grafana_cloud_traces duration=70.375µs
ts=2023-05-09T09:37:15.303782Z component=otelcol.processor.memory_limiter.default level=info msg="Memory limiter configured" limit_mib=150 spike_limit_mib=30 check_interval=1s
ts=2023-05-09T09:37:15.303802Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.memory_limiter.default duration=100.334µs
ts=2023-05-09T09:37:15.303853Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.batch.default duration=44.75µs
Expand Down

0 comments on commit e09a285

Please sign in to comment.