diff --git a/docs/examples/complete-docs-example.md b/docs/examples/complete-docs-example.md index 0160fb4..830e5d2 100644 --- a/docs/examples/complete-docs-example.md +++ b/docs/examples/complete-docs-example.md @@ -13,7 +13,7 @@ killercoda: imageid: ubuntu --- - + # Sending OpenTelemetry logs to Loki using Alloy @@ -29,11 +29,11 @@ Before you begin, ensure you have the following to run the demo: - Docker - Docker Compose - + {{< admonition type="note" >}} Alternatively, you can try out this example in our online sandbox. Which is a fully configured environment with all the dependencies pre-installed. You can access the sandbox [here](https://killercoda.com/grafana-labs/course/loki/alloy-otel-logs). {{< /admonition >}} - + ## Scenario @@ -49,34 +49,34 @@ In this scenario, we have a microservices application called the Carnivourse Gre Each service generates logs using the OpenTelemetry SDK and exports to Alloy in the OpenTelemetry format. Alloy then ingests the logs and sends them to Loki. We will configure Alloy to ingest OpenTelemetry logs, send them to Loki, and view the logs in Grafana. - + - + ## Step 1: Environment setup In this step, we will set up our environment by cloning the repository that contains our demo application and spinning up our observability stack using Docker Compose. 1. To get started, clone the repository that contains our demo application: - + ```bash git clone -b microservice-otel https://github.com/grafana/loki-fundamentals.git ``` - + 1. Next we will spin up our observability stack using Docker Compose: - + ```bash docker compose -f loki-fundamentals/docker-compose.yml up -d ``` - + - + - + This will spin up the following services: ```bash @@ -88,24 +88,24 @@ In this step, we will set up our environment by cloning the repository that cont We will be access two UI interfaces: - Alloy at [http://localhost:12345](http://localhost:12345) - Grafana at [http://localhost:3000](http://localhost:3000) - + - + ## Step 2: Configure Alloy to ingest OpenTelemetry logs To configure Alloy to ingest OpenTelemetry logs, we need to update the Alloy configuration file. To start, we will update the `config.alloy` file to include the OpenTelemetry logs configuration. - + - + ### OpenTelelmetry Logs Receiver First, we will configure the OpenTelemetry logs receiver. This receiver will accept logs via HTTP and gRPC. Open the `config.alloy` file in the `loki-fundamentals` directory and copy the following configuration: - + ```alloy otelcol.receiver.otlp "default" { @@ -118,7 +118,7 @@ Open the `config.alloy` file in the `loki-fundamentals` directory and copy the f } ``` - + ### OpenTelemetry Logs Processor @@ -126,7 +126,7 @@ Open the `config.alloy` file in the `loki-fundamentals` directory and copy the f Next, we will configure the OpenTelemetry logs processor. This processor will batch the logs before sending them to the logs exporter. Open the `config.alloy` file in the `loki-fundamentals` directory and copy the following configuration: - + ```alloy otelcol.processor.batch "default" { output { @@ -134,14 +134,14 @@ otelcol.processor.batch "default" { } } ``` - + ### OpenTelemetry Logs Exporter Lastly, we will configure the OpenTelemetry logs exporter. This exporter will send the logs to Loki. Open the `config.alloy` file in the `loki-fundamentals` directory and copy the following configuration: - + ```alloy otelcol.exporter.otlphttp "default" { client { @@ -149,16 +149,16 @@ otelcol.exporter.otlphttp "default" { } } ``` - + ### Reload the Alloy configuration Once added, save the file. Then run the following command to request Alloy to reload the configuration: - + ```bash curl -X POST http://localhost:12345/-/reload ``` - + The new configuration will be loaded this can be verified by checking the Alloy UI: [http://localhost:12345](http://localhost:12345). @@ -166,42 +166,42 @@ The new configuration will be loaded this can be verified by checking the Alloy If you get stuck or need help creating the configuration, you can copy and replace the entire `config.alloy` using the completed configuration file: - + ```bash cp loki-fundamentals/completed/config.alloy loki-fundamentals/config.alloy curl -X POST http://localhost:12345/-/reload ``` - + - + - + ## Step 3: Start the Carnivorous Greenhouse In this step, we will start the Carnivorous Greenhouse application. To start the application, run the following command: - + {{< admonition type="note" >}} This docker-compose file relies on the `loki-fundamentals_loki` docker network. If you have not started the observability stack, you will need to start it first. {{< /admonition >}} - + - + - + - + ```bash docker compose -f lloki-fundamentals/greenhouse/docker-compose-micro.yml up -d --build ``` - + - + - + This will start the following services: ```bash @@ -224,4 +224,4 @@ Once started, you can access the Carnivorous Greenhouse application at [http://l Finally to view the logs in Loki, navigate to the Loki Logs Explore view in Grafana at [http://localhost:3000/a/grafana-lokiexplore-app/explore](http://localhost:3000/a/grafana-lokiexplore-app/explore). - \ No newline at end of file + \ No newline at end of file diff --git a/docs/transformer.md b/docs/transformer.md index c626569..c25a270 100644 --- a/docs/transformer.md +++ b/docs/transformer.md @@ -66,25 +66,25 @@ Copy directives tell the transform tool to make the contained fenced code block The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` #### Examples ````markdown - + ```bash echo 'Hello, world!' ``` - + ```` Produces: @@ -106,25 +106,25 @@ Exec directives tell the transform tool to make the contained fenced code block The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` #### Examples ````markdown - + ```bash echo 'Hello, world!' ``` - + ```` Produces: @@ -146,13 +146,13 @@ The finish directive specifies the start and end of the section of the file to u The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` ### Ignore @@ -162,13 +162,13 @@ The ignore directive tells the transform tool to skip the contents within the ma The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` #### Examples @@ -176,11 +176,11 @@ The end marker is: ```markdown Information common to both pages. - + Information unique to the Grafana website page. - + ``` Produces: @@ -197,13 +197,13 @@ The HTML comments aren't rendered on the Grafana website. The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` #### Examples @@ -211,9 +211,9 @@ The end marker is: ```markdown Information common to both pages. - + - + ``` Produces: @@ -231,13 +231,13 @@ The intro directive specifies the start and end of the section of the file to us The start marker is: ```markdown - + ``` The end marker is: ```markdown - + ``` ### Step @@ -247,13 +247,13 @@ The step directive specifies the start and end of the section of the file to use The start marker is the following, where _``_ is the number of the step: ```markdown - + ``` The end marker is the following, where _``_ is the number of the step: ```markdown - + ``` ## Generate a tutorial @@ -299,7 +299,7 @@ To generate a tutorial: Each step starts at the [step](#step) directive start marker and ends at the [step](#step) directive end marker. Include at least one step. - The first step use the start marker `` and the end marker `` + The first step use the start marker `` and the end marker `` 1. Configure a finish page. diff --git a/loki/alloy-otel-logs/index.json b/loki/alloy-otel-logs/index.json index c1cf1a9..418c521 100644 --- a/loki/alloy-otel-logs/index.json +++ b/loki/alloy-otel-logs/index.json @@ -1,6 +1,6 @@ { - "title": "Ingesting OpenTelemetry logs to Loki using Alloy", - "description": "Configuring Grafana Alloy to ingest OpenTelemetry logs to Loki.", + "title": "Sending OpenTelemetry logs to Loki using Alloy", + "description": "Configuring Grafana Alloy to send OpenTelemetry logs to Loki.", "details": { "intro": { "text": "intro.md" diff --git a/loki/alloy-otel-logs/intro.md b/loki/alloy-otel-logs/intro.md index 10a9d07..b3bc95f 100644 --- a/loki/alloy-otel-logs/intro.md +++ b/loki/alloy-otel-logs/intro.md @@ -1,6 +1,12 @@ -# Ingesting OpenTelemetry logs to Loki using Alloy +# Sending OpenTelemetry logs to Loki using Alloy -Alloy natively supports ingesting OpenTelemetry logs. In this example, we will configure Alloy to ingest OpenTelemetry logs to Loki. +Alloy natively supports receiving logs in the OpenTelemetry format. This allows you to send logs from applications instrumented with OpenTelemetry to Alloy, which can then be sent to Loki for storage and visualization in Grafana. In this example, we will make use of 3 Alloy components to achieve this: + +- **OpenTelemetry Logs Receiver:** This receiver will accept logs via HTTP and gRPC. + +- **OpenTelemetry Logs Processor:** This processor will batch the logs before sending them to the logs exporter. + +- **OpenTelemetry Logs Exporter:** This exporter will send the logs to Loki. ## Dependencies diff --git a/tools/transformer/main.go b/tools/transformer/main.go index d5c6769..8c8c01b 100644 --- a/tools/transformer/main.go +++ b/tools/transformer/main.go @@ -219,7 +219,7 @@ func transform(srcFilePath, dstDirPath string) error { ) for i := 1; i <= 5; i++ { - if regexp.MustCompile(fmt.Sprintf(``, i)).Match(data) { + if regexp.MustCompile(fmt.Sprintf(``, i)).Match(data) { steps++ if err := writeStep(dstDirPath, i, data, transformers, renderer); err != nil { diff --git a/tools/transformer/transform.go b/tools/transformer/transform.go index d9d676e..eb6656e 100644 --- a/tools/transformer/transform.go +++ b/tools/transformer/transform.go @@ -19,18 +19,18 @@ var ( ) const ( - copyStartMarker = "" - copyEndMarker = "" - execStartMarker = "" - execEndMarker = "" - finishStartMarker = "" - finishEndMarker = "" - ignoreStartMarker = "" - ignoreEndMarker = "" - includeStartMarker = "" - includeEndMarker = "" - introStartMarker = "" - introEndMarker = "" + copyStartMarker = "" + copyEndMarker = "" + execStartMarker = "" + execEndMarker = "" + finishStartMarker = "" + finishEndMarker = "" + ignoreStartMarker = "" + ignoreEndMarker = "" + includeStartMarker = "" + includeEndMarker = "" + introStartMarker = "" + introEndMarker = "" ) func isMarker(node ast.Node, source []byte, marker string) bool { @@ -509,8 +509,8 @@ type StepTransformer struct { // It removes all other nodes resulting in a document that only contains the content between the markers. // It removes the markers themselves. func (t *StepTransformer) Transform(root *ast.Document, reader text.Reader, _ parser.Context) { - stepStartMarker := fmt.Sprintf("", t.Step) - stepEndMarker := fmt.Sprintf("", t.Step) + stepStartMarker := fmt.Sprintf("", t.Step) + stepEndMarker := fmt.Sprintf("", t.Step) source := reader.Source() diff --git a/tools/transformer/transform_test.go b/tools/transformer/transform_test.go index 69baca5..c12955f 100644 --- a/tools/transformer/transform_test.go +++ b/tools/transformer/transform_test.go @@ -32,23 +32,23 @@ func TestActionTransformer_Transform(t *testing.T) { src := []byte("1. Create a directory called `evaluate-loki` for the demo environment.\n" + " Make `evaluate-loki` your current working directory:\n" + "\n" + - " \n" + + " \n" + "\n" + " ```bash\n" + " mkdir evaluate-loki\n" + " cd evaluate-loki\n" + " ```\n" + "\n" + - " \n" + + " \n" + "\n" + - " \n" + + " \n" + "\n" + " ```bash\n" + " mkdir evaluate-loki\n" + " cd evaluate-loki\n" + " ```\n" + "\n" + - " \n") + " \n") root := md.Parser().Parse(text.NewReader(src)) require.NoError(t, md.Renderer().Render(w, src, root)) @@ -208,13 +208,13 @@ func TestIgnoreTransformer_Transform(t *testing.T) { src := []byte(`## Install Loki and collecting sample logs - + {{< admonition type="note" >}} This quickstart assumes you are running Linux. {{< /admonition >}} - + **To install Loki locally, follow these steps:** `) @@ -241,11 +241,11 @@ func TestIncludeTransformer_Transform(t *testing.T) { md.Parser().AddOptions(parser.WithASTTransformers(util.Prioritized(&IncludeTransformer{}, 0))) md.SetRenderer(renderer.NewRenderer(renderer.WithNodeRenderers(util.Prioritized(markdown.NewRenderer(), 1000)))) - src := []byte("\n" + + src := []byte("\n" + "\n" + "\n" + "\n" + - "\n") + "\n") root := md.Parser().Parse(text.NewReader(src)) require.NoError(t, md.Renderer().Render(w, src, root)) @@ -268,7 +268,7 @@ func TestIntroTransformer_Transform(t *testing.T) { md.Parser().AddOptions(parser.WithASTTransformers(util.Prioritized(&IntroTransformer{}, 0))) md.SetRenderer(renderer.NewRenderer(renderer.WithNodeRenderers(util.Prioritized(markdown.NewRenderer(), 1000)))) - src := []byte(` + src := []byte(` # Quickstart to run Loki locally @@ -279,7 +279,7 @@ The Docker Compose configuration instantiates the following components, each in - **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway. - **Grafana** which provides visualization of the log lines captured within Loki. - + `) root := md.Parser().Parse(text.NewReader(src))