Collector writes the body field as message template and doesn't replace the place holder strings

[HHobeck]

Dear community.

Maybe this is by design and the problem is located somewhere else... but in my setup I have messages in Elastic which are not rendered propably with the replaced meta data. I'm sending the application log data from the otel-collector via otelp to the APM Server in Elastic.

Following configuration I have used runnin the otel-collector:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  otlp/elastic: 
    # Elastic APM server https endpoint without the "https://" prefix
    endpoint: https://tempuri.aws.cloud.es.io:443
    headers:
      # Elastic APM Server secret token
      Authorization: Bearer xxxxxxxxxxxxxx

extensions:
  health_check:

service:
  pipelines:
    logs: 
      receivers: [otlp]
      exporters: [otlp/elastic] 

Here is an example otelp log which has been sent to Elastic:

{
    "resourceLogs": [
        {
            "resource": {
                "attributes": [
                    {
                        "key": "service.name",
                        "value": {
                            "stringValue": "MyNamespace.Application"
                        }
                    },
                    {
                        "key": "service.instance.id",
                        "value": {
                            "stringValue": "fa0ebbb6-0c98-071b-bdab-720dd5fbc3e4"
                        }
                    },
                    {
                        "key": "telemetry.sdk.name",
                        "value": {
                            "stringValue": "opentelemetry"
                        }
                    },
                    {
                        "key": "telemetry.sdk.language",
                        "value": {
                            "stringValue": "dotnet"
                        }
                    },
                    {
                        "key": "telemetry.sdk.version",
                        "value": {
                            "stringValue": "1.7.0"
                        }
                    }
                ]
            },
            "scopeLogs": [
                {
                    "scope": {
                        "name": "System.Net.Http.HttpClient.MyNamespace.HttpClientProxy.ClientHandler"
                    },
                    "logRecords": [
                        {
                            "timeUnixNano": "1707903724700129600",
                            "observedTimeUnixNano": "1707903724700129600",
                            "severityNumber": 9,
                            "severityText": "Information",
                            "body": {
                                "stringValue": "Sending HTTP request {HttpMethod} {Uri}"
                            },
                            "attributes": [
                                {
                                    "key": "HttpMethod",
                                    "value": {
                                        "stringValue": "PUT"
                                    }
                                },
                                {
                                    "key": "Uri",
                                    "value": {
                                        "stringValue": "https://tempuri.org/"
                                    }
                                }
                            ],
                            "traceId": "12182ed13ca72ba553150644bffe83c4",
                            "spanId": "12cd0a28725009e0"
                        },
                        {
                            "timeUnixNano": "1707903725065731600",
                            "observedTimeUnixNano": "1707903725065731600",
                            "severityNumber": 9,
                            "severityText": "Information",
                            "body": {
                                "stringValue": "Received HTTP response headers after {ElapsedMilliseconds}ms - {StatusCode}"
                            },
                            "attributes": [
                                {
                                    "key": "ElapsedMilliseconds",
                                    "value": {
                                        "doubleValue": 357.8655
                                    }
                                },
                                {
                                    "key": "StatusCode",
                                    "value": {
                                        "intValue": "200"
                                    }
                                }
                            ],
                            "traceId": "12182ed13ca72ba553150644bffe83c4",
                            "spanId": "12cd0a28725009e0"
                        }
                    ]
                }
            ]
        }
    ]
}

The Elastic APM Server converts the otlp message to the ecs standard which looks good. But I see the place holder string in Elastic in the message which is not what I would expected. My question is: Is it by design that otelp contains the message template in the body field and not the rendered message itself? If not how can render the message template and replace the strings with the attribute field values in the otel-collector?

Thank you very much for the help.

Regards
Hardy

[HHobeck]

Obviously the opentelemetry collector expects the full rendered message in the body field. Maybe it might be an idea to create a own processor to support also message templating. Anyway I have missed the option in OpenTelemetry.Logs.OpenTelemetryLoggerOptions on the application client:

loggingBuilder.AddOpenTelemetry(options =>
{
    options.IncludeFormattedMessage = true;
    options.AddOtlpExporter(options =>
    {
        options.Endpoint = new Uri("http://localhost:4318/v1/logs");
        options.Protocol = OtlpExportProtocol.HttpProtobuf;
    });
});