Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(opentelemetry): add opentelemetry variables #8871

Merged
merged 10 commits into from
Sep 21, 2023
16 changes: 16 additions & 0 deletions apisix/plugins/opentelemetry.lua
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ local attr_schema = {
},
default = {},
},
set_ngx_var = {
type = "boolean",
description = "set nginx variables",
default = false,
},
Comment on lines +116 to +120
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
set_ngx_var = {
type = "boolean",
description = "set nginx variables",
default = false,
},
set_ngx_var = {
type = "boolean",
description = "set nginx variables",
default = false,
},

},
}

Expand Down Expand Up @@ -332,6 +337,17 @@ function _M.rewrite(conf, api_ctx)
kind = span_kind.server,
attributes = attributes,
})

if plugin_info.set_ngx_var then
local span_context = ctx:span():context()
ngx_var.opentelemetry_context_traceparent = string.format("00-%s-%s-%02x",
span_context.trace_id,
span_context.span_id,
span_context.trace_flags)
ngx_var.opentelemetry_trace_id = span_context.trace_id
ngx_var.opentelemetry_span_id = span_context.span_id
end

api_ctx.otel_context_token = ctx:attach()

-- inject trace context into the headers of upstream HTTP request
Expand Down
26 changes: 26 additions & 0 deletions docs/en/latest/plugins/opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,32 @@ plugin_attr:
max_export_batch_size: 2
```

## Variables

The following nginx variables are set by OpenTelemetry:

- `opentelemetry_context_traceparent` - [W3C trace context](https://www.w3.org/TR/trace-context/#trace-context-http-headers-format), e.g.: `00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01`
- `opentelemetry_trace_id` - Trace Id of the current span
- `opentelemetry_span_id` - Span Id of the current span

How to use variables? you have to add it to your configuration file (`conf/config.yaml`):

```yaml title="./conf/config.yaml"
nginx_config:
http_server_configuration_snippet: |
set $opentelemetry_context_traceparent ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I think there is another way that we can add those lines in cli/ngx_tpl.lua automatically if we enabled the "OpenTelemetry" plugins.

Else APISIX will panic the request is we forget to set those line but enable the set_ngx_var options in the plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I added some code

set $opentelemetry_trace_id ""
set $opentelemetry_span_id ""
http:
enable_access_log: true
access_log: "/dev/stdout"
access_log_format: '{"time": "$time_iso8601","opentelemetry_context_traceparent": "$opentelemetry_context_traceparent","opentelemetry_trace_id": "$opentelemetry_trace_id","opentelemetry_span_id": "$opentelemetry_span_id","remote_addr": "$remote_addr","uri": "$uri"}'
access_log_format_escape: json
plugin_attr:
opentelemetry:
set_ngx_var: true
```

## Enabling the Plugin

To enable the Plugin, you have to add it to your configuration file (`conf/config.yaml`):
Expand Down
26 changes: 26 additions & 0 deletions docs/zh/latest/plugins/opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,32 @@ plugin_attr:
max_export_batch_size: 2
```

## 如何使用变量

以下`nginx`变量是由`opentelemetry` 设置的。

- `opentelemetry_context_traceparent` - [W3C trace context](https://www.w3.org/TR/trace-context/#trace-context-http-headers-format), 例如:`00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01`
- `opentelemetry_trace_id` - 当前 span 的 trace_id
- `opentelemetry_span_id` - 当前 span 的 span_id

如何使用?你需要在配置文件(`./conf/config.yaml`)设置如下:

```yaml title="./conf/config.yaml"
nginx_config:
http_server_configuration_snippet: |
set $opentelemetry_context_traceparent ""
set $opentelemetry_trace_id ""
set $opentelemetry_span_id ""
http:
enable_access_log: true
access_log: "/dev/stdout"
access_log_format: '{"time": "$time_iso8601","opentelemetry_context_traceparent": "$opentelemetry_context_traceparent","opentelemetry_trace_id": "$opentelemetry_trace_id","opentelemetry_span_id": "$opentelemetry_span_id","remote_addr": "$remote_addr","uri": "$uri"}'
access_log_format_escape: json
plugin_attr:
opentelemetry:
set_ngx_var: true
```

## 如何启用

`opentelemetry` 插件默认为禁用状态,你需要在配置文件(`./conf/config.yaml`)中开启该插件:
Expand Down
168 changes: 168 additions & 0 deletions t/plugin/opentelemetry3.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

use t::APISIX 'no_plan';

repeat_each(1);
no_long_string();
no_root_location();
log_level("info");

add_block_preprocessor(sub {
my ($block) = @_;

if (!$block->extra_yaml_config) {
my $extra_yaml_config = <<_EOC_;
plugins:
- http-logger
- opentelemetry
plugin_attr:
opentelemetry:
set_ngx_var: true
batch_span_processor:
max_export_batch_size: 1
inactive_timeout: 0.5
_EOC_
$block->set_value("extra_yaml_config", $extra_yaml_config);
}

my $upstream_server_config = $block->upstream_server_config // <<_EOC_;
set \$opentelemetry_context_traceparent "";
set \$opentelemetry_trace_id "";
set \$opentelemetry_span_id "";
access_log /tmp/access.log opentelemetry_log;
_EOC_

$block->set_value("upstream_server_config", $upstream_server_config);

my $http_config = $block->http_config // <<_EOC_;
log_format opentelemetry_log '{"time": "\$time_iso8601","opentelemetry_context_traceparent": "\$opentelemetry_context_traceparent","opentelemetry_trace_id": "\$opentelemetry_trace_id","opentelemetry_span_id": "\$opentelemetry_span_id","remote_addr": "\$remote_addr","uri": "\$uri"}';
_EOC_

$block->set_value("http_config", $http_config);

if (!$block->extra_init_by_lua) {
my $extra_init_by_lua = <<_EOC_;
-- mock exporter http client
local client = require("opentelemetry.trace.exporter.http_client")
client.do_request = function()
ngx.log(ngx.INFO, "opentelemetry export span")
return "ok"
end
_EOC_

$block->set_value("extra_init_by_lua", $extra_init_by_lua);
}


if (!$block->request) {
$block->set_value("request", "GET /t");
}

$block;
});

run_tests;

__DATA__

=== TEST 1: add plugin metadata
--- config
location /t {
content_by_lua_block {
local t = require("lib.test_admin").test
local code, body = t('/apisix/admin/plugin_metadata/http-logger',
ngx.HTTP_PUT,
[[{
"log_format": {
"opentelemetry_context_traceparent": "$opentelemetry_context_traceparent",
"opentelemetry_trace_id": "$opentelemetry_trace_id",
"opentelemetry_span_id": "$opentelemetry_span_id"
}
}]]
)
if code >= 300 then
ngx.status = code
return body
end

local code, body = t('/apisix/admin/routes/1',
ngx.HTTP_PUT,
[[{
"plugins": {
"http-logger": {
"uri": "http://127.0.0.1:1980/log",
"batch_max_size": 1,
"max_retry_count": 1,
"retry_delay": 2,
"buffer_duration": 2,
"inactive_timeout": 2,
"concat_method": "new_line"
},
"opentelemetry": {
"sampler": {
"name": "always_on"
}
}
},
"upstream": {
"nodes": {
"127.0.0.1:1982": 1
},
"type": "roundrobin"
},
"uri": "/hello"
}]]
)

if code >=300 then
ngx.status = code
end
ngx.say(body)
}
}
--- request
GET /t
--- response_body
passed


=== TEST 2: trigger opentelemetry with open set variables
--- request
GET /hello
--- response_body
hello world
--- wait: 1
--- grep_error_log eval
qr/opentelemetry export span/
--- grep_error_log_out
opentelemetry export span
--- error_log eval
qr/request log: \{.*"opentelemetry_context_traceparent":"00-\w{32}-\w{16}-01".*\}/


=== TEST 3: trigger opentelemetry with disable set variables
--- yaml_config
plugin_attr:
opentelemetry:
set_ngx_var: false
--- request
GET /hello
--- response_body
hello world
--- error_log eval
qr/request log: \{.*"opentelemetry_context_traceparent":"".*\}/