OpenTelemetry Logging with Django

[willianantunes]

Django uses the dictionary LOGGING to configure the logging system (check its documentation). Talking about programmatically configuration, OpenTelemetry Logging Instrumentation can be configured in the manage.py file and gunicorn_config.py file if you are using Gunicorn. If you set LoggingInstrumentor().instrument(tracer_provider=provider, set_logging_format=True) in both files, it won't work properly, because Django will call logging.config.dictConfig(LOGGING) later, and it will overwrite the configuration set by OpenTelemetry. To solve this issue, you can create a class like the following:

import logging

from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._logs import LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter

class CustomLoggingHandler(LoggingHandler):
    def __init__(self, level=logging.NOTSET, logger_provider=None) -> None:
        logger_provider = LoggerProvider()
        set_logger_provider(logger_provider)
        logger_provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))
        super().__init__(level, logger_provider)

Then you can set the LOGGING dictionary like this:

...
},
"handlers": {
    "console": {
        "class": "logging.StreamHandler",
        "filters": ["request_id", "redact_filter"],
        "formatter": os.getenv("DEFAULT_LOG_FORMATTER", "standard"),
    },
    "otlp": {"class": "otlp.CustomLoggingHandler"},
},
"loggers": {
    "": {"level": os.getenv("ROOT_LOG_LEVEL", "INFO"), "handlers": ["console", "otlp"]},
...

Notice the custom handler named otlp. Remember, according to the documentation, the APIs within opentelemetry.sdk._logs are subject to change. So, it is important to keep an eye on the updates. Libraries I used at the time I posted this comment:

django = "5.1.3"
opentelemetry-sdk = "1.28.1"
opentelemetry-instrumentation-django = "0.49b1"
opentelemetry-instrumentation-psycopg = "0.49b1"
opentelemetry-instrumentation-wsgi = "0.49b1"
opentelemetry-instrumentation-logging = "0.49b1"
opentelemetry-exporter-otlp = "1.28.1"

Sample project.