MDC in streams

[oleksii-suprun]

Does it make any sense to use MDC in streaming applications? If it does, what is the best place to initialize it? I want to attach some information to each log message from environment variables and from the message itself.

[garyrussell]

Depending on the topology, Kafka Streams can switch threads internally so MDC is pretty useless, except for extremely simple streams.

[oleksii-suprun]

@garyrussell, thank you for the clarification. Since Kafka Streams can switch streams, it is dangerous to use MDC because even if I have a simple stream today, there are no guarantees it will not become more complex tomorrow.

Do you have any advice on how I can improve logging in my streaming application to have more context during troubleshooting without MDC? Is there any better way than manually putting the required values to each logging message?

[artembilan]

This is something outside of Spring scope.
See some ideas around tracing: https://blog.jdriven.com/2019/10/distributed-tracing-with-kafka-streams/

[oleksii-suprun]

Thank you, @artembilan! I'll look and see how I can apply it to my app.