Microsoft.Extensions.Logging - Async Logger Example

[gumbarros]

Making logging async would be very disruptive as it would mean every code path that calls logging would have to be async. It also means that the caller is likely blocked (even if via await) waiting for the log message to be sent.

The recommendation here is to use background worker. This is even what the console logger does (even though it's all in-process). Any logger that does heavy I/O should consider adopting this pattern:

1. The logger is synchronous and just adds messages to an internal buffer
2. A background IHostedService runs and collects messages from this buffer, flushing them to the I/O sink (remote service, console, file, etc.)

Originally posted by @analogrelay in dotnet/extensions#2141 (comment)

Can anyone provide an example of the recommendation above? Is my implementation fine?

FileLogger:
https://github.com/JJConsulting/JJMasterData/blob/sun/src/Commons/Logging/File/FileLogger.cs

FileLoggerBackgroundService:
https://github.com/JJConsulting/JJMasterData/blob/sun/src/Commons/Logging/File/FileLoggerBackgroundService.cs

LoggerBackgroundService:
https://github.com/JJConsulting/JJMasterData/blob/sun/src/Commons/Logging/LoggerBackgroundService.cs

[davidfowl]

1. The FileLogger should not capture any arguments from the call to log, it should serialize or copy the data immediately before passing it to the background service.
2. The buffer needs a maximum size or you'll run out of memory if logging happens too quickly.

https://github.com/JJConsulting/JJMasterData/blob/f639ca2b96d8eadbc9b63215fad4ea4149db04c1/src/Commons/Logging/File/FileLoggerBackgroundService.cs#L82

Consider using System.Text.Json instead of JSON.NET to serialize, did you measure the performance of this?

[gumbarros]

Hi @davidfowl ! First of all thanks for your response! Sorry for the late reply, I was on vacation.

1. So I need to send the log data as a string instead of a LogMessage object?
2. Can you provide an example on how to do this at my LoggerBuffer ?

[davidfowl]

So I need to send the log data as a string instead of a LogMessage object?

String/Memory/char[] Some materialized version of the logged message.

The buffer needs a maximum size or you'll run out of memory if logging happens too quickly.

A naive implementation would remove items from the queue on a size check. You can also consider using a bounded channel (Channel.CreateBounded(..)).