Custom userdata in spdlog::details::log_msg

[sketch34]

I want to check if this is acceptable before attempting a PR or if there is existing work. Could be related to structured logging, but I think what I need is simpler.

Proposal

It would be great if I could attach custom user data to spdlog::details::log_msg via a simple void* pointer. From there I can add a regular spdlog::custom_flag_formatter to use the pointer data however I need.

In my case I want a way to add arbitrary prefixes at the start of the log line. spdlog supports custom log level names but I need more prefixes than spdlog::level::n_levels and I need the semantics separated from the logging levels.

<my custom prefix> 09/11 00:18:04.261 16511 source.cpp:129 [void namespace::func()]: <msg content>

This could be optional behind #ifdef SPDLOG_LOG_MSG_USERDATA.

Alternative

I've already implemented an alternative where I encode the info I need in the log_msg::payload and then decode it in a custom flag formatter. Two main downsides to this approach:

• Additional CPU to encode / decode.
• Requires a const_cast in the flag formatter to "trim" the encoded info out of the payload. Works now but could violate spdlog preconditions in the future.

I'm not happy with this alternative as a production solution.

class prefix_formatter_flag : public spdlog::custom_flag_formatter
{
public:
    void format(spdlog::details::log_msg const& msg, const std::tm&, spdlog::memory_buf_t& dest) override
    {
      auto& payload = *const_cast<spdlog::string_view_t*>(&msg.payload);
      auto const prefix = trim_prefix(payload);
      dest.append(prefix);
    }
    std::unique_ptr<custom_flag_formatter> clone() const override
    {
      return spdlog::details::make_unique<prefix_formatter_flag>();
    }
};

inline spdlog::string_view_t trim_prefix(spdlog::string_view_t& in_out_sv)
{
  if (in_out_sv.size() == 0) {
    BOOST_ASSERT(false);
    return {};
  }

  auto p = in_out_sv.data();

  // Never allow an unbounded string length check, using memchr in this case.
  auto const end = static_cast<decltype(p)>(std::memchr(p, k_aelog_prefix_terminator, in_out_sv.size()));
  auto prefix_length = static_cast<size_t>(end - p);
  spdlog::string_view_t prefix{p, prefix_length};

  BOOST_ASSERT(prefix.size());

  #ifdef SPDLOG_USE_STD_FORMAT
    in_out_sv = in_out_sv.substr(prefix.size() + 1);
  #else
    in_out_sv.remove_prefix(prefix.size() + 1);  // + 1 for the terminator.
  #endif

  return prefix;
}

[tt4g]

Since structures are incompatible if even one attribute is added, adding one attribute would be no different than allowing users to use their own log_msg.
With that in mind, allowing users to use their own log_msg (compatible with spdlog's API) would provide a better logging experience.
However, to do this, log_msg would have to be a template parameter (breaking change).
@gabime's feedback would be essential for any breaking change.

[sketch34]

Yeah I was thinking about a templated log_msg as well, but that would probably involve a lot of source changes. A simple void* pointer achieves almost the same thing, but with the downside of worse locality of reference. Could also use a macro to optionally inject a custom member variable declaration into log_msg. Would need to be careful with compiled vs header-only spdlog. Need to think some more about it. If @gabime is open to this idea in general I'll do so.

[gabime]

How about using mdc ?

[sketch34]

I looked into this but two issues:

• No way to tell each log_msg which data in the mdc to use.
• mdc doesn't support async_logger.