'style' appears to be nonfunctional

[nonnull-ca]

How can I configure this library to use new-style {} for formatting?

I note the code appears to have style arguments in appropriate places, e.g. here - but I cannot actually get this working.

Some things I tried:

import logger_tt
logger_tt.setup_logging(full_context=1)
logger_tt.logger.critical("hello {}", "world")
# TypeError: not all arguments converted during string formatting

import logger_tt
logger_tt.setup_logging(full_context=1, style="{")
logger_tt.logger.critical("hello {}", "world")
# ValueError: logger_tt unknown fields: {'style'}

The following are all using this python and different cfg.yamls:

import logger_tt
logger_tt.setup_logging(config_path="cfg.yaml")
logger_tt.logger.critical("hello {}", "world")

Default file, verbatim:

version: 1
disable_existing_loggers: False
formatters:
  simple:
    format: "[%(asctime)s] [%(name)s:%(lineno)d %(levelname)s] %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"
  brief:
    format: "[%(asctime)s] %(levelname)s: %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"

handlers:
  console:
    class: logging.StreamHandler
    level: INFO
    formatter: brief
    stream: ext://sys.stdout

  error_file_handler:
    class: logging.handlers.TimedRotatingFileHandler
    level: DEBUG
    formatter: simple
    filename: logs/log.txt
    backupCount: 15
    encoding: utf8
    when: midnight
  
  buffer_stream_handler:
    class: logger_tt.handlers.StreamHandlerWithBuffer
    level: DEBUG
    formatter: brief
    stream: ext://sys.stdout
    buffer_time: 0.5
    buffer_lines: 0
    debug: False
    
  telegram_handler:
    class: logger_tt.handlers.TelegramHandler
    level: NOTICE
    formatter: brief
    debug: False
    token: "your bot token here or set the below env key to fetch from environ for better security"
    unique_ids: "semicolon separated of [name:]chat_id[@message_thread_id]"
    env_token_key: "TELEGRAM_BOT_LOG_TOKEN"
    env_unique_ids_key: "TELEGRAM_BOT_LOG_DEST"

loggers:
  urllib3:
    level: WARNING
    handlers: [console, error_file_handler]
    propagate: no

root:
  level: DEBUG
  handlers: [console, error_file_handler]

logger_tt:
  suppress: ["exchangelib"]
  suppress_level_below: "WARNING"
  capture_print: False
  strict: False
  guess_level: False
  full_context: 0
  use_multiprocessing: False
  limit_line_length: 1000
  analyze_raise_statement: False
  default_logger_formats:
    normal: ["%(name)s", "%(filename)s"]
    thread: ["%(message)s", "%(threadName)s %(message)s"]
    multiprocess: ["%(message)s", "%(processName)s %(message)s"]
    both: ["%(message)s", "%(processName)s %(threadName)s %(message)s"]

TypeError: not all arguments converted during string formatting

Sure, this makes sense. So let's mark the formatters as style: {:

version: 1
disable_existing_loggers: False
formatters:
  simple:
    style: '{'
    format: "[%(asctime)s] [%(name)s:%(lineno)d %(levelname)s] %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"
  brief:
    style: '{'
    format: "[%(asctime)s] %(levelname)s: %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"

[...]

([...] in the above is unchanged from the first example)

Traceback (most recent call last):
  File "[...]/logging/config.py", line 544, in configure
    formatters[name] = self.configure_formatter(
  File "[...]/logging/config.py", line 683, in configure_formatter
    result = c(fmt, dfmt, style)
  File "[...]/site-packages/logger_tt/core.py", line 383, in __init__
    self._logger_tt_formatters[case] = logging.Formatter(fmt=fmt, datefmt=datefmt, style=style)
  File "[...]/logging/__init__.py", line 589, in __init__
    self._style.validate()
  File "[...]logging/__init__.py", line 476, in validate
    raise ValueError('invalid format: no fields')
ValueError: invalid format: no fields

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/twotothe/logger_tt_newstyle.py", line 2, in <module>
    logger_tt.setup_logging(config_path="cfg.yaml")
  File "[...]/site-packages/logger_tt/__init__.py", line 217, in setup_logging
    logging.config.dictConfig(config)
  File "[...]/logging/config.py", line 810, in dictConfig
    dictConfigClass(config).configure()
  File "[...]/logging/config.py", line 547, in configure
    raise ValueError('Unable to configure '
ValueError: Unable to configure formatter 'simple'

It's complaining that the formatter has no fields. Sure, it probably needs the format changed too to newstyle. Let's try that:

version: 1
disable_existing_loggers: False
formatters:
  simple:
    style: '{'
    format: "[{asctime}] [{name}:{lineno} {levelname}] {message}"
    datefmt: "%Y-%m-%d %H:%M:%S"
  brief:
    style: '{'
    format: "[{asctime}] {levelname}: {message}"
    datefmt: "%Y-%m-%d %H:%M:%S"

[...]

Traceback (most recent call last):
  File "[...]/logging/config.py", line 544, in configure
    formatters[name] = self.configure_formatter(
  File "[...]/logging/config.py", line 683, in configure_formatter
    result = c(fmt, dfmt, style)
  File "[...]/site-packages/logger_tt/core.py", line 379, in __init__
    super(DefaultFormatter, self).__init__(fmt=fmt, datefmt=datefmt)
  File "[...]/logging/__init__.py", line 589, in __init__
    self._style.validate()
  File "[...]/logging/__init__.py", line 429, in validate
    raise ValueError("Invalid format '%s' for '%s' style" % (self._fmt, self.default_format[0]))
ValueError: Invalid format '[{asctime}] [{name}:{lineno} {levelname}] {message}' for '%' style

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[...]/logger_tt_newstyle.py", line 2, in <module>
    logger_tt.setup_logging(config_path="cfg.yaml")
  File "[...]/site-packages/logger_tt/__init__.py", line 217, in setup_logging
    logging.config.dictConfig(config)
  File "[...]/logging/config.py", line 810, in dictConfig
    dictConfigClass(config).configure()
  File "[...]/logging/config.py", line 547, in configure
    raise ValueError('Unable to configure '
ValueError: Unable to configure formatter 'simple'

...wait, what? But it just complained a moment ago that %-style formatting didn't work!

[Dragon2fly]

Hi @nonnull-ca

Firstly, the problem was indeed the following line:

  File "[...]/site-packages/logger_tt/core.py", line 379, in __init__
    super(DefaultFormatter, self).__init__(fmt=fmt, datefmt=datefmt)

It should have also passed in the style param when calling super.

But this fix is just applied for the format of the formatters as the Python doc says so.
eg. this line: format: "[{asctime}] [{name}:{lineno} {levelname}] {message}"

Secondly, since your target is to be able to use the bracket in the individual log message,
like logger_tt.logger.critical("hello {}", "world"),
this needs more work as it is not supported by the standard logging module.

Thirdly, please note that since logger-tt uses QueueHandler and SocketHandler behind the scene, these handlers will merge the message string and arguments before putting the log record into the queue or sending it. This is just to ensure the log record is pickleable. This behavior will also defeat the purpose of not evaluating the message string if the log record is filtered and only if you have that intention in mind. Anyway, since logger-tt does the logging in a different thread or process, the performance impact is neglectable.

So if you want to use the bracket format in the individual log message, please wait a little more until this function is implemented.