german umlauts in domains breaks logparser

[tonioo]

From @egon0 on March 9, 2018 13:14

Impacted versions

• Modoboa: 1.10.3
• installer used: Yes
• Webserver: Nginx

Steps to reproduce

add a domain or a domain-alias with german umlauts

Current behavior

logparser fails with

UnicodeEncodeError: 'ascii' codec can't encode characters in position 2-3: ordinal not in range(128)

Expected behavior

logparser should parse the logs ;)

Copied from original issue: modoboa/modoboa#1431

[tonioo]

@egon0 Can you please copy the full error?

[tonioo]

From @y0va on March 10, 2018 15:38

Here's the same, also for 10.0.3:

(env)email# python manage.py logparser
/srv/modoboa/env/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 586, in handle
    None, greylist
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 72, in __init__
    self._load_domain_list()
  File "/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_stats/management/commands/logparser.py", line 130, in _load_domain_list
    aliasname = str(alias.name)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 4: ordinal not in range(128)

[tonioo]

From @egon0 on March 12, 2018 20:56

@tonioo @y0va was faster. :)

[tonioo]

Thank you, I'm going to move this issue to the right repository.

[incase]

Any chance there will be any work to fix this?
It seems to almost be the only thing that prevents me from using my umlaut domain (well, except that postfix-mysql doesn't use a utf8 charset by default, so a fix to generate_postfix_maps and a my.cnf setting are required as well).