Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Labcontroller compatible with Python 2 and Python 3 #227

Closed
wants to merge 81 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
557e290
feat: enable lab-controller build for Python 3
StykMartin Jan 19, 2024
6783017
fix(async): rename module to avoid use of reserved keyword
StykMartin Jan 19, 2024
046ec78
fix(test): use range instead of xrange
StykMartin Jan 19, 2024
b3b291c
fix(test): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
8bd61eb
fix(watchdog): use six to provide xmlrpc client
StykMartin Jan 19, 2024
59ca7ed
fix(watchdog): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
2d257ed
chore(watchdog): format code and sort imports
StykMartin Jan 19, 2024
d015ee2
chore(utils): format code and sort imports
StykMartin Jan 19, 2024
97f1633
fix(transfer): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
160f2d8
chore(transfer): format code and sort imports
StykMartin Jan 19, 2024
e182e16
fix(transfer): handle SSL error on Python 6
StykMartin Jan 19, 2024
fe53e5a
chore(test): format code and sort imports
StykMartin Jan 19, 2024
392ae39
fix(test): use proper octal format
StykMartin Jan 19, 2024
1babbbe
fix(pxemenu): use xmlrpc client and urllib from six
StykMartin Jan 19, 2024
b365a10
chore(pxemenu): format code and sort imports
StykMartin Jan 19, 2024
6a00b9a
fix(proxy): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
f8c3eea
fix(proxy): use xmlrpc move from six
StykMartin Jan 19, 2024
4698017
chore(proxy): format code and sort imports
StykMartin Jan 19, 2024
df27135
fix(provision): use xmlrpc move from six
StykMartin Jan 19, 2024
257de1c
fix(provision): use unicode from six
StykMartin Jan 19, 2024
97b3d7b
fix(provision): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
ceb0a39
fix(provision): handle iterators and dict with six
StykMartin Jan 19, 2024
cd9aecd
chore(provision): format code and sort imports
StykMartin Jan 19, 2024
f315c74
fix(netboot): import StringIO from six
StykMartin Jan 19, 2024
087709f
fix(netboot): use urllib from six
StykMartin Jan 19, 2024
08a8953
fix(netboot): use file mode compatible with six
StykMartin Jan 19, 2024
a078345
chore(netboot): format code and sort imports
StykMartin Jan 19, 2024
13505c8
fix(proxy-main): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
278181b
fix(proxy-main): make xmlrpc imports compatible
StykMartin Jan 19, 2024
8b842b6
chore(proxy-main): format code and sort imports
StykMartin Jan 19, 2024
96efb20
fix(log-storage): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
f9bdd1e
fix(log-storage): use proper octal format
StykMartin Jan 19, 2024
8842505
chore(log-storage): format code and sort imports
StykMartin Jan 19, 2024
40e6bb3
fix(pxemenu): use xmlrpc client and urllib from six
StykMartin Jan 19, 2024
2a9fbe2
chore(expire-distros): format code and sort imports
StykMartin Jan 19, 2024
283e3fd
fix(distro-import): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
ca713cd
fix(distro-import): call fn print
StykMartin Jan 19, 2024
313b0ee
fix(pxemenu): use xmlrpc client,urllib, and configparser from six
StykMartin Jan 19, 2024
9483957
fix(distro-import): use anonymous fn instead string::strip
StykMartin Jan 19, 2024
40a936e
chore(distro-import): format code and sort imports
StykMartin Jan 19, 2024
3919ba2
fix(distro-import): use is to compare None
StykMartin Jan 19, 2024
8857893
fix(distro-import): simplify comparison
StykMartin Jan 19, 2024
8df208e
chore(config): format code and sort imports
StykMartin Jan 19, 2024
8d649ec
fix(concurrency): update exception syntax for Python 3 compatibility
StykMartin Jan 19, 2024
1d2f31c
chore(concurrency): format code and sort imports
StykMartin Jan 19, 2024
daef1d6
chore(clear-netboot): format code and sort imports
StykMartin Jan 19, 2024
d1fce19
tests: use pytest w/ python3
StykMartin Jan 19, 2024
fa58bd9
fix(proxy): use absolute import to import utils
StykMartin Jan 19, 2024
5069073
ci: run unit tests during pull requests
StykMartin Jan 19, 2024
c14e540
fix(provision): use datetime fn instead on using custom
StykMartin Jan 19, 2024
75bef29
fix(common-helpers): always encode unicode to str
StykMartin Jan 19, 2024
497f87b
fix(spec): add werkzeug to builddeps
StykMartin Jan 19, 2024
964772c
fix(test): use file mode compatible with python six
StykMartin Jan 19, 2024
c0736c4
fix(test): always write bytes to test images
StykMartin Jan 19, 2024
9cc592f
fix(test): drop deprecated fn names
StykMartin Jan 19, 2024
12e8abc
fix(concurrency): wait till fileno is ready
StykMartin Jan 21, 2024
a06a82f
fix(concurrency): decode bytes
StykMartin Jan 21, 2024
b89bd27
test: extend test suite to validate concurrency
StykMartin Jan 21, 2024
3ac3e93
fix(proxy-main): use pywsgi instead of deprecated wsgi
StykMartin Jan 21, 2024
3fe496a
test: refactor _assert_process_group_is_removed to use psutil
StykMartin Jan 21, 2024
d83fbc9
ci: run init inside the container that forwards signals
StykMartin Jan 21, 2024
359066a
ci: define timeout for unit and integration tests
StykMartin Jan 21, 2024
4d08ee5
ci: run unit test on CentOS 9 Stream
StykMartin Jan 21, 2024
491b538
ci: show python environment
StykMartin Jan 21, 2024
af764fe
chore: remove init.d configuration
StykMartin Jan 21, 2024
0693013
Make source code cloneable on Windows
JohnVillalovos Dec 4, 2021
0d8c870
chore: remove unused check_output
StykMartin Jan 21, 2024
f8262f1
fix(proxy): explicitly convert ascii_control_chars to list for Py3 co…
StykMartin Jan 21, 2024
138a41d
fix(proxy): use raw string notation for ANSI escape code regex
StykMartin Jan 21, 2024
1932974
fix: use warning instead of deprecated warn in logger
StykMartin Jan 21, 2024
451e548
chore: noqa for broad exceptions in main loops
StykMartin Jan 21, 2024
11d6660
refactor(watchdog): remove shadowing for variable greenlet
StykMartin Jan 21, 2024
f6e8b00
fix(watchdog): handle missing 'Running' task in recipe abort process
StykMartin Jan 21, 2024
932c4aa
refactor(utils): remove unused code
StykMartin Jan 21, 2024
ab2bd02
refactor(test): rename 'id' to 'entity_id' to avoid shadowing built-i…
StykMartin Jan 21, 2024
3c38986
refactor(proxy): always close hub
StykMartin Jan 21, 2024
0ae7e70
docs: generate man page for beaker-import
StykMartin Jan 21, 2024
7062f60
ci: execute unit tests on fedora
StykMartin Jan 21, 2024
584f728
build: manage logrotate and log dir for py3 targets
StykMartin Jan 21, 2024
d84bcdd
refactor: use absolute import for utility bkr utility library
StykMartin Jan 21, 2024
20896e9
fix(provision): encode power env fields on python 2
StykMartin Jan 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions LabController/src/bkr/labcontroller/provision.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,13 +230,27 @@ def find_power_script(power_type):
raise ValueError("Invalid power type %r" % power_type)


def _decode(value):
# Decode if we are running python2 and value is unicode
if six.PY2 and isinstance(value, six.text_type):
return value.encode("utf8")
return value


def build_power_env(command):
env = dict(os.environ)
env["power_address"] = (command["power"].get("address") or "").encode("utf8")
env["power_id"] = (command["power"].get("id") or "").encode("utf8")
env["power_user"] = (command["power"].get("user") or "").encode("utf8")
env["power_pass"] = (command["power"].get("passwd") or "").encode("utf8")
env["power_mode"] = command["action"].encode("utf8")
power_mapping = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice rewrite!

"address": "power_address",
"id": "power_id",
"user": "power_user",
"passwd": "power_pass",
}

for k, v in six.iteritems(power_mapping):
env[v] = _decode(command["power"].get(k, ""))

env["power_mode"] = _decode(command["action"])

return env


Expand Down
54 changes: 54 additions & 0 deletions LabController/src/bkr/labcontroller/test_provision.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Copyright Contributors to the Beaker project.
# SPDX-License-Identifier: GPL-2.0-or-later

import unittest

import six

from bkr.common.helpers import SensitiveUnicode
from bkr.labcontroller.provision import build_power_env


class TestBuildPowerEnv(unittest.TestCase):
def test_build_power_env(self):
t_command = {
"power": {
"address": u"192.168.1.1",
"id": u"42",
"user": u"root",
"passwd": SensitiveUnicode(u"toor"),
},
"action": u"reboot",
}

expected = {
"power_address": "192.168.1.1",
"power_id": "42",
"power_user": "root",
"power_pass": "toor",
"power_mode": "reboot",
}

actual = build_power_env(t_command)

for key, value in six.iteritems(expected):
self.assertEqual(expected[key], actual[key])

def test_build_power_env_with_missing_fields(self):
t_command = {
"power": {"address": u"192.168.1.1", "passwd": SensitiveUnicode(u"toor")},
"action": u"reboot",
}

expected = {
"power_address": "192.168.1.1",
"power_id": "",
"power_user": "",
"power_pass": "toor",
"power_mode": "reboot",
}

actual = build_power_env(t_command)

for key, value in six.iteritems(expected):
self.assertEqual(expected[key], actual[key])
2 changes: 2 additions & 0 deletions beaker.spec
Original file line number Diff line number Diff line change
Expand Up @@ -230,11 +230,13 @@ BuildRequires: python3-gevent
BuildRequires: python3-lxml
BuildRequires: python3-werkzeug
BuildRequires: python3-psutil
BuildRequires: python3-daemon
%else
# python2-gevent112 is a special build created for labcontroller. It includes backports to ensure compatibility with py2.7.9 SSL backport.
BuildRequires: python2-gevent112
BuildRequires: python2-psutil
BuildRequires: python-lxml
BuildRequires: python-daemon
%endif

# Syslinux is only available on x86_64. This package is used to provide pxelinux.0, which is then copied to the TFTP directory.
Expand Down
Loading