pytest with randomly via tox

[synarete]

Use tox to run pytest. Use pytest-randomly to have better control on random seed upon re-run due to test failure.

[phlogistonjohn]

LGTM generally, I'm just not entirely sure why "pytest" and "sanity" are separate envs, since there appears to be overlap between them. I approve, but would like to add a comment/documentation on what each env is meant to accomplish.

[synarete]

LGTM generally, I'm just not entirely sure why "pytest" and "sanity" are separate envs, since there appears to be overlap between them. I approve, but would like to add a comment/documentation on what each env is meant to accomplish.

It preserves the current existing logic in top-level Makefile (I did not want to do such cleanup in this PR).

[phlogistonjohn]

OK, thanks. I overlooked that was already the case. My general suggestion to document these different cases still stands, but not for this PR. :-)

[spuiuk]

retest this please

[spuiuk]

@synarete , we may need to install tox on the test system in sit-environment before we can commit this change.

From the console output of the test failure.

STDERR:

/bin/sh: tox: command not found
make: *** [Makefile:15: test] Error 127


MSG:

non-zero return code

[synarete]

@synarete , we may need to install tox on the test system in sit-environment before we can commit this change.

From the console output of the test failure.

STDERR:

/bin/sh: tox: command not found
make: *** [Makefile:15: test] Error 127


MSG:

non-zero return code

Indeed. For CentOS8 this is not an option due to old Python version. For CentOPS9-stream it should be fine. Do you think we should install it a rpm package on host or per user with python -m pip install --user tox?

[spuiuk]

/retest all

[spuiuk]

ACK - but one caveat. We introduce conftest.py but it is never populated. Should we hold on introducing the file until we actually want to add some pytest conf?

[spuiuk]

testcases/mount/test_mount.py::test_mount[192.168.123.11-gluster-vol-disperse] FAILED [ 87%]
testcases/mount/test_mount.py::test_mount[192.168.123.10-gluster-vol-disperse] FAILED [ 92%]
..

>       return strfunc(str(pathobj), *args)
E       OSError: [Errno 9] Bad file descriptor: '/tmp/11864/mnt_0/mount_test/test_io_consistency/0'

/usr/lib64/python3.6/pathlib.py:387: OSError

This is a known issue and not affected by the changes here.

The only difference is that the order in which the tests were run has changed. This is expected as part of the pytest-randomly use.

[spuiuk]

/retest all