Loading shared library inside python module being tested #10732
-
|
Hi All, I'm using pytest to validate a python + C/C++ library that I'm developing. The C/C++ library is compiled as a shared object .so and is being loaded in python using ctypes (CDLL). When the test finishes, as part of the teardown I'm using dlclose() and deleting the shared library reference in python (see: https://stackoverflow.com/questions/359498/how-can-i-unload-a-dll-using-ctypes-in-python). The problem is that the shared object doesn't unload and when I try to access it from a second test it reuses an already used structures and fails. I know that I can cleanup the shared object to re-init cleanly, but it's a giant project and I also don't think it's the cleanest approach - I want to test a clean new run. I did try using the --forked option (pytest plugin) and it does seems to solve the problem, but it comes with a new issue - which is the -s option is ignored and I can't see any of the tests prints (which make it more difficult to debug the tests). I'm also not sure it would work together with the pytest-dependecy plugin (even if I don't use the -n distributed option). So here are a few questions:
Any other ideas? Thanks, |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hi, I can answer some of the questions: 1)- pytest does not spawn a new process for each test, they are executed sequentially in the same process -- that is by design, as many test suites and features rely on this (for example sharing fixtures between tests). 3)- Not at the moment, that's a problem with xdist's implementation -- see here: https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html#output-stdout-and-stderr-from-workers 4)- Not sure what you mean with "work with dependencies", but there's no guarantee that the tests will run in the order of the parametrization, as pytest-forked is an extension of pytest-xdist which does not guarantee it -- pytest-xdist is designed to speed up the test run, so it will split the tests in chunks, and send those chunks to workers as they finish running their allocated tests, so it cannot guarantee order in that scenario. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! I guess the best option for me is to make sure the shared object is going through re-init successfully. |
Beta Was this translation helpful? Give feedback.
Hi,
I can answer some of the questions:
1)- pytest does not spawn a new process for each test, they are executed sequentially in the same process -- that is by design, as many test suites and features rely on this (for example sharing fixtures between tests).
3)- Not at the moment, that's a problem with xdist's implementation -- see here: https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html#output-stdout-and-stderr-from-workers
4)- Not sure what you mean with "work with dependencies", but there's no guarantee that the tests will run in the order of the parametrization, as pytest-forked is an extension of pytest-xdist which does not guarantee it -- pytest-xdist is designed …