Can pytest benefit from PEP 703 (making the GIL optional)?

[gabrieldemarmiesse]

Hi!

I'm Gabriel de Marmiesse and I'm helping Sam Gross to find out if maintainers of selected Python packages would benefit from PEP703.

If you don't know what is PEP 703, it's about making the global interpreter lock (GIL) optional.
Long story short, there would be a ~6% performance impact on single threaded programs, but would allow multithreaded programs to take full advantage of multiple cores.
You can play with it here.

If you have the time, we would like to know if the maintainers of pytest would benefit from PEP 703 or not. For example to collect tests faster, or to run multiple independant fixtures simultanously. I also know about pytest-xdist using processes but maybe isolation is a desired feature there so we wouldn't change anything. There are multiple situations pytest could be in.

1. pytest already uses threads and PEP 703 will likely make some features faster by allowing parallelization of cpu computation without any change.
2. pytest doesn't use threads, but PEP 703 would allow you to rewrite some features with multithreading, thus making it faster.
3. pytest uses multithreading with a low-level language (rust, C, C++...) or multiprocessing and using PEP 703 would allow a rewrite and lower the maintenance burden.
4. There is no computation that can be parallelized.
5. pytest wouldn't use multithreading, but it's very likely that users will call this package functions and classes from multiple threads because the workload can often be parallelized.

The Python language could never use multiple cores with multithreading, so it's hard to imagine what the future of the language would look like if we were to enable it. Feel free to tell us if you see new potential use cases or problems with this PEP. Thanks for your time!

If you want to know more about PEP 703:

• The PEP
• Python 3.9 implementation with install instructions
• Python 3.12 implementation
• Discussion about PEP 703
• Discussion about the multicore future of Python

[RonnyPfannschmidt]

Currently pytest is not affected

Pytest itself isn't using threads and the internals are not threadsafe