Test discovery failure / regression for uniitest

[bigjools]

Behaviour

Similar to the issue reported here
#16593
and somewhat in here
#17641 (comment)

Test discovery is crashing (see output below).
This looks like it's the old problem of syspath being wrong and it is trying to import a local module called "logging" instead of the system one.
This appears to be a regression, is it was working previously and nothing else has changed for me other than upgrading vscode and extensions.

Steps to reproduce:

1. Create a repo with its own logging.py file
2. Configure unittest as test runner
3. Start the debugger and watch it fail to discover the tests

Diagnostic data

settings.json test section:

    "python.testing.unittestEnabled": true,
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        "helios/tests",
        "-t",
        "${workspaceFolder}",
        "-p",
        "test*.py"
    ],

Debugger output:

2024-10-21 10:38:59.984 [info] shell: bash
2024-10-21 10:39:00.014 [info] Discovering unittest tests for workspace /home/juledwar/src/helios with arguments: /home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter/discovery.py,--udiscovery,-v,-s,helios/tests,-t,/home/juledwar/src/helios,-p,test*.py

2024-10-21 10:39:00.014 [info] > ./.hatch/bin/python ~/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter/discovery.py --udiscovery -v -s helios/tests -t . -p test*.py
2024-10-21 10:39:00.014 [info] cwd: .
2024-10-21 10:39:00.230 [error] Unittest test discovery error for workspace:  /home/juledwar/src/helios 
 Failed to import test module: helios.tests.anchore.test_client
Traceback (most recent call last):
  File "/usr/lib/python3.12/unittest/loader.py", line 394, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/unittest/loader.py", line 337, in _get_module_from_name
    __import__(name)
  File "/home/juledwar/src/helios/helios/tests/anchore/test_client.py", line 5, in <module>
    from unittest import mock
  File "/usr/lib/python3.12/unittest/mock.py", line 26, in <module>
    import asyncio
  File "/usr/lib/python3.12/asyncio/__init__.py", line 8, in <module>
    from .base_events import *
  File "/usr/lib/python3.12/asyncio/base_events.py", line 18, in <module>
    import concurrent.futures
  File "/usr/lib/python3.12/concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 43, in <module>
    LOGGER = logging.getLogger("concurrent.futures")
             ^^^^^^^^^^^^^^^^^
AttributeError: module 'logging' has no attribute 'getLogger'

[eleanorjboyd]

what python version do you have? We haven't made any changes in this area recently (edit I see now it says 3.12)

[eleanorjboyd]

What happens if you try and run unittest from the command line with the same arguments you have in your settings? Can you also try looking at your python path variable from the terminal vs the vscode run? I am thinking something in your path might not be showing up right as you are looking for a custom logger file

[bigjools]

python3 -m unittest discover -v -s helios/tests -t . -p 'test*.py'
in the terminal works just fine.
Can you remind me how to work out what sys.path it would be using in its debugger runs?

Thanks.

[eleanorjboyd]

if you add

import sys
print(sys.path)

at the very top of your file, before it attempts this import Failed to import test module: helios.tests.anchore.test_client, which I think is caused by this import: File "/home/juledwar/src/helios/helios/tests/anchore/test_client.py", line 5, in <module> from unittest import mock

[bigjools]

That first module looks mighty suspicious.

2024-10-29 08:56:44.240 [info] ['helios', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter/lib/python', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter', '/home/juledwar/src/helios/.hatch/lib/python3.12/site-packages/_pdbpp_path_hack', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/home/juledwar/src/helios/.hatch/lib/python3.12/site-packages', '/home/juledwar/src/helios', '/home/juledwar/src/python-heliosclient', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/unittestadapter', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/lib/python', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files', '/home/juledwar/.vscode/extensions/ms-python.python-2024.16.1-linux-x64/python_files/lib/python']

[eleanorjboyd]

ok yes seems like we are adding your cwd to your path which is not the expected behavior. I will have to review why we started inserting it in the first place because I likely added it due to another bug in the opposite direction. Moving this back to the python extension as this is related to the testing infrastructure there more.

[eleanorjboyd]

@karthiknadig thoughts here? Not sure what is the best pick with the python path ordering since I had added the rootdir to the path to fix a previous issue: bd6a924

[bigjools]

It seems like you shouldn't really be inserting anything at the start of the path that can override the system modules.

[karthiknadig]

@eleanorjboyd typically we should be prefixing all modules that we have with vscode, this way there is no overlap with system modules. This is what we do in debugpy and pydevd