Test Discovery failing

[eleanorjboyd]

Details: Currently test discovery will "hang" with version 2024.22.1 if you have >30 tests in your workspace due to a buffer overflow issue. This should only occur for Mac and Linux machines.

Workaround: Downgrade your python extension version to 2024.20.*.

[brianoflondon]

If you don't have more than 30 tests in your workspace, are you even testing 🤪.

Merry Christmas, Happy Holidays, thanks for all you do, we'll survive till you fix it.

The way to downgrade: on the Extension page, click the drop down arrow on the Uninstall Button and you can "Install Specific Version...". Also make sure Auto Update is switched off or it will bounce you straight back to the broken one.

2024.22.0 seems to be working for me.

[jflam]

Just ran into this bug this AM. Confirming that downgrading is the correct interim solution for this.

[jbhannah]

The codebase that I encountered this with and reported in #24655 is pretty small, only 16 tests and (per cov) 258 total statements. Downgrading to 2024.22.0, or disabling the pythonTestAdapter experiment in 2024.22.1, fixed the issue for me (but disabling the experiment also took away the neat little coverage indicators 😔, so I've chosen to downgrade for now).

[WellingtonNico]

Same problem here, there is not error logs, never ends finding tests, but in the python test logs pytest collect almost finished

this is the image of the hanging

this is the logs when it collects successfully

[1m============================= test session starts ==============================�[0m
platform darwin -- Python 3.11.3, pytest-8.3.4, pluggy-1.5.0
rootdir: /Users/teste/projetos/pessoal/python/fastapi-todo-list
configfile: pytest.ini
plugins: asyncio-0.24.0, cov-6.0.0, anyio-4.7.0, xdist-3.6.1
asyncio: mode=Mode.AUTO, default_loop_scope=None
collected 1 item

<Dir fastapi-todo-list>
  <Package tests>
    <Module test_first.py>
      <Function test_first_test>

�[32m========================== �[32m1 test collected�[0m�[32m in 0.01s�[0m�[32m ===========================�[0m
Starting now, all test run output will be sent to the Test Result panel, while test discovery output will be sent to the "Python" output channel instead of the "Python Test Log" channel. The "Python Test Log" channel will be deprecated within the next month. See https://github.com/microsoft/vscode-python/wiki/New-Method-for-Output-Handling-in-Python-Testing for details.

this is the log for that project that does hang when collecting

you can notice there is a log missing at the end that exists in the above log: test collected

�[1m============================= test session starts ==============================�[0m
platform darwin -- Python 3.12.8, pytest-8.3.4, pluggy-1.5.0 -- /Users/teste/projetos/pessoal/python/fastapi-todo_list/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/teste/projetos/pessoal/python/fastapi-todo_list
configfile: pytest.ini
plugins: asyncio-0.24.0, cov-6.0.0, anyio-4.7.0, xdist-3.6.1
asyncio: mode=Mode.AUTO, default_loop_scope=None
�[1mcollecting ... �[0mcollected 17 items

<Dir fastapi-todo_list>
  <Package tests>
    <Dir auth>
      <Module test_auth_api.py>
        <Coroutine test_signup>
        <Coroutine test_login>
        <Coroutine test_me_with_todos>
        <Coroutine test_me_without_todos>
        <Coroutine test_logout_on_get_and_post>
    <Dir main>
      <Module test_main.py>
        <Coroutine test_healthy>
    <Dir todo>
      <Module test_todo_api.py>
        <Coroutine test_get_user_todos_security>
        <Coroutine test_get_user_todos>
        <Coroutine test_create_todo_security>
        <Coroutine test_create_todo>
        <Coroutine test_create_todo_title_validation>
        <Coroutine test_update_todo_security>
        <Coroutine test_update_todo>
        <Coroutine test_delete_todo_security>
        <Coroutine test_delete_todo>
        <Coroutine test_count_user_todos>
        <Coroutine test_get_todos_full>

I tried both with same python version and it does not work as well, both projects have the same dependencies, the only difference are the tests. In my Django projects the tests stopped working too, they were working just fine yesterday(2024/12/23), the same problem occurs, hangs when collecting the tests.

what I already tried:

• fresh install
• new user with totally fresh install and only the needed extensions(I thought it was gonna work)
• different python versions

I'm currently using macOS, latest version with m3 pro chip, I also made sure I'm using the correct build version for apple silicon.

update

inside devcontainer it finds the tests and also run, but I was not able to debug them like yesterday, the debug menu appears, I can even set the breakpoints but the code never pauses(and yes, I'm did debug tests inside dev container before and it was working).
I have just a few extensions inside dev container, more than when I tested with fresh instalation with new user, for me it looks like something maybe related to macos

what worked

I downgraded python, and debugpy extension version, the tests were found, for all projects, but I still was not able to debug them, accidentaly I noticed it worked for a new project that I created just to test this case, so I cloned my repo into a new folder and everything worked again, so I also tried to update the python extensions again and It broke again, :(

[patrickkidd]

I also had this issue with 2024.22.1 - same symptoms and output panel contents - and downgrading to 2024.22.0 fixes it for me. I am reproducing it by adding additional test cases or longer test case names like the following snippet which suggests a buffer overflow to me. Sometimes it freezes at different buffer lengths but usually within 100 chars of the example in the snippet:

def test_one():
    pass

def test_two():
    pass

def test_three():
    pass

def test_four():
    pass

def test_five():
    pass

def test_six():
    pass

def test_seven():
    pass

def test_eight():
    pass

def test_nine():
    pass

def test_ten():
    pass

def test_eleven():
    pass

def test_twelve():
    pass

def test_thirteen():
    pass

def test_fourteen():
    pass

def test_fifteen():
    pass

def test_sixteen():
    pass

def test_seventeen():
    pass

def test_eighteen():
    pass

def test_nineteen():
    pass

def test_twenty():
    pass

def test_twenty_one():
    pass

def test_twenty_two():
    pass

def test_twenty_three():
    pass

def test_twenty_four():
    pass

def test_twenty_five():
    pass

def test_twenty_six():
    pass

## Uncomment only one of the below examples at a time to reproduce the problem, at least on my machine.

## SUCCEEDS

# def test_twenty_seven():
#     pass


## SUCCEEDS

# def test_twenty_1234567891234567890123456789012345():
#     pass

## FREEZES

# def test_twenty_12345678912345678901234567890123456():
#     pass


## FREEZES

# def test_twenty_seven():
#     pass

# def test_twenty_nine():
#     pass

[BobKerns]

Probably redundant since it's been identified as a buffer overflow issue, but the specific place it hangs in the plugin is line 943 in __init__.py :

            request = f"""content-length: {len(data)}\ncontent-type: application/json\n\n{data}"""
            __writer.write(request) # HANGS HERE 😞

            __writer.flush()

This may be a repeat bug? I've had discovery suddenly start hanging like this in the past, but didn't suspect an update, and never got to track it this far. It may be worth a warning in the code on the other end, and a test case.

[BobKerns]

Maybe one symptom of a larger problem? I'm seeing similar hangs elsewhere:

• Markdown Preview
• Markdown Preview Enhanced
• Source Control

Source Control involves a large number of pending changes.
Markdown fails even with a small .md file, but perhaps prepended CSS, etc. makes it start off large?

[omriperl]

Any chance for a fix here soon?

[ricchyways]

Details

[dnwillia]

I can confirm that v2024.22.1 is hanging on multiple repos using pytest (eg: https://github.com/ansys/pyansys-units), and that downgrading to v2024.22.0 makes it go away.

[Montags24]

I can confirm that v2024.22.1 is hanging on multiple repos using pytest (eg: https://github.com/ansys/pyansys-units), and that downgrading to v2024.22.0 makes it go away.

Can confirm this works as well.

[sm373373]

Same issue, downgrading to v2024.22.0 fixed it for me

[elanou]

This happens with 'experiments' enabled in settings. Don't know why that is enabled by default...

[freerror]

I tried downgrading the python extension progressively but it did not resolve my error. However, when I switched to the pre-release version my test was discovered 🎉

Returning to the release version and disabling A/B experiments in the settings also seems to work.