Python test debugging falls back to default launch configuration after running custom debug-test configuration once

[RLThomaz]

Type: Bug

Behaviour

Expected vs. Actual

[Expected] When debugging a test using a custom launch configuration that includes "purpose": [ "debug-test" ], I expect subsequent debugging sessions to continue using the same, custom launch configuration. [Actual] However, after debugging a test for the first time, any subsequent test debugging (for any test) will fall back to the default launch configuration, i.e., as if the custom launch configuration was not properly configured.

My own investigation

I have debugged the extension and followed the breadcrumbs when running through the steps to reproduce the issue, as written in the section below. Here are my findings:

When debugging a test, the custom launch configuration is read from a *.code-workspace file by invoking getConfiguration in the code line below:

vscode-python/src/client/debugger/extension/configuration/launch.json/launchJsonReader.ts

Line 15 in eb96141

const codeWorkspaceConfig = getConfiguration('launch');

When running the debugger for the first time, the call to getConfiguration will return the correct launch configuration found in the .code-workspace file (e.g., the one in the section below). Looking at the variables panel, we can see that the first time getConfiguration is called, it returns everything correctly.

However, the second time it's invoked, the field purpose is empty and some other fields show up:

This means that, the following if branch in readDebugConfig will always be false, thus returning undefined since cfg.request === 'test' and (cfg as LaunchRequestArguments).purpose?.includes(DebugPurpose.DebugTest)) will always be false:

vscode-python/src/client/testing/common/debugLauncher.ts

Lines 117 to 129 in eb96141

	const configs = await getConfigurationsForWorkspace(workspaceFolder);
	for (const cfg of configs) {
	if (
	cfg.name &&
	cfg.type === DebuggerTypeName &&
	(cfg.request === 'test' ||
	(cfg as LaunchRequestArguments).purpose?.includes(DebugPurpose.DebugTest))
	) {
	// Return the first one.
	return cfg as LaunchRequestArguments;
	}
	}
	return undefined;

Since readDebugConfig returns undefined, the next code executed (below) will create a default debugConfig in getLaunchArgs. This explains why the second run uses the default launch configuration.

vscode-python/src/client/testing/common/debugLauncher.ts

Lines 78 to 85 in eb96141

	if (!debugConfig) {
	debugConfig = {
	name: 'Debug Unit Test',
	type: 'python',
	request: 'test',
	subProcess: true,
	};
	}

Steps to reproduce:

1. Create a custom launch configuration to debug Python tests.

    "launch": {
        "version": "0.2.0",
        "configurations": [
			{
				"name": "Python: Debug Tests",
				"type": "python",
				"request": "launch",
				"program": "${file}",
				"purpose": [
					"debug-test"
				],
				"console": "internalConsole",
				"env": {
					"PYTEST_ADDOPTS": "--no-cov"
				},
				"justMyCode": false,
				"autoReload": {
					"enable": true
				},
			},
        ],
        "compounds": []
	},

2. Open any Python project that contains tests to run (or create anything simple using pytest).
3. Add a breakpoint inside the unit-test.
4. Debug the unit-test, either by right-clicking on the "Run button" next to the test, and selecting "Debug" or by clicking the "Debug" button on the "Tests" tab.
5. Watch as the breakpoint is reached, and continue running the test.
6. Repeat steps 3 to 5, but now watch the breakpoint being skipped.
7. Open the DEBUG CONSOLE and observe the windows available (see image below), there should be two: Python: Debug Tests, i.e., the name of our custom configuration, and Debug Unit Test, the default launch configuration in VSCode.

8. Observe that the output for Python in the Output panel suggests that both debugging sessions used the custom launch configuration Using launch configuration in workspace folder.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.18
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Unknown
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-11-26 09:54:21.654 [info] Using launch configuration in workspace folder.
2023-11-26 09:54:22.632 [info] DAP Server launched with command: /local/home/rlthomaz/REDACTED/Python/farm/bin/python /local/home/rlthomaz/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter
2023-11-26 09:55:03.308 [info] Using launch configuration in workspace folder.
2023-11-26 09:55:04.145 [info] DAP Server launched with command: /local/home/rlthomaz/REDACTED/Python/farm/bin/python /local/home/rlthomaz/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Extension version: 2023.20.0
VS Code version: Code 1.84.2 (Universal) (1a5daa3a0231a0fbba4f14db7ec463cf99d7768e, 2023-11-09T10:52:33.687Z)
OS version: Darwin x64 22.6.0
Modes:
Remote OS version: Linux x64 5.4.259-180.361.amzn2int.x86_64
Remote OS version: Linux x64 5.4.259-180.361.amzn2int.x86_64
Remote OS version: Linux x64 5.4.259-180.361.amzn2int.x86_64

[RLThomaz]

@paulacamargo25, anyway I can help this ticket getting more traction? I may also be able to provide a PR if I get some extra pointers where to look for a root-cause :)

[taylermulligan]

Also experiencing this behaviour, although I can't even get my launch profile to register on the first launch. It seems to be ignored after the move to debugpy

[MichaelSquires]

I've been experiencing similar issues. I've found that I can get vscode to re-read and use the custom launch configuration by editing the launch configuration in the *.code-workspace file. I put a custom environment variable (DUMMY=1) that doesn't do anything in the configuration and I just comment/un-comment it as needed to update the file. Doing this resets something and causes vscode to use my custom launch config. It's frustrating having to do this though when it should just work.

I'm also happy to collaborate on getting this fixed.