Set sys.abiflags on Windows to make check free-threaded/debug builds easier

[XuehaiPan]

Feature or enhancement

Proposal:

Currently, the sys.abiflags attribute is only set on Unix systems. On Windows, there are no sys.abiflags set (i.e. hasattr(sys, 'abiflags') -> False). It is hard to tell if Python is in free-threaded or debug builds. Although developers can use something like sysconfig.get_config_var('Py_GIL_DISABLED'). It is not well documented and is not convenient.

cpython/Lib/test/test_sys.py

Lines 686 to 687 in 3afb639

	if not sys.platform.startswith('win'):
	self.assertIsInstance(sys.abiflags, str)

cpython/Lib/test/test_sys.py

Lines 1294 to 1297 in 3afb639

	@test.support.cpython_only
	@unittest.skipUnless(hasattr(sys, 'abiflags'), 'need sys.abiflags')
	def test_disable_gil_abi(self):
	self.assertEqual('t' in sys.abiflags, support.Py_GIL_DISABLED)

It would be nice if we could set ABIFLAGS in pyconfig.h and enable sys.abiflags on Windows. This would simplify much code in the wild world and also in the CPython repo.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-127405: Set sys.abiflags on Windows #127406

[zooba]

I'm not convinced this is non-impactful elsewhere. We would need to make sure that anywhere that uses something "defined" by sys.abiflags is actually using those flags (e.g. the debug flag doesn't currently appear in ABI tags, and so we may be breaking the definition by putting this attribute here).

The lack of documentation for experimental builds is somewhat deliberate right now. It's being documented outside of the main docs, as we don't want to confuse users with features that are not officially part of CPython. If you want more info about how to work with free-threaded builds, you'll have to find that (I don't remember where it is, but the discussion forums are a good place to start looking).

[XuehaiPan]

I'm not convinced this is non-impactful elsewhere. We would need to make sure that anywhere that uses something "defined" by sys.abiflags is actually using those flags

Here is the search result of the following statements on GitHub:

• getattr(sys, 'abiflags', ''): 5.8k matches https://github.com/search?q=getattr%28sys%2C+%27abiflags%27%2C+%27%27%29&type=code
• getattr(sys, "abiflags", ""): ~200 matches https://github.com/search?q=getattr%28sys%2C+%22abiflags%22%2C+%22%22%29&type=code

Issue on the cpython GitHub repo: Cross compiling issue on Windows

• sysconfig --generate-posix-vars fails because of AttributeError #103130

---

(e.g. the debug flag doesn't currently appear in ABI tags, and so we may be breaking the definition by putting this attribute here).

The debug flag is included in the ABI tags. Here is the result that the Python is configured with --with-pydebug.

$ python3 -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))'
.cpython-314td-darwin.so

d is included in the abiflags for debug builds: https://docs.python.org/3/using/configure.html#python-debug-build

3.3.8. Python Debug Build

A debug build is Python built with the --with-pydebug configure option.

Effects of a debug build:

• Display all warnings by default: the list of default warning filters is empty in the warnings module.
• Add d to sys.abiflags. <====
• Add sys.gettotalrefcount() function.
  ...

A fresh installation built with --with-pydebug --disable-gil:

$ python3 -VV
Python 3.14.0a2+ experimental free-threading build (heads/main:c46acd35888, Dec  3 2024, 01:17:32) [Clang 16.0.0 (clang-1600.0.26.4)]
$ python3 -c 'import sys; print(sys.abiflags)'
td
$ python3 -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))'
.cpython-314td-darwin.so

[zooba]

The debug flag is included in the ABI tags.

Not on Windows - we use a _d suffix on the name rather than putting it in the ABI (historical reasons, but not easily changed).

>>> sysconfig.get_config_var("EXT_SUFFIX")
'_d.cp314-win_amd64.pyd'

There are a number of differences like this between Windows and other platforms. It's really meaningless to test any of this on non-Windows and then assume it applies cross-platform, it really won't. (Also don't put too much faith in the docs. They're largely written by Linux developers for Linux developers, and only rarely branch out into other platforms.)

[XuehaiPan]

The debug flag is included in the ABI tags.

Not on Windows - we use a _d suffix on the name rather than putting it in the ABI (historical reasons, but not easily changed).

>>> sysconfig.get_config_var("EXT_SUFFIX")
'_d.cp314-win_amd64.pyd'

There are no ABIFLAGS in C and sys.abiflags in Python on Windows.

$ .\arm64\python3.14t_d.exe -c 'import pprint, sysconfig; pprint.pprint(sysconfig.get_config_vars())'
{'BINDIR': 'C:\\Users\\PanXuehai\\Projects\\cpython\\PCbuild\\arm64',
 'BINLIBDEST': 'C:\\Users\\PanXuehai\\Projects\\cpython\\Lib',
 'EXE': '.exe',
 'EXT_SUFFIX': '_d.cp314t-win_arm64.pyd',
 'INCLUDEPY': 'C:\\Users\\PanXuehai\\Projects\\cpython\\Include',
 'LDLIBRARY': 'python314t_d.dll',
 'implementation_lower': 'python',
 'installed_base': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'installed_platbase': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'platbase': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'platlibdir': 'DLLs',
 'prefix': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'projectbase': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'py_version': '3.14.0a2+',
 'py_version_nodot': '314',
 'py_version_nodot_plat': '314t-arm64',
 'py_version_short': '3.14',
 'srcdir': 'C:\\Users\\PanXuehai\\Projects\\cpython',
 'userbase': 'C:\\Users\\PanXuehai\\AppData\\Roaming\\Python'}

I wonder if adding something that did not exist before would break historical things. EXT_SUFFIX is calculated in C here which does not rely on ABIFLAGS (not exists):

cpython/Include/internal/pycore_importdl.h

Lines 106 to 133 in 6cf7794

	#ifdef MS_WINDOWS
	#include <windows.h>
	typedef FARPROC dl_funcptr;
	#ifdef _DEBUG
	# define PYD_DEBUG_SUFFIX "_d"
	#else
	# define PYD_DEBUG_SUFFIX ""
	#endif
	#ifdef Py_GIL_DISABLED
	# define PYD_THREADING_TAG "t"
	#else
	# define PYD_THREADING_TAG ""
	#endif
	#ifdef PYD_PLATFORM_TAG
	# define PYD_SOABI "cp" Py_STRINGIFY(PY_MAJOR_VERSION) Py_STRINGIFY(PY_MINOR_VERSION) PYD_THREADING_TAG "-" PYD_PLATFORM_TAG
	#else
	# define PYD_SOABI "cp" Py_STRINGIFY(PY_MAJOR_VERSION) Py_STRINGIFY(PY_MINOR_VERSION) PYD_THREADING_TAG
	#endif
	#define PYD_TAGGED_SUFFIX PYD_DEBUG_SUFFIX "." PYD_SOABI ".pyd"
	#define PYD_UNTAGGED_SUFFIX PYD_DEBUG_SUFFIX ".pyd"
	#else
	typedef void (*dl_funcptr)(void);
	#endif

[zooba]

I wonder if adding something that did not exist before would break historical things.

Yes, this is my concern, but I'm far more concerned about 3rd party code using sys.abiflags to calculate things that currently don't depend on ABIFLAGS and won't also be updated.

This seems much like the discussion we had about using cp3.10 instead of cp310, where eventually we decided to not make the change because it would break practically everyone who made any assumption about the tag at all for limited benefit.

Being able to add t here would be an actual benefit, but I'm not sure it's worth the risk of breakage for 2-3 versions (when it should go away again when we go back to a single ABI). If there are other ABI flags worth adding that would be permanent, we should consider it, but I'm not aware of any right now.

[effigies]

Being able to add t here would be an actual benefit, but I'm not sure it's worth the risk of breakage for 2-3 versions (when it should go away again when we go back to a single ABI). If there are other ABI flags worth adding that would be permanent, we should consider it, but I'm not aware of any right now.

sys.abiflags has been around since 3.2 and was empty from 3.8-3.12. It doesn't seem to be a problem to leave it as an empty string for years on end, and it provides a standard place to make a mark if there is ever another need for multiple ABIs.

For a concrete use-case, I'm trying to patch tox-gh-actions to detect free-threaded builds (ymyzk/tox-gh-actions#199), which need slightly different configurations from the regular GIL builds. It's breaking for Windows, and falling back to 3.13 for Windows just means I can't test free-threaded Python on Windows unless I find some other contortion. For the moment, I will probably parse EXT_SUFFIX, but that hardly seems less brittle than a sequence of character codes that may be empty.

[zooba]

sys.abiflags has been around since 3.2 and was empty from 3.8-3.12

On Windows, it has been absent for all of that time. Changing from AttributeError to a meaningful value is a significant change for existing code. We can only hope that they all assume that "absent" implies "empty string".

For a concrete use-case, I'm trying to patch tox-gh-actions to detect free-threaded builds

Does sysconfig.get_config_var("Py_GIL_DISABLED") == 1 not suffice?

[XuehaiPan]

We can only hope that they all assume that "absent" implies "empty string".

For Python 3.8-3.12, it is true for normal builds. However, for Python 3.13, we have two different builds (the normal build and the free-threading build). It is inappropriate to assume the "absent" implies "empty string" in Python 3.13 and onwards.

[zooba]

I'm talking about backwards compatibility, and how programs will change behaviour if we change our behaviour. As I've said a couple of times, that is my primary concern here.

If you don't have any reason why it's safe, then let's deprecate sys.abiflags being missing on Windows and in 3.16 we can add it back.

Until then, use sysconfig to find out if CPython was built with free-threading enabled.

[effigies]

let's deprecate sys.abiflags being missing on Windows

I'm curious what this would mean in practice. Is this just a note in the docs?

[zooba]

Yeah, basically. The docs explicitly say it's only available on Unix, so it would get updated to indicate that it may be set on all platforms, and probably specify that a missing attribute is intended to be interpreted as an empty string. It may also need clarification that it isn't necessarily used the same on all platforms (it shouldn't be used to construct extension module suffixes, for example).

In Porting Notes for the next two releases we'd say that Windows is going to start setting it to an empty string rather than being absent in 3.16. No reason to actually use the word "deprecation" except to explain what process we're following for changing a user-visible and documented API.

[XuehaiPan]

In Porting Notes for the next two releases we'd say that Windows is going to start setting it to an empty string rather than being absent in 3.16.

Sounds reasonable to me.

[zooba]

The first step is the doc update to rewrite the abiflags doc to not be only defined using the old PEP, but to use some kind of definition that makes sense on all platforms. Don't change the availability yet.