diff --git a/.github/workflows/build_msi.yml b/.github/workflows/build_msi.yml index 29282dffa37ec0..65d32c734e7745 100644 --- a/.github/workflows/build_msi.yml +++ b/.github/workflows/build_msi.yml @@ -32,6 +32,8 @@ jobs: strategy: matrix: type: [x86, x64, arm64] + env: + IncludeFreethreaded: true steps: - uses: actions/checkout@v4 - name: Build CPython installer diff --git a/.github/workflows/reusable-windows.yml b/.github/workflows/reusable-windows.yml index ae27c108d8368c..c0209e0e1c92e9 100644 --- a/.github/workflows/reusable-windows.yml +++ b/.github/workflows/reusable-windows.yml @@ -20,7 +20,7 @@ jobs: - name: Display build info run: .\python.bat -m test.pythoninfo - name: Tests - run: .\PCbuild\rt.bat -p Win32 -d -q --fast-ci + run: .\PCbuild\rt.bat -p Win32 -d -q --fast-ci ${{ inputs.free-threading && '--disable-gil' || '' }} build_win_amd64: name: 'build and test (x64)' @@ -37,7 +37,7 @@ jobs: - name: Display build info run: .\python.bat -m test.pythoninfo - name: Tests - run: .\PCbuild\rt.bat -p x64 -d -q --fast-ci + run: .\PCbuild\rt.bat -p x64 -d -q --fast-ci ${{ inputs.free-threading && '--disable-gil' || '' }} build_win_arm64: name: 'build (arm64)' diff --git a/Doc/c-api/stable.rst b/Doc/c-api/stable.rst index 63a100a6f26f24..5b9e43874c7f2b 100644 --- a/Doc/c-api/stable.rst +++ b/Doc/c-api/stable.rst @@ -16,7 +16,7 @@ CPython's Application Binary Interface (ABI) is forward- and backwards-compatible across a minor release (if these are compiled the same way; see :ref:`stable-abi-platform` below). So, code compiled for Python 3.10.0 will work on 3.10.8 and vice versa, -but will need to be compiled separately for 3.9.x and 3.10.x. +but will need to be compiled separately for 3.9.x and 3.11.x. There are two tiers of C API with different stability expectations: diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst index e4bb6f5ec958c6..1bfcd69f72df2e 100644 --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -134,7 +134,7 @@ That's all you need to know to start making productive use of :mod:`doctest`! Jump in. The following sections provide full details. Note that there are many examples of doctests in the standard Python test suite and libraries. Especially useful examples can be found in the standard test file -:file:`Lib/test/test_doctest.py`. +:file:`Lib/test/test_doctest/test_doctest.py`. .. _doctest-simple-testmod: diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index f58d93da6ed687..adea067e082615 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -40,9 +40,9 @@ over the object tree. The :class:`EmailMessage` dictionary-like interface is indexed by the header names, which must be ASCII values. The values of the dictionary are strings with some extra methods. Headers are stored and returned in case-preserving -form, but field names are matched case-insensitively. Unlike a real dict, -there is an ordering to the keys, and there can be duplicate keys. Additional -methods are provided for working with headers that have duplicate keys. +form, but field names are matched case-insensitively. The keys are ordered, +but unlike a real dict, there can be duplicates. Addtional methods are +provided for working with headers that have duplicate keys. The *payload* is either a string or bytes object, in the case of simple message objects, or a list of :class:`EmailMessage` objects, for MIME container diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 20222bfb3611ab..07b15e23b2c10a 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -838,7 +838,7 @@ Utilities and Decorators * ``FIRST = auto()`` will work (auto() is replaced with ``1``); * ``SECOND = auto(), -2`` will work (auto is replaced with ``2``, so ``2, -2`` is - used to create the ``SECOND`` enum member; + used to create the ``SECOND`` enum member; * ``THREE = [auto(), -3]`` will *not* work (``, -3`` is used to create the ``THREE`` enum member) diff --git a/Doc/library/tkinter.ttk.rst b/Doc/library/tkinter.ttk.rst index 1609dc2ce9218e..bd0d8b3799a0f1 100644 --- a/Doc/library/tkinter.ttk.rst +++ b/Doc/library/tkinter.ttk.rst @@ -1118,7 +1118,7 @@ ttk.Treeview as the item identifier; *iid* must not already exist in the tree. Otherwise, a new unique identifier is generated. - See `Item Options`_ for the list of available points. + See `Item Options`_ for the list of available options. .. method:: item(item, option=None, **kw) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 3f6d5bfafee9d1..e543c1228d4d19 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -14,7 +14,7 @@ be used to describe syntax, not lexical analysis. When (one alternative of) a syntax rule has the form .. productionlist:: python-grammar - name: `othername` + name: othername and no semantics are given, the semantics of this form of ``name`` are the same as for ``othername``. @@ -422,7 +422,8 @@ Yield expressions .. productionlist:: python-grammar yield_atom: "(" `yield_expression` ")" - yield_expression: "yield" [`expression_list` | "from" `expression`] + yield_from: "yield" "from" `expression` + yield_expression: "yield" `expression_list` | `yield_from` The yield expression is used when defining a :term:`generator` function or an :term:`asynchronous generator` function and diff --git a/Doc/using/win_install_freethreaded.png b/Doc/using/win_install_freethreaded.png new file mode 100644 index 00000000000000..0aa01c1df6e051 Binary files /dev/null and b/Doc/using/win_install_freethreaded.png differ diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 598bf3ca9bcc04..2a0e7b4b06f586 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -307,6 +307,46 @@ settings and replace any that have been removed or modified. "Uninstall" will remove Python entirely, with the exception of the :ref:`launcher`, which has its own entry in Programs and Features. +.. _install-freethreaded-windows: + +Installing Free-threaded Binaries +--------------------------------- + +.. versionadded:: 3.13 (Experimental) + +.. note:: + + Everything described in this section is considered experimental, + and should be expected to change in future releases. + +To install pre-built binaries with free-threading enabled (see :pep:`703`), you +should select "Customize installation". The second page of options includes the +"Download free-threaded binaries" checkbox. + +.. image:: win_install_freethreaded.png + +Selecting this option will download and install additional binaries to the same +location as the main Python install. The main executable is called +``python3.13t.exe``, and other binaries either receive a ``t`` suffix or a full +ABI suffix. Python source files and bundled third-party dependencies are shared +with the main install. + +The free-threaded version is registered as a regular Python install with the +tag ``3.13t`` (with a ``-32`` or ``-arm64`` suffix as normal for those +platforms). This allows tools to discover it, and for the :ref:`launcher` to +support ``py.exe -3.13t``. Note that the launcher will interpret ``py.exe -3`` +(or a ``python3`` shebang) as "the latest 3.x install", which will prefer the +free-threaded binaries over the regular ones, while ``py.exe -3.13`` will not. +If you use the short style of option, you may prefer to not install the +free-threaded binaries at this time. + +To specify the install option at the command line, use +``Include_freethreaded=1``. See :ref:`install-layout-option` for instructions on +pre-emptively downloading the additional binaries for offline install. The +options to include debug symbols and binaries also apply to the free-threaded +builds. + +Free-threaded binaries are also available :ref:`on nuget.org `. .. _windows-store: @@ -450,9 +490,29 @@ automatically use the headers and import libraries in your build. The package information pages on nuget.org are `www.nuget.org/packages/python `_ -for the 64-bit version and `www.nuget.org/packages/pythonx86 -`_ for the 32-bit version. +for the 64-bit version, `www.nuget.org/packages/pythonx86 +`_ for the 32-bit version, and +`www.nuget.org/packages/pythonarm64 +`_ for the ARM64 version + +Free-threaded packages +---------------------- + +.. versionadded:: 3.13 (Experimental) + +.. note:: + Everything described in this section is considered experimental, + and should be expected to change in future releases. + +Packages containing free-threaded binaries are named +`python-freethreaded `_ +for the 64-bit version, `pythonx86-freethreaded +`_ for the 32-bit +version, and `pythonarm64-freethreaded +`_ for the ARM64 +version. These packages contain both the ``python3.13t.exe`` and +``python.exe`` entry points, both of which run free threaded. .. _windows-embeddable: diff --git a/Include/internal/pycore_freelist.h b/Include/internal/pycore_freelist.h index 566d47dbea11af..4ab93ee2bf6c32 100644 --- a/Include/internal/pycore_freelist.h +++ b/Include/internal/pycore_freelist.h @@ -19,12 +19,14 @@ extern "C" { # define PyList_MAXFREELIST 80 # define PyFloat_MAXFREELIST 100 # define PyContext_MAXFREELIST 255 +# define _PyAsyncGen_MAXFREELIST 80 #else # define PyTuple_NFREELISTS 0 # define PyTuple_MAXFREELIST 0 # define PyList_MAXFREELIST 0 # define PyFloat_MAXFREELIST 0 # define PyContext_MAXFREELIST 0 +# define _PyAsyncGen_MAXFREELIST 0 #endif struct _Py_list_state { @@ -77,12 +79,27 @@ struct _Py_context_state { #endif }; +struct _Py_async_gen_state { +#ifdef WITH_FREELISTS + /* Freelists boost performance 6-10%; they also reduce memory + fragmentation, as _PyAsyncGenWrappedValue and PyAsyncGenASend + are short-living objects that are instantiated for every + __anext__() call. */ + struct _PyAsyncGenWrappedValue* value_freelist[_PyAsyncGen_MAXFREELIST]; + int value_numfree; + + struct PyAsyncGenASend* asend_freelist[_PyAsyncGen_MAXFREELIST]; + int asend_numfree; +#endif +}; + typedef struct _Py_freelist_state { struct _Py_float_state float_state; struct _Py_tuple_state tuple_state; struct _Py_list_state list_state; struct _Py_slice_state slice_state; struct _Py_context_state context_state; + struct _Py_async_gen_state async_gen_state; } _PyFreeListState; #ifdef __cplusplus diff --git a/Include/internal/pycore_gc.h b/Include/internal/pycore_gc.h index 52e8b39ed0485d..d53de97709a782 100644 --- a/Include/internal/pycore_gc.h +++ b/Include/internal/pycore_gc.h @@ -251,7 +251,7 @@ extern void _PyFloat_ClearFreeList(_PyFreeListState *state, int is_finalization) extern void _PyList_ClearFreeList(_PyFreeListState *state, int is_finalization); extern void _PySlice_ClearCache(_PyFreeListState *state); extern void _PyDict_ClearFreeList(PyInterpreterState *interp); -extern void _PyAsyncGen_ClearFreeLists(PyInterpreterState *interp); +extern void _PyAsyncGen_ClearFreeLists(_PyFreeListState *state, int is_finalization); extern void _PyContext_ClearFreeList(_PyFreeListState *state, int is_finalization); extern void _Py_ScheduleGC(PyInterpreterState *interp); extern void _Py_RunGC(PyThreadState *tstate); diff --git a/Include/internal/pycore_genobject.h b/Include/internal/pycore_genobject.h index cf58a2750a31f9..5ad63658051e86 100644 --- a/Include/internal/pycore_genobject.h +++ b/Include/internal/pycore_genobject.h @@ -8,6 +8,8 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_freelist.h" + extern PyObject *_PyGen_yf(PyGenObject *); extern void _PyGen_Finalize(PyObject *self); @@ -26,34 +28,7 @@ extern PyTypeObject _PyAsyncGenAThrow_Type; /* runtime lifecycle */ -extern void _PyAsyncGen_Fini(PyInterpreterState *); - - -/* other API */ - -#ifndef WITH_FREELISTS -// without freelists -# define _PyAsyncGen_MAXFREELIST 0 -#endif - -#ifndef _PyAsyncGen_MAXFREELIST -# define _PyAsyncGen_MAXFREELIST 80 -#endif - -struct _Py_async_gen_state { -#if _PyAsyncGen_MAXFREELIST > 0 - /* Freelists boost performance 6-10%; they also reduce memory - fragmentation, as _PyAsyncGenWrappedValue and PyAsyncGenASend - are short-living objects that are instantiated for every - __anext__() call. */ - struct _PyAsyncGenWrappedValue* value_freelist[_PyAsyncGen_MAXFREELIST]; - int value_numfree; - - struct PyAsyncGenASend* asend_freelist[_PyAsyncGen_MAXFREELIST]; - int asend_numfree; -#endif -}; - +extern void _PyAsyncGen_Fini(_PyFreeListState *); #ifdef __cplusplus } diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index 7ec963005aba7e..922c84543a1393 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -190,7 +190,6 @@ struct _is { struct _Py_tuple_state tuple; struct _Py_dict_state dict_state; - struct _Py_async_gen_state async_gen; struct _Py_exc_state exc_state; struct ast_state ast; diff --git a/Lib/csv.py b/Lib/csv.py index 77f30c8d2b1f61..a079279b8b8cbc 100644 --- a/Lib/csv.py +++ b/Lib/csv.py @@ -1,28 +1,90 @@ -""" -csv.py - read/write/investigate CSV files +r""" +CSV parsing and writing. + +This module provides classes that assist in the reading and writing +of Comma Separated Value (CSV) files, and implements the interface +described by PEP 305. Although many CSV files are simple to parse, +the format is not formally defined by a stable specification and +is subtle enough that parsing lines of a CSV file with something +like line.split(",") is bound to fail. The module supports three +basic APIs: reading, writing, and registration of dialects. + + +DIALECT REGISTRATION: + +Readers and writers support a dialect argument, which is a convenient +handle on a group of settings. When the dialect argument is a string, +it identifies one of the dialects previously registered with the module. +If it is a class or instance, the attributes of the argument are used as +the settings for the reader or writer: + + class excel: + delimiter = ',' + quotechar = '"' + escapechar = None + doublequote = True + skipinitialspace = False + lineterminator = '\r\n' + quoting = QUOTE_MINIMAL + +SETTINGS: + + * quotechar - specifies a one-character string to use as the + quoting character. It defaults to '"'. + * delimiter - specifies a one-character string to use as the + field separator. It defaults to ','. + * skipinitialspace - specifies how to interpret spaces which + immediately follow a delimiter. It defaults to False, which + means that spaces immediately following a delimiter is part + of the following field. + * lineterminator - specifies the character sequence which should + terminate rows. + * quoting - controls when quotes should be generated by the writer. + It can take on any of the following module constants: + + csv.QUOTE_MINIMAL means only when required, for example, when a + field contains either the quotechar or the delimiter + csv.QUOTE_ALL means that quotes are always placed around fields. + csv.QUOTE_NONNUMERIC means that quotes are always placed around + fields which do not parse as integers or floating point + numbers. + csv.QUOTE_STRINGS means that quotes are always placed around + fields which are strings. Note that the Python value None + is not a string. + csv.QUOTE_NOTNULL means that quotes are only placed around fields + that are not the Python value None. + csv.QUOTE_NONE means that quotes are never placed around fields. + * escapechar - specifies a one-character string used to escape + the delimiter when quoting is set to QUOTE_NONE. + * doublequote - controls the handling of quotes inside fields. When + True, two consecutive quotes are interpreted as one during read, + and when writing, each quote character embedded in the data is + written as two quotes """ import re import types -from _csv import Error, __version__, writer, reader, register_dialect, \ +from _csv import Error, writer, reader, register_dialect, \ unregister_dialect, get_dialect, list_dialects, \ field_size_limit, \ QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE, \ - QUOTE_STRINGS, QUOTE_NOTNULL, \ - __doc__ + QUOTE_STRINGS, QUOTE_NOTNULL from _csv import Dialect as _Dialect from io import StringIO __all__ = ["QUOTE_MINIMAL", "QUOTE_ALL", "QUOTE_NONNUMERIC", "QUOTE_NONE", "QUOTE_STRINGS", "QUOTE_NOTNULL", - "Error", "Dialect", "__doc__", "excel", "excel_tab", + "Error", "Dialect", "excel", "excel_tab", "field_size_limit", "reader", "writer", "register_dialect", "get_dialect", "list_dialects", "Sniffer", - "unregister_dialect", "__version__", "DictReader", "DictWriter", + "unregister_dialect", "DictReader", "DictWriter", "unix_dialect"] +__version__ = "1.0" + + class Dialect: """Describe a CSV dialect. diff --git a/Lib/ctypes/_endian.py b/Lib/ctypes/_endian.py index 3febb3118b8230..6382dd22b8acc8 100644 --- a/Lib/ctypes/_endian.py +++ b/Lib/ctypes/_endian.py @@ -15,8 +15,8 @@ def _other_endian(typ): # if typ is array if isinstance(typ, _array_type): return _other_endian(typ._type_) * typ._length_ - # if typ is structure - if issubclass(typ, Structure): + # if typ is structure or union + if issubclass(typ, (Structure, Union)): return typ raise TypeError("This type does not support other endian: %s" % typ) diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 2fba32b5ffbc1e..3335821f3f32e9 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1332,58 +1332,69 @@ class C: def _asdict_inner(obj, dict_factory): - if type(obj) in _ATOMIC_TYPES: + obj_type = type(obj) + if obj_type in _ATOMIC_TYPES: return obj - elif _is_dataclass_instance(obj): - # fast path for the common case + elif hasattr(obj_type, _FIELDS): + # dataclass instance: fast path for the common case if dict_factory is dict: return { f.name: _asdict_inner(getattr(obj, f.name), dict) for f in fields(obj) } else: - result = [] - for f in fields(obj): - value = _asdict_inner(getattr(obj, f.name), dict_factory) - result.append((f.name, value)) - return dict_factory(result) - elif isinstance(obj, tuple) and hasattr(obj, '_fields'): - # obj is a namedtuple. Recurse into it, but the returned - # object is another namedtuple of the same type. This is - # similar to how other list- or tuple-derived classes are - # treated (see below), but we just need to create them - # differently because a namedtuple's __init__ needs to be - # called differently (see bpo-34363). - - # I'm not using namedtuple's _asdict() - # method, because: - # - it does not recurse in to the namedtuple fields and - # convert them to dicts (using dict_factory). - # - I don't actually want to return a dict here. The main - # use case here is json.dumps, and it handles converting - # namedtuples to lists. Admittedly we're losing some - # information here when we produce a json list instead of a - # dict. Note that if we returned dicts here instead of - # namedtuples, we could no longer call asdict() on a data - # structure where a namedtuple was used as a dict key. - - return type(obj)(*[_asdict_inner(v, dict_factory) for v in obj]) - elif isinstance(obj, (list, tuple)): - # Assume we can create an object of this type by passing in a - # generator (which is not true for namedtuples, handled - # above). - return type(obj)(_asdict_inner(v, dict_factory) for v in obj) - elif isinstance(obj, dict): - if hasattr(type(obj), 'default_factory'): + return dict_factory([ + (f.name, _asdict_inner(getattr(obj, f.name), dict_factory)) + for f in fields(obj) + ]) + # handle the builtin types first for speed; subclasses handled below + elif obj_type is list: + return [_asdict_inner(v, dict_factory) for v in obj] + elif obj_type is dict: + return { + _asdict_inner(k, dict_factory): _asdict_inner(v, dict_factory) + for k, v in obj.items() + } + elif obj_type is tuple: + return tuple([_asdict_inner(v, dict_factory) for v in obj]) + elif issubclass(obj_type, tuple): + if hasattr(obj, '_fields'): + # obj is a namedtuple. Recurse into it, but the returned + # object is another namedtuple of the same type. This is + # similar to how other list- or tuple-derived classes are + # treated (see below), but we just need to create them + # differently because a namedtuple's __init__ needs to be + # called differently (see bpo-34363). + + # I'm not using namedtuple's _asdict() + # method, because: + # - it does not recurse in to the namedtuple fields and + # convert them to dicts (using dict_factory). + # - I don't actually want to return a dict here. The main + # use case here is json.dumps, and it handles converting + # namedtuples to lists. Admittedly we're losing some + # information here when we produce a json list instead of a + # dict. Note that if we returned dicts here instead of + # namedtuples, we could no longer call asdict() on a data + # structure where a namedtuple was used as a dict key. + return obj_type(*[_asdict_inner(v, dict_factory) for v in obj]) + else: + return obj_type(_asdict_inner(v, dict_factory) for v in obj) + elif issubclass(obj_type, dict): + if hasattr(obj_type, 'default_factory'): # obj is a defaultdict, which has a different constructor from # dict as it requires the default_factory as its first arg. - result = type(obj)(getattr(obj, 'default_factory')) + result = obj_type(obj.default_factory) for k, v in obj.items(): result[_asdict_inner(k, dict_factory)] = _asdict_inner(v, dict_factory) return result - return type(obj)((_asdict_inner(k, dict_factory), - _asdict_inner(v, dict_factory)) - for k, v in obj.items()) + return obj_type((_asdict_inner(k, dict_factory), + _asdict_inner(v, dict_factory)) + for k, v in obj.items()) + elif issubclass(obj_type, list): + # Assume we can create an object of this type by passing in a + # generator + return obj_type(_asdict_inner(v, dict_factory) for v in obj) else: return copy.deepcopy(obj) @@ -1416,11 +1427,10 @@ def _astuple_inner(obj, tuple_factory): if type(obj) in _ATOMIC_TYPES: return obj elif _is_dataclass_instance(obj): - result = [] - for f in fields(obj): - value = _astuple_inner(getattr(obj, f.name), tuple_factory) - result.append(value) - return tuple_factory(result) + return tuple_factory([ + _astuple_inner(getattr(obj, f.name), tuple_factory) + for f in fields(obj) + ]) elif isinstance(obj, tuple) and hasattr(obj, '_fields'): # obj is a namedtuple. Recurse into it, but the returned # object is another namedtuple of the same type. This is @@ -1558,14 +1568,14 @@ class C: return _replace(obj, **changes) -def _replace(obj, /, **changes): +def _replace(self, /, **changes): # We're going to mutate 'changes', but that's okay because it's a - # new dict, even if called with 'replace(obj, **my_changes)'. + # new dict, even if called with 'replace(self, **my_changes)'. # It's an error to have init=False fields in 'changes'. - # If a field is not in 'changes', read its value from the provided obj. + # If a field is not in 'changes', read its value from the provided 'self'. - for f in getattr(obj, _FIELDS).values(): + for f in getattr(self, _FIELDS).values(): # Only consider normal fields or InitVars. if f._field_type is _FIELD_CLASSVAR: continue @@ -1582,11 +1592,11 @@ def _replace(obj, /, **changes): if f._field_type is _FIELD_INITVAR and f.default is MISSING: raise TypeError(f"InitVar {f.name!r} " f'must be specified with replace()') - changes[f.name] = getattr(obj, f.name) + changes[f.name] = getattr(self, f.name) # Create the new object, which calls __init__() and # __post_init__() (if defined), using all of the init fields we've # added and/or left in 'changes'. If there are values supplied in # changes that aren't fields, this will correctly raise a # TypeError. - return obj.__class__(**changes) + return self.__class__(**changes) diff --git a/Lib/idlelib/News3.txt b/Lib/idlelib/News3.txt index ee36fa7e1d5341..241b1f48e5c1d8 100644 --- a/Lib/idlelib/News3.txt +++ b/Lib/idlelib/News3.txt @@ -4,6 +4,8 @@ Released on 2024-10-xx ========================= +gh-96905: In idlelib code, stop redefining built-ins 'dict' and 'object'. + gh-72284: Improve the lists of features, editor key bindings, and shell key bingings in the IDLE doc. diff --git a/Lib/idlelib/debugger.py b/Lib/idlelib/debugger.py index f487b4c4b16a60..d90dbcd11f9f61 100644 --- a/Lib/idlelib/debugger.py +++ b/Lib/idlelib/debugger.py @@ -508,11 +508,11 @@ def show_source(self, index): class NamespaceViewer: "Global/local namespace viewer for debugger GUI." - def __init__(self, master, title, dict=None): + def __init__(self, master, title, odict=None): # XXX odict never passed. width = 0 height = 40 - if dict: - height = 20*len(dict) # XXX 20 == observed height of Entry widget + if odict: + height = 20*len(odict) # XXX 20 == observed height of Entry widget self.master = master self.title = title import reprlib @@ -533,24 +533,24 @@ def __init__(self, master, title, dict=None): canvas["yscrollcommand"] = vbar.set self.subframe = subframe = Frame(canvas) self.sfid = canvas.create_window(0, 0, window=subframe, anchor="nw") - self.load_dict(dict) + self.load_dict(odict) - dict = -1 + prev_odict = -1 # Needed for initial comparison below. - def load_dict(self, dict, force=0, rpc_client=None): - if dict is self.dict and not force: + def load_dict(self, odict, force=0, rpc_client=None): + if odict is self.prev_odict and not force: return subframe = self.subframe frame = self.frame for c in list(subframe.children.values()): c.destroy() - self.dict = None - if not dict: + self.prev_odict = None + if not odict: l = Label(subframe, text="None") l.grid(row=0, column=0) else: #names = sorted(dict) - ### + # # Because of (temporary) limitations on the dict_keys type (not yet # public or pickleable), have the subprocess to send a list of # keys, not a dict_keys object. sorted() will take a dict_keys @@ -560,12 +560,12 @@ def load_dict(self, dict, force=0, rpc_client=None): # interpreter gets into a loop requesting non-existing dict[0], # dict[1], dict[2], etc from the debugger_r.DictProxy. # TODO recheck above; see debugger_r 159ff, debugobj 60. - keys_list = dict.keys() + keys_list = odict.keys() names = sorted(keys_list) - ### + row = 0 for name in names: - value = dict[name] + value = odict[name] svalue = self.repr.repr(value) # repr(value) # Strip extra quotes caused by calling repr on the (already) # repr'd value sent across the RPC interface: @@ -577,7 +577,7 @@ def load_dict(self, dict, force=0, rpc_client=None): l.insert(0, svalue) l.grid(row=row, column=1, sticky="nw") row = row+1 - self.dict = dict + self.prev_odict = odict # XXX Could we use a callback for the following? subframe.update_idletasks() # Alas! width = subframe.winfo_reqwidth() diff --git a/Lib/idlelib/debugger_r.py b/Lib/idlelib/debugger_r.py index 26204438858d8a..ad3355d9f82765 100644 --- a/Lib/idlelib/debugger_r.py +++ b/Lib/idlelib/debugger_r.py @@ -125,16 +125,16 @@ def frame_attr(self, fid, name): def frame_globals(self, fid): frame = frametable[fid] - dict = frame.f_globals - did = id(dict) - dicttable[did] = dict + gdict = frame.f_globals + did = id(gdict) + dicttable[did] = gdict return did def frame_locals(self, fid): frame = frametable[fid] - dict = frame.f_locals - did = id(dict) - dicttable[did] = dict + ldict = frame.f_locals + did = id(ldict) + dicttable[did] = ldict return did def frame_code(self, fid): @@ -158,20 +158,17 @@ def code_filename(self, cid): def dict_keys(self, did): raise NotImplementedError("dict_keys not public or pickleable") -## dict = dicttable[did] -## return dict.keys() +## return dicttable[did].keys() - ### Needed until dict_keys is type is finished and pickealable. + ### Needed until dict_keys type is finished and pickleable. + # xxx finished. pickleable? ### Will probably need to extend rpc.py:SocketIO._proxify at that time. def dict_keys_list(self, did): - dict = dicttable[did] - return list(dict.keys()) + return list(dicttable[did].keys()) def dict_item(self, did, key): - dict = dicttable[did] - value = dict[key] - value = reprlib.repr(value) ### can't pickle module 'builtins' - return value + value = dicttable[did][key] + return reprlib.repr(value) # Can't pickle module 'builtins'. #----------end class IdbAdapter---------- diff --git a/Lib/idlelib/debugobj.py b/Lib/idlelib/debugobj.py index 156377f8ed26ac..fb448ece2fa25e 100644 --- a/Lib/idlelib/debugobj.py +++ b/Lib/idlelib/debugobj.py @@ -1,3 +1,5 @@ +"""Define tree items for debug stackviewer, which is only user. +""" # XXX TO DO: # - popup menu # - support partial or total redisplay @@ -17,9 +19,9 @@ myrepr.maxother = 100 class ObjectTreeItem(TreeItem): - def __init__(self, labeltext, object, setfunction=None): + def __init__(self, labeltext, object_, setfunction=None): self.labeltext = labeltext - self.object = object + self.object = object_ self.setfunction = setfunction def GetLabelText(self): return self.labeltext @@ -51,8 +53,8 @@ def GetSubList(self): item = make_objecttreeitem( str(key) + " =", value, - lambda value, key=key, object=self.object: - setattr(object, key, value)) + lambda value, key=key, object_=self.object: + setattr(object_, key, value)) sublist.append(item) return sublist @@ -85,8 +87,8 @@ def GetSubList(self): value = self.object[key] except KeyError: continue - def setfunction(value, key=key, object=self.object): - object[key] = value + def setfunction(value, key=key, object_=self.object): + object_[key] = value item = make_objecttreeitem(f"{key!r}:", value, setfunction) sublist.append(item) return sublist @@ -111,13 +113,13 @@ def keys(self): type: ClassTreeItem, } -def make_objecttreeitem(labeltext, object, setfunction=None): - t = type(object) +def make_objecttreeitem(labeltext, object_, setfunction=None): + t = type(object_) if t in dispatch: c = dispatch[t] else: c = ObjectTreeItem - return c(labeltext, object, setfunction) + return c(labeltext, object_, setfunction) def _debug_object_browser(parent): # htest # diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index 15e1ff3f3cf717..28c196a42672fc 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -79,6 +79,7 @@ class SB: __call__ = None tiptest(list.append, '(self, object, /)' + append_doc) tiptest(List.append, '(self, object, /)' + append_doc) tiptest([].append, '(object, /)' + append_doc) + # The use of 'object' above matches the signature text. tiptest(types.MethodType, '(function, instance, /)\n' diff --git a/Lib/idlelib/rpc.py b/Lib/idlelib/rpc.py index b08b80c9004551..3f0b2230dd185d 100644 --- a/Lib/idlelib/rpc.py +++ b/Lib/idlelib/rpc.py @@ -158,8 +158,8 @@ def debug(self, *args): s = s + " " + str(a) print(s, file=sys.__stderr__) - def register(self, oid, object): - self.objtable[oid] = object + def register(self, oid, object_): + self.objtable[oid] = object_ def unregister(self, oid): try: diff --git a/Lib/idlelib/stackviewer.py b/Lib/idlelib/stackviewer.py index 977c56ef15f2ae..95042d4debdc03 100644 --- a/Lib/idlelib/stackviewer.py +++ b/Lib/idlelib/stackviewer.py @@ -106,8 +106,8 @@ def GetSubList(self): value = self.object[key] except KeyError: continue - def setfunction(value, key=key, object=self.object): - object[key] = value + def setfunction(value, key=key, object_=self.object): + object_[key] = value item = make_objecttreeitem(key + " =", value, setfunction) sublist.append(item) return sublist diff --git a/Lib/subprocess.py b/Lib/subprocess.py index d5bd9a9e31aa04..20db7747d5db13 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1944,16 +1944,21 @@ def _execute_child(self, args, executable, preexec_fn, close_fds, SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) - child_exec_never_called = (err_msg == "noexec") - if child_exec_never_called: + if err_msg == "noexec:chdir": err_msg = "" # The error must be from chdir(cwd). err_filename = cwd + elif err_msg == "noexec": + err_msg = "" + err_filename = None else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) - raise child_exception_type(errno_num, err_msg, err_filename) + if err_filename is not None: + raise child_exception_type(errno_num, err_msg, err_filename) + else: + raise child_exception_type(errno_num, err_msg) raise child_exception_type(err_msg) diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 8e4e0765d46809..6a050fa541db1e 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -2693,12 +2693,17 @@ def test_make_pool(self): p.join() def test_terminate(self): + # Simulate slow tasks which take "forever" to complete + sleep_time = support.LONG_TIMEOUT + if self.TYPE == 'threads': - self.skipTest("Threads cannot be terminated") + # Thread pool workers can't be forced to quit, so if the first + # task starts early enough, we will end up waiting for it. + # Sleep for a shorter time, so the test doesn't block. + sleep_time = 1 - # Simulate slow tasks which take "forever" to complete p = self.Pool(3) - args = [support.LONG_TIMEOUT for i in range(10_000)] + args = [sleep_time for i in range(10_000)] result = p.map_async(time.sleep, args, chunksize=1) p.terminate() p.join() diff --git a/Lib/test/libregrtest/findtests.py b/Lib/test/libregrtest/findtests.py index 78343775bc5b99..ee890b5b1db4cd 100644 --- a/Lib/test/libregrtest/findtests.py +++ b/Lib/test/libregrtest/findtests.py @@ -19,6 +19,7 @@ SPLITTESTDIRS: set[TestName] = { "test_asyncio", "test_concurrent_futures", + "test_doctests", "test_future_stmt", "test_gdb", "test_inspect", diff --git a/Lib/test/support/pty_helper.py b/Lib/test/support/pty_helper.py index 11037d22516448..6587fd40333c51 100644 --- a/Lib/test/support/pty_helper.py +++ b/Lib/test/support/pty_helper.py @@ -58,3 +58,23 @@ def terminate(proc): input = b"" # Stop writing if not input: sel.modify(master, selectors.EVENT_READ) + + +###################################################################### +## Fake stdin (for testing interactive debugging) +###################################################################### + +class FakeInput: + """ + A fake input stream for pdb's interactive debugger. Whenever a + line is read, print it (to simulate the user typing it), and then + return it. The set of lines to return is specified in the + constructor; they should not have trailing newlines. + """ + def __init__(self, lines): + self.lines = lines + + def readline(self): + line = self.lines.pop(0) + print(line) + return line + '\n' diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py index 282cff4344b77f..36da86e6a2c622 100644 --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -1416,8 +1416,7 @@ def test_ordered_dict_reader(self): class MiscTestCase(unittest.TestCase): def test__all__(self): - extra = {'__doc__', '__version__'} - support.check__all__(self, csv, ('csv', '_csv'), extra=extra) + support.check__all__(self, csv, ('csv', '_csv')) def test_subclassable(self): # issue 44089 diff --git a/Lib/test/test_ctypes/test_byteswap.py b/Lib/test/test_ctypes/test_byteswap.py index b97b57646ecd71..78eff0392c4548 100644 --- a/Lib/test/test_ctypes/test_byteswap.py +++ b/Lib/test/test_ctypes/test_byteswap.py @@ -363,6 +363,24 @@ class TestUnion(parent): self.assertEqual(s.point.x, 1) self.assertEqual(s.point.y, 2) + def test_build_struct_union_opposite_system_byteorder(self): + # gh-105102 + if sys.byteorder == "little": + _Structure = BigEndianStructure + _Union = BigEndianUnion + else: + _Structure = LittleEndianStructure + _Union = LittleEndianUnion + + class S1(_Structure): + _fields_ = [("a", c_byte), ("b", c_byte)] + + class U1(_Union): + _fields_ = [("s1", S1), ("ab", c_short)] + + class S2(_Structure): + _fields_ = [("u1", U1), ("c", c_byte)] + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_ctypes/test_loading.py b/Lib/test/test_ctypes/test_loading.py index 22db97b818c17e..59d7f51935f3cd 100644 --- a/Lib/test/test_ctypes/test_loading.py +++ b/Lib/test/test_ctypes/test_loading.py @@ -141,7 +141,7 @@ def test_load_hasattr(self): def test_load_dll_with_flags(self): _sqlite3 = import_helper.import_module("_sqlite3") src = _sqlite3.__file__ - if src.lower().endswith("_d.pyd"): + if os.path.basename(src).partition(".")[0].lower().endswith("_d"): ext = "_d.dll" else: ext = ".dll" diff --git a/Lib/test/test_doctest/__init__.py b/Lib/test/test_doctest/__init__.py new file mode 100644 index 00000000000000..4b16ecc31156a5 --- /dev/null +++ b/Lib/test/test_doctest/__init__.py @@ -0,0 +1,5 @@ +import os +from test.support import load_package_tests + +def load_tests(*args): + return load_package_tests(os.path.dirname(__file__), *args) diff --git a/Lib/test/doctest_aliases.py b/Lib/test/test_doctest/doctest_aliases.py similarity index 100% rename from Lib/test/doctest_aliases.py rename to Lib/test/test_doctest/doctest_aliases.py diff --git a/Lib/test/doctest_lineno.py b/Lib/test/test_doctest/doctest_lineno.py similarity index 100% rename from Lib/test/doctest_lineno.py rename to Lib/test/test_doctest/doctest_lineno.py diff --git a/Lib/test/sample_doctest.py b/Lib/test/test_doctest/sample_doctest.py similarity index 91% rename from Lib/test/sample_doctest.py rename to Lib/test/test_doctest/sample_doctest.py index 89eb5cb7cf1d97..049f737a0a44ac 100644 --- a/Lib/test/sample_doctest.py +++ b/Lib/test/test_doctest/sample_doctest.py @@ -32,8 +32,8 @@ def bar(): def test_silly_setup(): """ - >>> import test.test_doctest - >>> test.test_doctest.sillySetup + >>> import test.test_doctest.test_doctest + >>> test.test_doctest.test_doctest.sillySetup True """ diff --git a/Lib/test/sample_doctest_no_docstrings.py b/Lib/test/test_doctest/sample_doctest_no_docstrings.py similarity index 100% rename from Lib/test/sample_doctest_no_docstrings.py rename to Lib/test/test_doctest/sample_doctest_no_docstrings.py diff --git a/Lib/test/sample_doctest_no_doctests.py b/Lib/test/test_doctest/sample_doctest_no_doctests.py similarity index 100% rename from Lib/test/sample_doctest_no_doctests.py rename to Lib/test/test_doctest/sample_doctest_no_doctests.py diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest/test_doctest.py similarity index 94% rename from Lib/test/test_doctest.py rename to Lib/test/test_doctest/test_doctest.py index 46a51007f9644d..7015255db1f7f0 100644 --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest/test_doctest.py @@ -4,6 +4,7 @@ from test import support from test.support import import_helper +from test.support.pty_helper import FakeInput # used in doctests import doctest import functools import os @@ -77,6 +78,15 @@ def get(self): """ return self.val + def setter(self, val): + """ + >>> s = SampleClass(-5) + >>> s.setter(1) + >>> print(s.val) + 1 + """ + self.val = val + def a_staticmethod(v): """ >>> print(SampleClass.a_staticmethod(10)) @@ -95,7 +105,7 @@ def a_classmethod(cls, v): return v+2 a_classmethod = classmethod(a_classmethod) - a_property = property(get, doc=""" + a_property = property(get, setter, doc=""" >>> print(SampleClass(22).a_property) 22 """) @@ -156,25 +166,6 @@ def get(self): """ return self.val -###################################################################### -## Fake stdin (for testing interactive debugging) -###################################################################### - -class _FakeInput: - """ - A fake input stream for pdb's interactive debugger. Whenever a - line is read, print it (to simulate the user typing it), and then - return it. The set of lines to return is specified in the - constructor; they should not have trailing newlines. - """ - def __init__(self, lines): - self.lines = lines - - def readline(self): - line = self.lines.pop(0) - print(line) - return line+'\n' - ###################################################################### ## Test Cases ###################################################################### @@ -468,14 +459,14 @@ def basics(): r""" We'll simulate a __file__ attr that ends in pyc: - >>> import test.test_doctest - >>> old = test.test_doctest.__file__ - >>> test.test_doctest.__file__ = 'test_doctest.pyc' + >>> from test.test_doctest import test_doctest + >>> old = test_doctest.__file__ + >>> test_doctest.__file__ = 'test_doctest.pyc' >>> tests = finder.find(sample_func) >>> print(tests) # doctest: +ELLIPSIS - [] + [] The exact name depends on how test_doctest was invoked, so allow for leading path components. @@ -483,7 +474,7 @@ def basics(): r""" >>> tests[0].filename # doctest: +ELLIPSIS '...test_doctest.py' - >>> test.test_doctest.__file__ = old + >>> test_doctest.__file__ = old >>> e = tests[0].examples[0] @@ -537,6 +528,7 @@ def basics(): r""" 1 SampleClass.a_staticmethod 1 SampleClass.double 1 SampleClass.get + 3 SampleClass.setter New-style classes are also supported: @@ -576,10 +568,10 @@ def basics(): r""" ... 'c': triple}}) >>> finder = doctest.DocTestFinder() - >>> # Use module=test.test_doctest, to prevent doctest from + >>> # Use module=test_doctest, to prevent doctest from >>> # ignoring the objects since they weren't defined in m. - >>> import test.test_doctest - >>> tests = finder.find(m, module=test.test_doctest) + >>> from test.test_doctest import test_doctest + >>> tests = finder.find(m, module=test_doctest) >>> for t in tests: ... print('%2s %s' % (len(t.examples), t.name)) 1 some_module @@ -593,23 +585,38 @@ def basics(): r""" 1 some_module.SampleClass.a_staticmethod 1 some_module.SampleClass.double 1 some_module.SampleClass.get + 3 some_module.SampleClass.setter 1 some_module.__test__.c 2 some_module.__test__.d 1 some_module.sample_func +However, doctest will ignore imported objects from other modules +(without proper `module=`): + + >>> import types + >>> m = types.ModuleType('poluted_namespace') + >>> m.__dict__.update({ + ... 'sample_func': sample_func, + ... 'SampleClass': SampleClass, + ... }) + + >>> finder = doctest.DocTestFinder() + >>> finder.find(m) + [] + Duplicate Removal ~~~~~~~~~~~~~~~~~ If a single object is listed twice (under different names), then tests will only be generated for it once: - >>> from test import doctest_aliases + >>> from test.test_doctest import doctest_aliases >>> assert doctest_aliases.TwoNames.f >>> assert doctest_aliases.TwoNames.g >>> tests = excl_empty_finder.find(doctest_aliases) >>> print(len(tests)) 2 >>> print(tests[0].name) - test.doctest_aliases.TwoNames + test.test_doctest.doctest_aliases.TwoNames TwoNames.f and TwoNames.g are bound to the same object. We can't guess which will be found in doctest's traversal of @@ -635,6 +642,7 @@ def basics(): r""" 1 SampleClass.a_staticmethod 1 SampleClass.double 1 SampleClass.get + 3 SampleClass.setter By default, that excluded objects with no doctests. exclude_empty=False tells it to include (empty) tests for objects with no doctests. This feature @@ -656,28 +664,29 @@ def basics(): r""" 1 SampleClass.a_staticmethod 1 SampleClass.double 1 SampleClass.get + 3 SampleClass.setter When used with `exclude_empty=False` we are also interested in line numbers of doctests that are empty. It used to be broken for quite some time until `bpo-28249`. - >>> from test import doctest_lineno + >>> from test.test_doctest import doctest_lineno >>> tests = doctest.DocTestFinder(exclude_empty=False).find(doctest_lineno) >>> for t in tests: ... print('%5s %s' % (t.lineno, t.name)) - None test.doctest_lineno - 22 test.doctest_lineno.ClassWithDocstring - 30 test.doctest_lineno.ClassWithDoctest - None test.doctest_lineno.ClassWithoutDocstring - None test.doctest_lineno.MethodWrapper - 53 test.doctest_lineno.MethodWrapper.classmethod_with_doctest - 39 test.doctest_lineno.MethodWrapper.method_with_docstring - 45 test.doctest_lineno.MethodWrapper.method_with_doctest - None test.doctest_lineno.MethodWrapper.method_without_docstring - 61 test.doctest_lineno.MethodWrapper.property_with_doctest - 4 test.doctest_lineno.func_with_docstring - 12 test.doctest_lineno.func_with_doctest - None test.doctest_lineno.func_without_docstring + None test.test_doctest.doctest_lineno + 22 test.test_doctest.doctest_lineno.ClassWithDocstring + 30 test.test_doctest.doctest_lineno.ClassWithDoctest + None test.test_doctest.doctest_lineno.ClassWithoutDocstring + None test.test_doctest.doctest_lineno.MethodWrapper + 53 test.test_doctest.doctest_lineno.MethodWrapper.classmethod_with_doctest + 39 test.test_doctest.doctest_lineno.MethodWrapper.method_with_docstring + 45 test.test_doctest.doctest_lineno.MethodWrapper.method_with_doctest + None test.test_doctest.doctest_lineno.MethodWrapper.method_without_docstring + 61 test.test_doctest.doctest_lineno.MethodWrapper.property_with_doctest + 4 test.test_doctest.doctest_lineno.func_with_docstring + 12 test.test_doctest.doctest_lineno.func_with_doctest + None test.test_doctest.doctest_lineno.func_without_docstring Turning off Recursion ~~~~~~~~~~~~~~~~~~~~~ @@ -1924,9 +1933,9 @@ def test_testsource(): r""" example code is converted to regular Python code. The surrounding words and expected output are converted to comments: - >>> import test.test_doctest - >>> name = 'test.test_doctest.sample_func' - >>> print(doctest.testsource(test.test_doctest, name)) + >>> from test.test_doctest import test_doctest + >>> name = 'test.test_doctest.test_doctest.sample_func' + >>> print(doctest.testsource(test_doctest, name)) # Blah blah # print(sample_func(22)) @@ -1936,8 +1945,8 @@ def test_testsource(): r""" # Yee ha! - >>> name = 'test.test_doctest.SampleNewStyleClass' - >>> print(doctest.testsource(test.test_doctest, name)) + >>> name = 'test.test_doctest.test_doctest.SampleNewStyleClass' + >>> print(doctest.testsource(test_doctest, name)) print('1\n2\n3') # Expected: ## 1 @@ -1945,8 +1954,8 @@ def test_testsource(): r""" ## 3 - >>> name = 'test.test_doctest.SampleClass.a_classmethod' - >>> print(doctest.testsource(test.test_doctest, name)) + >>> name = 'test.test_doctest.test_doctest.SampleClass.a_classmethod' + >>> print(doctest.testsource(test_doctest, name)) print(SampleClass.a_classmethod(10)) # Expected: ## 12 @@ -1969,7 +1978,7 @@ def test_debug(): r""" Create some fake stdin input, to feed to the debugger: >>> real_stdin = sys.stdin - >>> sys.stdin = _FakeInput(['next', 'print(x)', 'continue']) + >>> sys.stdin = FakeInput(['next', 'print(x)', 'continue']) Run the debugger on the docstring, and then restore sys.stdin. @@ -2012,7 +2021,7 @@ def test_pdb_set_trace(): captures our debugger input: >>> real_stdin = sys.stdin - >>> sys.stdin = _FakeInput([ + >>> sys.stdin = FakeInput([ ... 'print(x)', # print data defined by the example ... 'continue', # stop debugging ... '']) @@ -2039,7 +2048,7 @@ def test_pdb_set_trace(): ... ''' >>> test = parser.get_doctest(doc, globals(), "foo-bar@baz", "foo-bar@baz.py", 0) >>> real_stdin = sys.stdin - >>> sys.stdin = _FakeInput([ + >>> sys.stdin = FakeInput([ ... 'print(y)', # print data defined in the function ... 'up', # out of function ... 'print(x)', # print data defined by the example @@ -2051,7 +2060,7 @@ def test_pdb_set_trace(): ... finally: ... sys.stdin = real_stdin --Return-- - > (3)calls_set_trace()->None + > (3)calls_set_trace()->None -> import pdb; pdb.set_trace() (Pdb) print(y) 2 @@ -2076,7 +2085,7 @@ def test_pdb_set_trace(): ... ''' >>> test = parser.get_doctest(doc, globals(), "foo-bar@baz", "foo-bar@baz.py", 0) >>> real_stdin = sys.stdin - >>> sys.stdin = _FakeInput([ + >>> sys.stdin = FakeInput([ ... 'list', # list source from example 2 ... 'next', # return from g() ... 'list', # list source from example 1 @@ -2148,7 +2157,7 @@ def test_pdb_set_trace_nested(): >>> runner = doctest.DocTestRunner(verbose=False) >>> test = parser.get_doctest(doc, globals(), "foo-bar@baz", "foo-bar@baz.py", 0) >>> real_stdin = sys.stdin - >>> sys.stdin = _FakeInput([ + >>> sys.stdin = FakeInput([ ... 'print(y)', # print data defined in the function ... 'step', 'step', 'step', 'step', 'step', 'step', 'print(z)', ... 'up', 'print(x)', @@ -2162,39 +2171,39 @@ def test_pdb_set_trace_nested(): ... finally: ... sys.stdin = real_stdin ... # doctest: +REPORT_NDIFF - > (5)calls_set_trace() + > (5)calls_set_trace() -> self.f1() (Pdb) print(y) 1 (Pdb) step --Call-- - > (7)f1() + > (7)f1() -> def f1(self): (Pdb) step - > (8)f1() + > (8)f1() -> x = 1 (Pdb) step - > (9)f1() + > (9)f1() -> self.f2() (Pdb) step --Call-- - > (11)f2() + > (11)f2() -> def f2(self): (Pdb) step - > (12)f2() + > (12)f2() -> z = 1 (Pdb) step - > (13)f2() + > (13)f2() -> z = 2 (Pdb) print(z) 1 (Pdb) up - > (9)f1() + > (9)f1() -> self.f2() (Pdb) print(x) 1 (Pdb) up - > (5)calls_set_trace() + > (5)calls_set_trace() -> self.f1() (Pdb) print(y) 1 @@ -2214,39 +2223,39 @@ def test_DocTestSuite(): by passing a module object: >>> import unittest - >>> import test.sample_doctest - >>> suite = doctest.DocTestSuite(test.sample_doctest) + >>> import test.test_doctest.sample_doctest + >>> suite = doctest.DocTestSuite(test.test_doctest.sample_doctest) >>> suite.run(unittest.TestResult()) We can also supply the module by name: - >>> suite = doctest.DocTestSuite('test.sample_doctest') + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest') >>> suite.run(unittest.TestResult()) The module need not contain any doctest examples: - >>> suite = doctest.DocTestSuite('test.sample_doctest_no_doctests') + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest_no_doctests') >>> suite.run(unittest.TestResult()) The module need not contain any docstrings either: - >>> suite = doctest.DocTestSuite('test.sample_doctest_no_docstrings') + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest_no_docstrings') >>> suite.run(unittest.TestResult()) We can use the current module: - >>> suite = test.sample_doctest.test_suite() + >>> suite = test.test_doctest.sample_doctest.test_suite() >>> suite.run(unittest.TestResult()) We can also provide a DocTestFinder: >>> finder = doctest.DocTestFinder() - >>> suite = doctest.DocTestSuite('test.sample_doctest', + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', ... test_finder=finder) >>> suite.run(unittest.TestResult()) @@ -2254,7 +2263,7 @@ def test_DocTestSuite(): The DocTestFinder need not return any tests: >>> finder = doctest.DocTestFinder() - >>> suite = doctest.DocTestSuite('test.sample_doctest_no_docstrings', + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest_no_docstrings', ... test_finder=finder) >>> suite.run(unittest.TestResult()) @@ -2263,14 +2272,14 @@ def test_DocTestSuite(): used instead of the module globals. Here we'll pass an empty globals, triggering an extra error: - >>> suite = doctest.DocTestSuite('test.sample_doctest', globs={}) + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', globs={}) >>> suite.run(unittest.TestResult()) Alternatively, we can provide extra globals. Here we'll make an error go away by providing an extra global variable: - >>> suite = doctest.DocTestSuite('test.sample_doctest', + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', ... extraglobs={'y': 1}) >>> suite.run(unittest.TestResult()) @@ -2278,7 +2287,7 @@ def test_DocTestSuite(): You can pass option flags. Here we'll cause an extra error by disabling the blank-line feature: - >>> suite = doctest.DocTestSuite('test.sample_doctest', + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', ... optionflags=doctest.DONT_ACCEPT_BLANKLINE) >>> suite.run(unittest.TestResult()) @@ -2286,27 +2295,27 @@ def test_DocTestSuite(): You can supply setUp and tearDown functions: >>> def setUp(t): - ... import test.test_doctest - ... test.test_doctest.sillySetup = True + ... from test.test_doctest import test_doctest + ... test_doctest.sillySetup = True >>> def tearDown(t): - ... import test.test_doctest - ... del test.test_doctest.sillySetup + ... from test.test_doctest import test_doctest + ... del test_doctest.sillySetup Here, we installed a silly variable that the test expects: - >>> suite = doctest.DocTestSuite('test.sample_doctest', + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', ... setUp=setUp, tearDown=tearDown) >>> suite.run(unittest.TestResult()) But the tearDown restores sanity: - >>> import test.test_doctest - >>> test.test_doctest.sillySetup + >>> from test.test_doctest import test_doctest + >>> test_doctest.sillySetup Traceback (most recent call last): ... - AttributeError: module 'test.test_doctest' has no attribute 'sillySetup' + AttributeError: module 'test.test_doctest.test_doctest' has no attribute 'sillySetup' The setUp and tearDown functions are passed test objects. Here we'll use the setUp function to supply the missing variable y: @@ -2314,7 +2323,7 @@ def test_DocTestSuite(): >>> def setUp(test): ... test.globs['y'] = 1 - >>> suite = doctest.DocTestSuite('test.sample_doctest', setUp=setUp) + >>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest', setUp=setUp) >>> suite.run(unittest.TestResult()) @@ -2345,7 +2354,7 @@ def test_DocFileSuite(): >>> suite = doctest.DocFileSuite('test_doctest.txt', ... 'test_doctest2.txt', ... 'test_doctest4.txt', - ... package='test') + ... package='test.test_doctest') >>> suite.run(unittest.TestResult()) @@ -2361,7 +2370,7 @@ def test_DocFileSuite(): ... suite = doctest.DocFileSuite('test_doctest.txt', ... 'test_doctest2.txt', ... 'test_doctest4.txt', - ... package='test') + ... package='test.test_doctest') ... suite.run(unittest.TestResult()) ... finally: ... if added_loader: @@ -2371,16 +2380,17 @@ def test_DocFileSuite(): '/' should be used as a path separator. It will be converted to a native separator at run time: - >>> suite = doctest.DocFileSuite('../test/test_doctest.txt') + >>> suite = doctest.DocFileSuite('../test_doctest/test_doctest.txt') >>> suite.run(unittest.TestResult()) If DocFileSuite is used from an interactive session, then files are resolved relative to the directory of sys.argv[0]: - >>> import types, os.path, test.test_doctest + >>> import types, os.path + >>> from test.test_doctest import test_doctest >>> save_argv = sys.argv - >>> sys.argv = [test.test_doctest.__file__] + >>> sys.argv = [test_doctest.__file__] >>> suite = doctest.DocFileSuite('test_doctest.txt', ... package=types.ModuleType('__main__')) >>> sys.argv = save_argv @@ -2390,7 +2400,7 @@ def test_DocFileSuite(): working directory): >>> # Get the absolute path of the test package. - >>> test_doctest_path = os.path.abspath(test.test_doctest.__file__) + >>> test_doctest_path = os.path.abspath(test_doctest.__file__) >>> test_pkg_path = os.path.split(test_doctest_path)[0] >>> # Use it to find the absolute path of test_doctest.txt. @@ -2430,12 +2440,12 @@ def test_DocFileSuite(): And, you can provide setUp and tearDown functions: >>> def setUp(t): - ... import test.test_doctest - ... test.test_doctest.sillySetup = True + ... from test.test_doctest import test_doctest + ... test_doctest.sillySetup = True >>> def tearDown(t): - ... import test.test_doctest - ... del test.test_doctest.sillySetup + ... from test.test_doctest import test_doctest + ... del test_doctest.sillySetup Here, we installed a silly variable that the test expects: @@ -2448,11 +2458,11 @@ def test_DocFileSuite(): But the tearDown restores sanity: - >>> import test.test_doctest - >>> test.test_doctest.sillySetup + >>> from test.test_doctest import test_doctest + >>> test_doctest.sillySetup Traceback (most recent call last): ... - AttributeError: module 'test.test_doctest' has no attribute 'sillySetup' + AttributeError: module 'test.test_doctest.test_doctest' has no attribute 'sillySetup' The setUp and tearDown functions are passed test objects. Here, we'll use a setUp function to set the favorite color in @@ -3212,8 +3222,8 @@ def test_run_doctestsuite_multiple_times(): http://bugs.python.org/issue9736 >>> import unittest - >>> import test.sample_doctest - >>> suite = doctest.DocTestSuite(test.sample_doctest) + >>> import test.test_doctest.sample_doctest + >>> suite = doctest.DocTestSuite(test.test_doctest.sample_doctest) >>> suite.run(unittest.TestResult()) >>> suite.run(unittest.TestResult()) @@ -3390,4 +3400,4 @@ def load_tests(loader, tests, pattern): if __name__ == '__main__': - unittest.main(module='test.test_doctest') + unittest.main(module='test.test_doctest.test_doctest') diff --git a/Lib/test/test_doctest.txt b/Lib/test/test_doctest/test_doctest.txt similarity index 100% rename from Lib/test/test_doctest.txt rename to Lib/test/test_doctest/test_doctest.txt diff --git a/Lib/test/test_doctest2.py b/Lib/test/test_doctest/test_doctest2.py similarity index 100% rename from Lib/test/test_doctest2.py rename to Lib/test/test_doctest/test_doctest2.py diff --git a/Lib/test/test_doctest2.txt b/Lib/test/test_doctest/test_doctest2.txt similarity index 77% rename from Lib/test/test_doctest2.txt rename to Lib/test/test_doctest/test_doctest2.txt index 2e14856c27d8b3..76dab94a9c0470 100644 --- a/Lib/test/test_doctest2.txt +++ b/Lib/test/test_doctest/test_doctest2.txt @@ -2,8 +2,8 @@ This is a sample doctest in a text file. In this example, we'll rely on some silly setup: - >>> import test.test_doctest - >>> test.test_doctest.sillySetup + >>> import test.test_doctest.test_doctest + >>> test.test_doctest.test_doctest.sillySetup True This test also has some (random) encoded (utf-8) unicode text: diff --git a/Lib/test/test_doctest3.txt b/Lib/test/test_doctest/test_doctest3.txt similarity index 100% rename from Lib/test/test_doctest3.txt rename to Lib/test/test_doctest/test_doctest3.txt diff --git a/Lib/test/test_doctest4.txt b/Lib/test/test_doctest/test_doctest4.txt similarity index 100% rename from Lib/test/test_doctest4.txt rename to Lib/test/test_doctest/test_doctest4.txt diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index f99d4ca204b5a7..d045739efa46b8 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3202,6 +3202,37 @@ class NTEnum(Enum): [TTuple(id=0, a=0, blist=[]), TTuple(id=1, a=2, blist=[4]), TTuple(id=2, a=4, blist=[0, 1, 2])], ) + self.assertRaises(AttributeError, getattr, NTEnum.NONE, 'id') + # + class NTCEnum(TTuple, Enum): + NONE = 0, 0, [] + A = 1, 2, [4] + B = 2, 4, [0, 1, 2] + self.assertEqual(repr(NTCEnum.NONE), "") + self.assertEqual(NTCEnum.NONE.value, TTuple(id=0, a=0, blist=[])) + self.assertEqual(NTCEnum.NONE.id, 0) + self.assertEqual(NTCEnum.A.a, 2) + self.assertEqual(NTCEnum.B.blist, [0, 1 ,2]) + self.assertEqual( + [x.value for x in NTCEnum], + [TTuple(id=0, a=0, blist=[]), TTuple(id=1, a=2, blist=[4]), TTuple(id=2, a=4, blist=[0, 1, 2])], + ) + # + class NTDEnum(Enum): + def __new__(cls, id, a, blist): + member = object.__new__(cls) + member.id = id + member.a = a + member.blist = blist + return member + NONE = TTuple(0, 0, []) + A = TTuple(1, 2, [4]) + B = TTuple(2, 4, [0, 1, 2]) + self.assertEqual(repr(NTDEnum.NONE), "") + self.assertEqual(NTDEnum.NONE.id, 0) + self.assertEqual(NTDEnum.A.a, 2) + self.assertEqual(NTDEnum.B.blist, [0, 1 ,2]) + def test_flag_with_custom_new(self): class FlagFromChar(IntFlag): def __new__(cls, c): diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py index 5545ee39d8e942..0bf55facad9fed 100644 --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -90,6 +90,7 @@ def test_basic(self): self.assertRaises(TypeError, int, 1, 12) + self.assertRaises(TypeError, int, "10", 2, 1) self.assertEqual(int('0o123', 0), 83) self.assertEqual(int('0x123', 16), 291) diff --git a/Lib/test/test_launcher.py b/Lib/test/test_launcher.py index bcd4ed63bf25a0..3da6173cfd3f13 100644 --- a/Lib/test/test_launcher.py +++ b/Lib/test/test_launcher.py @@ -19,8 +19,10 @@ PY_EXE = "py.exe" +DEBUG_BUILD = False if sys.executable.casefold().endswith("_d.exe".casefold()): PY_EXE = "py_d.exe" + DEBUG_BUILD = True # Registry data to create. On removal, everything beneath top-level names will # be deleted. @@ -232,7 +234,7 @@ def run_py(self, args, env=None, allow_fail=False, expect_returncode=0, argv=Non p.stdin.close() p.wait(10) out = p.stdout.read().decode("utf-8", "replace") - err = p.stderr.read().decode("ascii", "replace") + err = p.stderr.read().decode("ascii", "replace").replace("\uFFFD", "?") if p.returncode != expect_returncode and support.verbose and not allow_fail: print("++ COMMAND ++") print([self.py_exe, *args]) @@ -273,7 +275,7 @@ def script(self, content, encoding="utf-8"): def fake_venv(self): venv = Path.cwd() / "Scripts" venv.mkdir(exist_ok=True, parents=True) - venv_exe = (venv / Path(sys.executable).name) + venv_exe = (venv / ("python_d.exe" if DEBUG_BUILD else "python.exe")) venv_exe.touch() try: yield venv_exe, {"VIRTUAL_ENV": str(venv.parent)} @@ -521,6 +523,9 @@ def test_virtualenv_in_list(self): self.assertEqual(str(venv_exe), m.group(1)) break else: + if support.verbose: + print(data["stdout"]) + print(data["stderr"]) self.fail("did not find active venv path") data = self.run_py(["-0"], env=env) @@ -616,25 +621,29 @@ def test_py_handle_64_in_ini(self): self.assertEqual("True", data["SearchInfo.oldStyleTag"]) def test_search_path(self): - stem = Path(sys.executable).stem + exe = Path("arbitrary-exe-name.exe").absolute() + exe.touch() + self.addCleanup(exe.unlink) with self.py_ini(TEST_PY_DEFAULTS): - with self.script(f"#! /usr/bin/env {stem} -prearg") as script: + with self.script(f"#! /usr/bin/env {exe.stem} -prearg") as script: data = self.run_py( [script, "-postarg"], - env={"PATH": f"{Path(sys.executable).parent};{os.getenv('PATH')}"}, + env={"PATH": f"{exe.parent};{os.getenv('PATH')}"}, ) - self.assertEqual(f"{sys.executable} -prearg {script} -postarg", data["stdout"].strip()) + self.assertEqual(f"{exe} -prearg {script} -postarg", data["stdout"].strip()) def test_search_path_exe(self): # Leave the .exe on the name to ensure we don't add it a second time - name = Path(sys.executable).name + exe = Path("arbitrary-exe-name.exe").absolute() + exe.touch() + self.addCleanup(exe.unlink) with self.py_ini(TEST_PY_DEFAULTS): - with self.script(f"#! /usr/bin/env {name} -prearg") as script: + with self.script(f"#! /usr/bin/env {exe.name} -prearg") as script: data = self.run_py( [script, "-postarg"], - env={"PATH": f"{Path(sys.executable).parent};{os.getenv('PATH')}"}, + env={"PATH": f"{exe.parent};{os.getenv('PATH')}"}, ) - self.assertEqual(f"{sys.executable} -prearg {script} -postarg", data["stdout"].strip()) + self.assertEqual(f"{exe} -prearg {script} -postarg", data["stdout"].strip()) def test_recursive_search_path(self): stem = self.get_py_exe().stem @@ -727,15 +736,18 @@ def test_shebang_command_in_venv(self): data = self.run_py([script], expect_returncode=103) with self.fake_venv() as (venv_exe, env): - # Put a real Python (ourselves) on PATH as a distraction. + # Put a "normal" Python on PATH as a distraction. # The active VIRTUAL_ENV should be preferred when the name isn't an # exact match. - env["PATH"] = f"{Path(sys.executable).parent};{os.environ['PATH']}" + exe = Path(Path(venv_exe).name).absolute() + exe.touch() + self.addCleanup(exe.unlink) + env["PATH"] = f"{exe.parent};{os.environ['PATH']}" with self.script(f'#! /usr/bin/env {stem} arg1') as script: data = self.run_py([script], env=env) self.assertEqual(data["stdout"].strip(), f"{venv_exe} arg1 {script}") - with self.script(f'#! /usr/bin/env {Path(sys.executable).stem} arg1') as script: + with self.script(f'#! /usr/bin/env {exe.stem} arg1') as script: data = self.run_py([script], env=env) - self.assertEqual(data["stdout"].strip(), f"{sys.executable} arg1 {script}") + self.assertEqual(data["stdout"].strip(), f"{exe} arg1 {script}") diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index 333d8a95dc4b7b..6f982ce37df692 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -16,9 +16,7 @@ from test import support from test.support import os_helper from test.support.import_helper import import_module -from test.support.pty_helper import run_pty -# This little helper class is essential for testing pdb under doctest. -from test.test_doctest import _FakeInput +from test.support.pty_helper import run_pty, FakeInput from unittest.mock import patch @@ -30,7 +28,7 @@ def __init__(self, input): def __enter__(self): self.real_stdin = sys.stdin - sys.stdin = _FakeInput(self.input) + sys.stdin = FakeInput(self.input) self.orig_trace = sys.gettrace() if hasattr(sys, 'gettrace') else None def __exit__(self, *exc): diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py index e828941f6c779d..89562fa5eac62c 100644 --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -845,6 +845,8 @@ def test_tools_buildbot_test(self): test_args.append('-x64') # 64-bit build if not support.Py_DEBUG: test_args.append('+d') # Release build, use python.exe + if sysconfig.get_config_var("Py_GIL_DISABLED"): + test_args.append('--disable-gil') self.run_batch(script, *test_args, *self.tests) @unittest.skipUnless(sys.platform == 'win32', 'Windows only') @@ -862,6 +864,8 @@ def test_pcbuild_rt(self): rt_args.append('-x64') # 64-bit build if support.Py_DEBUG: rt_args.append('-d') # Debug build, use python_d.exe + if sysconfig.get_config_var("Py_GIL_DISABLED"): + rt_args.append('--disable-gil') self.run_batch(script, *rt_args, *self.regrtest_args, *self.tests) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index 944a7de4210bc9..c44a778d5bbefe 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1991,9 +1991,9 @@ def test_process_group_0(self): @unittest.skipUnless(hasattr(os, 'setreuid'), 'no setreuid on platform') def test_user(self): - # For code coverage of the user parameter. We don't care if we get an - # EPERM error from it depending on the test execution environment, that - # still indicates that it was called. + # For code coverage of the user parameter. We don't care if we get a + # permission error from it depending on the test execution environment, + # that still indicates that it was called. uid = os.geteuid() test_users = [65534 if uid != 65534 else 65533, uid] @@ -2017,11 +2017,11 @@ def test_user(self): "import os; print(os.getuid())"], user=user, close_fds=close_fds) - except PermissionError: # (EACCES, EPERM) - pass - except OSError as e: - if e.errno not in (errno.EACCES, errno.EPERM): - raise + except PermissionError as e: # (EACCES, EPERM) + if e.errno == errno.EACCES: + self.assertEqual(e.filename, sys.executable) + else: + self.assertIsNone(e.filename) else: if isinstance(user, str): user_uid = pwd.getpwnam(user).pw_uid @@ -2065,8 +2065,8 @@ def test_group(self): "import os; print(os.getgid())"], group=group, close_fds=close_fds) - except PermissionError: # (EACCES, EPERM) - pass + except PermissionError as e: # (EACCES, EPERM) + self.assertIsNone(e.filename) else: if isinstance(group, str): group_gid = grp.getgrnam(group).gr_gid @@ -2114,7 +2114,8 @@ def _test_extra_groups_impl(self, *, gid, group_list): [sys.executable, "-c", "import os, sys, json; json.dump(os.getgroups(), sys.stdout)"], extra_groups=group_list) - except PermissionError: + except PermissionError as e: + self.assertIsNone(e.filename) self.skipTest("setgroup() EPERM; this test may require root.") else: parent_groups = os.getgroups() diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 218e7560cff3f5..6852625c36c62b 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -223,8 +223,14 @@ def test_prompt(self): def test_upgrade_dependencies(self): builder = venv.EnvBuilder() - bin_path = 'Scripts' if sys.platform == 'win32' else 'bin' + bin_path = 'bin' python_exe = os.path.split(sys.executable)[1] + if sys.platform == 'win32': + bin_path = 'Scripts' + if os.path.normcase(os.path.splitext(python_exe)[0]).endswith('_d'): + python_exe = 'python_d.exe' + else: + python_exe = 'python.exe' with tempfile.TemporaryDirectory() as fake_env_dir: expect_exe = os.path.normcase( os.path.join(fake_env_dir, bin_path, python_exe) @@ -283,7 +289,9 @@ def test_sysconfig(self): # build environment ('is_python_build()', str(sysconfig.is_python_build())), ('get_makefile_filename()', sysconfig.get_makefile_filename()), - ('get_config_h_filename()', sysconfig.get_config_h_filename())): + ('get_config_h_filename()', sysconfig.get_config_h_filename()), + ('get_config_var("Py_GIL_DISABLED")', + str(sysconfig.get_config_var("Py_GIL_DISABLED")))): with self.subTest(call): cmd[2] = 'import sysconfig; print(sysconfig.%s)' % call out, err = check_output(cmd, encoding='utf-8') @@ -315,7 +323,9 @@ def test_sysconfig_symlinks(self): # build environment ('is_python_build()', str(sysconfig.is_python_build())), ('get_makefile_filename()', sysconfig.get_makefile_filename()), - ('get_config_h_filename()', sysconfig.get_config_h_filename())): + ('get_config_h_filename()', sysconfig.get_config_h_filename()), + ('get_config_var("Py_GIL_DISABLED")', + str(sysconfig.get_config_var("Py_GIL_DISABLED")))): with self.subTest(call): cmd[2] = 'import sysconfig; print(sysconfig.%s)' % call out, err = check_output(cmd, encoding='utf-8') diff --git a/Lib/test/test_zipimport_support.py b/Lib/test/test_zipimport_support.py index 7bf50a33728e53..71039d2a8e7ab9 100644 --- a/Lib/test/test_zipimport_support.py +++ b/Lib/test/test_zipimport_support.py @@ -29,8 +29,9 @@ # test_cmd_line_script (covers the zipimport support in runpy) # Retrieve some helpers from other test cases -from test import (test_doctest, sample_doctest, sample_doctest_no_doctests, - sample_doctest_no_docstrings) +from test.test_doctest import (test_doctest, + sample_doctest, sample_doctest_no_doctests, + sample_doctest_no_docstrings) def _run_object_doctest(obj, module): @@ -100,18 +101,18 @@ def test_doctest_issue4197(self): # everything still works correctly test_src = inspect.getsource(test_doctest) test_src = test_src.replace( - "from test import test_doctest", + "from test.test_doctest import test_doctest", "import test_zipped_doctest as test_doctest") - test_src = test_src.replace("test.test_doctest", + test_src = test_src.replace("test.test_doctest.test_doctest", "test_zipped_doctest") - test_src = test_src.replace("test.sample_doctest", + test_src = test_src.replace("test.test_doctest.sample_doctest", "sample_zipped_doctest") # The sample doctest files rewritten to include in the zipped version. sample_sources = {} for mod in [sample_doctest, sample_doctest_no_doctests, sample_doctest_no_docstrings]: src = inspect.getsource(mod) - src = src.replace("test.test_doctest", "test_zipped_doctest") + src = src.replace("test.test_doctest.test_doctest", "test_zipped_doctest") # Rewrite the module name so that, for example, # "test.sample_doctest" becomes "sample_zipped_doctest". mod_name = mod.__name__.split(".")[-1] diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 2590acdc87e695..e0db41dd915ece 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -4715,7 +4715,7 @@ def panes(self): def _test(): root = Tk() - text = "This is Tcl/Tk version %s" % TclVersion + text = "This is Tcl/Tk %s" % root.globalgetvar('tk_patchLevel') text += "\nThis should be a cedilla: \xe7" label = Label(root, text=text) label.pack() diff --git a/Lib/trace.py b/Lib/trace.py index 7cb6f897634b14..7886959fa64f68 100755 --- a/Lib/trace.py +++ b/Lib/trace.py @@ -265,8 +265,7 @@ def write_results(self, show_missing=True, summary=False, coverdir=None, *, modulename = _modname(filename) else: dir = coverdir - if not os.path.exists(dir): - os.makedirs(dir) + os.makedirs(dir, exist_ok=True) modulename = _fullmodname(filename) # If desired, get a list of the line numbers which represent diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py index d960bf3bd82ac5..f04ca8fafcc33b 100644 --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -139,6 +139,11 @@ def create_if_needed(d): 'check that your PATH environment variable is ' 'correctly set.') dirname, exename = os.path.split(os.path.abspath(executable)) + if sys.platform == 'win32': + # Always create the simplest name in the venv. It will either be a + # link back to executable, or a copy of the appropriate launcher + _d = '_d' if os.path.splitext(exename)[0].endswith('_d') else '' + exename = f'python{_d}.exe' context.executable = executable context.python_dir = dirname context.python_exe = exename @@ -222,67 +227,26 @@ def create_configuration(self, context): args = ' '.join(args) f.write(f'command = {sys.executable} -m venv {args}\n') - if os.name != 'nt': - def symlink_or_copy(self, src, dst, relative_symlinks_ok=False): - """ - Try symlinking a file, and if that fails, fall back to copying. - """ - force_copy = not self.symlinks - if not force_copy: - try: - if not os.path.islink(dst): # can't link to itself! - if relative_symlinks_ok: - assert os.path.dirname(src) == os.path.dirname(dst) - os.symlink(os.path.basename(src), dst) - else: - os.symlink(src, dst) - except Exception: # may need to use a more specific exception - logger.warning('Unable to symlink %r to %r', src, dst) - force_copy = True - if force_copy: - shutil.copyfile(src, dst) - else: - def symlink_or_copy(self, src, dst, relative_symlinks_ok=False): - """ - Try symlinking a file, and if that fails, fall back to copying. - """ - bad_src = os.path.lexists(src) and not os.path.exists(src) - if self.symlinks and not bad_src and not os.path.islink(dst): - try: + def symlink_or_copy(self, src, dst, relative_symlinks_ok=False): + """ + Try symlinking a file, and if that fails, fall back to copying. + (Unused on Windows, because we can't just copy a failed symlink file: we + switch to a different set of files instead.) + """ + assert os.name != 'nt' + force_copy = not self.symlinks + if not force_copy: + try: + if not os.path.islink(dst): # can't link to itself! if relative_symlinks_ok: assert os.path.dirname(src) == os.path.dirname(dst) os.symlink(os.path.basename(src), dst) else: os.symlink(src, dst) - return - except Exception: # may need to use a more specific exception - logger.warning('Unable to symlink %r to %r', src, dst) - - # On Windows, we rewrite symlinks to our base python.exe into - # copies of venvlauncher.exe - basename, ext = os.path.splitext(os.path.basename(src)) - srcfn = os.path.join(os.path.dirname(__file__), - "scripts", - "nt", - basename + ext) - # Builds or venv's from builds need to remap source file - # locations, as we do not put them into Lib/venv/scripts - if sysconfig.is_python_build() or not os.path.isfile(srcfn): - if basename.endswith('_d'): - ext = '_d' + ext - basename = basename[:-2] - if basename == 'python': - basename = 'venvlauncher' - elif basename == 'pythonw': - basename = 'venvwlauncher' - src = os.path.join(os.path.dirname(src), basename + ext) - else: - src = srcfn - if not os.path.exists(src): - if not bad_src: - logger.warning('Unable to copy %r', src) - return - + except Exception: # may need to use a more specific exception + logger.warning('Unable to symlink %r to %r', src, dst) + force_copy = True + if force_copy: shutil.copyfile(src, dst) def create_git_ignore_file(self, context): @@ -298,22 +262,23 @@ def create_git_ignore_file(self, context): 'see https://docs.python.org/3/library/venv.html\n') file.write('*\n') - def setup_python(self, context): - """ - Set up a Python executable in the environment. + if os.name != 'nt': + def setup_python(self, context): + """ + Set up a Python executable in the environment. - :param context: The information for the environment creation request - being processed. - """ - binpath = context.bin_path - path = context.env_exe - copier = self.symlink_or_copy - dirname = context.python_dir - if os.name != 'nt': + :param context: The information for the environment creation request + being processed. + """ + binpath = context.bin_path + path = context.env_exe + copier = self.symlink_or_copy + dirname = context.python_dir copier(context.executable, path) if not os.path.islink(path): os.chmod(path, 0o755) - for suffix in ('python', 'python3', f'python3.{sys.version_info[1]}'): + for suffix in ('python', 'python3', + f'python3.{sys.version_info[1]}'): path = os.path.join(binpath, suffix) if not os.path.exists(path): # Issue 18807: make copies if @@ -321,30 +286,105 @@ def setup_python(self, context): copier(context.env_exe, path, relative_symlinks_ok=True) if not os.path.islink(path): os.chmod(path, 0o755) - else: - if self.symlinks: - # For symlinking, we need a complete copy of the root directory - # If symlinks fail, you'll get unnecessary copies of files, but - # we assume that if you've opted into symlinks on Windows then - # you know what you're doing. - suffixes = [ - f for f in os.listdir(dirname) if - os.path.normcase(os.path.splitext(f)[1]) in ('.exe', '.dll') - ] - if sysconfig.is_python_build(): - suffixes = [ - f for f in suffixes if - os.path.normcase(f).startswith(('python', 'vcruntime')) - ] + + else: + def setup_python(self, context): + """ + Set up a Python executable in the environment. + + :param context: The information for the environment creation request + being processed. + """ + binpath = context.bin_path + dirname = context.python_dir + exename = os.path.basename(context.env_exe) + exe_stem = os.path.splitext(exename)[0] + exe_d = '_d' if os.path.normcase(exe_stem).endswith('_d') else '' + if sysconfig.is_python_build(): + scripts = dirname + else: + scripts = os.path.join(os.path.dirname(__file__), + 'scripts', 'nt') + if not sysconfig.get_config_var("Py_GIL_DISABLED"): + python_exe = os.path.join(dirname, f'python{exe_d}.exe') + pythonw_exe = os.path.join(dirname, f'pythonw{exe_d}.exe') + link_sources = { + 'python.exe': python_exe, + f'python{exe_d}.exe': python_exe, + 'pythonw.exe': pythonw_exe, + f'pythonw{exe_d}.exe': pythonw_exe, + } + python_exe = os.path.join(scripts, f'venvlauncher{exe_d}.exe') + pythonw_exe = os.path.join(scripts, f'venvwlauncher{exe_d}.exe') + copy_sources = { + 'python.exe': python_exe, + f'python{exe_d}.exe': python_exe, + 'pythonw.exe': pythonw_exe, + f'pythonw{exe_d}.exe': pythonw_exe, + } else: - suffixes = {'python.exe', 'python_d.exe', 'pythonw.exe', 'pythonw_d.exe'} - base_exe = os.path.basename(context.env_exe) - suffixes.add(base_exe) + exe_t = f'3.{sys.version_info[1]}t' + python_exe = os.path.join(dirname, f'python{exe_t}{exe_d}.exe') + pythonw_exe = os.path.join(dirname, f'pythonw{exe_t}{exe_d}.exe') + link_sources = { + 'python.exe': python_exe, + f'python{exe_d}.exe': python_exe, + f'python{exe_t}.exe': python_exe, + f'python{exe_t}{exe_d}.exe': python_exe, + 'pythonw.exe': pythonw_exe, + f'pythonw{exe_d}.exe': pythonw_exe, + f'pythonw{exe_t}.exe': pythonw_exe, + f'pythonw{exe_t}{exe_d}.exe': pythonw_exe, + } + python_exe = os.path.join(scripts, f'venvlaunchert{exe_d}.exe') + pythonw_exe = os.path.join(scripts, f'venvwlaunchert{exe_d}.exe') + copy_sources = { + 'python.exe': python_exe, + f'python{exe_d}.exe': python_exe, + f'python{exe_t}.exe': python_exe, + f'python{exe_t}{exe_d}.exe': python_exe, + 'pythonw.exe': pythonw_exe, + f'pythonw{exe_d}.exe': pythonw_exe, + f'pythonw{exe_t}.exe': pythonw_exe, + f'pythonw{exe_t}{exe_d}.exe': pythonw_exe, + } + + do_copies = True + if self.symlinks: + do_copies = False + # For symlinking, we need all the DLLs to be available alongside + # the executables. + link_sources.update({ + f: os.path.join(dirname, f) for f in os.listdir(dirname) + if os.path.normcase(f).startswith(('python', 'vcruntime')) + and os.path.normcase(os.path.splitext(f)[1]) == '.dll' + }) + + to_unlink = [] + for dest, src in link_sources.items(): + dest = os.path.join(binpath, dest) + try: + os.symlink(src, dest) + to_unlink.append(dest) + except OSError: + logger.warning('Unable to symlink %r to %r', src, dst) + do_copies = True + for f in to_unlink: + try: + os.unlink(f) + except OSError: + logger.warning('Failed to clean up symlink %r', + f) + logger.warning('Retrying with copies') + break - for suffix in suffixes: - src = os.path.join(dirname, suffix) - if os.path.lexists(src): - copier(src, os.path.join(binpath, suffix)) + if do_copies: + for dest, src in copy_sources.items(): + dest = os.path.join(binpath, dest) + try: + shutil.copy2(src, dest) + except OSError: + logger.warning('Unable to copy %r to %r', src, dest) if sysconfig.is_python_build(): # copy init.tcl @@ -437,6 +477,14 @@ def install_scripts(self, context, path): """ binpath = context.bin_path plen = len(path) + if os.name == 'nt': + def skip_file(f): + f = os.path.normcase(f) + return (f.startswith(('python', 'venv')) + and f.endswith(('.exe', '.pdb'))) + else: + def skip_file(f): + return False for root, dirs, files in os.walk(path): if root == path: # at top-level, remove irrelevant dirs for d in dirs[:]: @@ -444,8 +492,7 @@ def install_scripts(self, context, path): dirs.remove(d) continue # ignore files in top level for f in files: - if (os.name == 'nt' and f.startswith('python') - and f.endswith(('.exe', '.pdb'))): + if skip_file(f): continue srcfile = os.path.join(root, f) suffix = root[plen:].split(os.sep)[2:] @@ -456,20 +503,25 @@ def install_scripts(self, context, path): if not os.path.exists(dstdir): os.makedirs(dstdir) dstfile = os.path.join(dstdir, f) + if os.name == 'nt' and srcfile.endswith(('.exe', '.pdb')): + shutil.copy2(srcfile, dstfile) + continue with open(srcfile, 'rb') as f: data = f.read() - if not srcfile.endswith(('.exe', '.pdb')): - try: - data = data.decode('utf-8') - data = self.replace_variables(data, context) - data = data.encode('utf-8') - except UnicodeError as e: - data = None - logger.warning('unable to copy script %r, ' - 'may be binary: %s', srcfile, e) - if data is not None: + try: + new_data = ( + self.replace_variables(data.decode('utf-8'), context) + .encode('utf-8') + ) + except UnicodeError as e: + logger.warning('unable to copy script %r, ' + 'may be binary: %s', srcfile, e) + continue + if new_data == data: + shutil.copy2(srcfile, dstfile) + else: with open(dstfile, 'wb') as f: - f.write(data) + f.write(new_data) shutil.copymode(srcfile, dstfile) def upgrade_dependencies(self, context): diff --git a/Makefile.pre.in b/Makefile.pre.in index 289ab97666e902..d251e7c481b52b 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2221,6 +2221,7 @@ TESTSUBDIRS= idlelib/idle_test \ test/test_cppext \ test/test_ctypes \ test/test_dataclasses \ + test/test_doctest \ test/test_email \ test/test_email/data \ test/test_future_stmt \ diff --git a/Misc/NEWS.d/next/Core and Builtins/2024-01-17-23-39-20.gh-issue-114050.Lnv1oq.rst b/Misc/NEWS.d/next/Core and Builtins/2024-01-17-23-39-20.gh-issue-114050.Lnv1oq.rst new file mode 100644 index 00000000000000..c35d2508e6bdda --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2024-01-17-23-39-20.gh-issue-114050.Lnv1oq.rst @@ -0,0 +1,2 @@ +Fix segmentation fault caused by an incorrect format string +in ``TypeError`` exception when more than two arguments are passed to ``int``. diff --git a/Misc/NEWS.d/next/Documentation/2024-01-17-11-40-03.gh-issue-114123.LuueXf.rst b/Misc/NEWS.d/next/Documentation/2024-01-17-11-40-03.gh-issue-114123.LuueXf.rst new file mode 100644 index 00000000000000..1d93a422840077 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2024-01-17-11-40-03.gh-issue-114123.LuueXf.rst @@ -0,0 +1,7 @@ +Move the :mod:`csv` module docstring to the :mod:`!csv` module +instead of reexporting it from the internal :mod:`!_csv` module, +and remove ``__doc__`` from ``csv.__all__``. + +Move :attr:`!csv.__version__` to the :mod:`!csv` module +instead of reexporting it from the internal :mod:`!_csv` module, +and remove ``__version__`` from ``csv.__all__``. diff --git a/Misc/NEWS.d/next/IDLE/2024-01-17-23-18-15.gh-issue-96905.UYaxoU.rst b/Misc/NEWS.d/next/IDLE/2024-01-17-23-18-15.gh-issue-96905.UYaxoU.rst new file mode 100644 index 00000000000000..fe7dde64c7c7d5 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2024-01-17-23-18-15.gh-issue-96905.UYaxoU.rst @@ -0,0 +1 @@ +In idlelib code, stop redefining built-ins 'dict' and 'object'. diff --git a/Misc/NEWS.d/next/Library/2023-05-30-18-30-11.gh-issue-105102.SnpK04.rst b/Misc/NEWS.d/next/Library/2023-05-30-18-30-11.gh-issue-105102.SnpK04.rst new file mode 100644 index 00000000000000..7ca21afefa3132 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-05-30-18-30-11.gh-issue-105102.SnpK04.rst @@ -0,0 +1,2 @@ +Allow :class:`ctypes.Union` to be nested in :class:`ctypes.Structure` when +the system endianness is the opposite of the classes. diff --git a/Misc/NEWS.d/next/Library/2023-09-22-22-17-45.gh-issue-38807.m9McRN.rst b/Misc/NEWS.d/next/Library/2023-09-22-22-17-45.gh-issue-38807.m9McRN.rst new file mode 100644 index 00000000000000..4219723d15b9e6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-09-22-22-17-45.gh-issue-38807.m9McRN.rst @@ -0,0 +1,3 @@ +Fix race condition in :mod:`trace`. Instead of checking if a directory +exists and creating it, directly call :func:`os.makedirs` with the kwarg +``exist_ok=True``. diff --git a/Misc/NEWS.d/next/Library/2023-10-04-11-09-30.gh-issue-110345.fZU1ud.rst b/Misc/NEWS.d/next/Library/2023-10-04-11-09-30.gh-issue-110345.fZU1ud.rst new file mode 100644 index 00000000000000..d9ccc0f12de47c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-10-04-11-09-30.gh-issue-110345.fZU1ud.rst @@ -0,0 +1 @@ +Show the Tcl/Tk patchlevel (rather than version) in :meth:`tkinter._test`. diff --git a/Misc/NEWS.d/next/Library/2024-01-15-19-54-41.gh-issue-114087.Xic5vY.rst b/Misc/NEWS.d/next/Library/2024-01-15-19-54-41.gh-issue-114087.Xic5vY.rst new file mode 100644 index 00000000000000..68b27a7b0c9f6c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-15-19-54-41.gh-issue-114087.Xic5vY.rst @@ -0,0 +1 @@ +Speed up ``dataclasses.asdict`` up to 1.35x. diff --git a/Misc/NEWS.d/next/Library/2024-01-16-15-59-06.gh-issue-114149.LJ8IPm.rst b/Misc/NEWS.d/next/Library/2024-01-16-15-59-06.gh-issue-114149.LJ8IPm.rst new file mode 100644 index 00000000000000..1403d78d0d4905 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-16-15-59-06.gh-issue-114149.LJ8IPm.rst @@ -0,0 +1 @@ +Enum: correctly handle tuple subclasses in custom ``__new__``. diff --git a/Misc/NEWS.d/next/Library/2024-01-17-18-53-51.gh-issue-104522.3NyDf4.rst b/Misc/NEWS.d/next/Library/2024-01-17-18-53-51.gh-issue-104522.3NyDf4.rst new file mode 100644 index 00000000000000..ca980945ea12d3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-17-18-53-51.gh-issue-104522.3NyDf4.rst @@ -0,0 +1,3 @@ +:exc:`OSError` raised when run a subprocess now only has *filename* +attribute set to *cwd* if the error was caused by a failed attempt to change +the current directory. diff --git a/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst b/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst new file mode 100644 index 00000000000000..fa047e288f807e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-01-18-10-07-52.gh-issue-114198.lK4Iif.rst @@ -0,0 +1,2 @@ +The signature for the ``__replace__`` method on :mod:`dataclasses` now has +the first argument named ``self``, rather than ``obj``. diff --git a/Misc/NEWS.d/next/Windows/2023-12-19-22-32-28.gh-issue-112984.F7kFMl.rst b/Misc/NEWS.d/next/Windows/2023-12-19-22-32-28.gh-issue-112984.F7kFMl.rst new file mode 100644 index 00000000000000..429cd5bc923e09 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2023-12-19-22-32-28.gh-issue-112984.F7kFMl.rst @@ -0,0 +1 @@ +Adds free-threaded binaries to Windows installer as an optional component. diff --git a/Modules/_csv.c b/Modules/_csv.c index d45a15aa8c255a..8d941563025580 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -8,8 +8,6 @@ module instead. */ -#define MODULE_VERSION "1.0" - // clinic/_csv.c.h uses internal pycore_modsupport.h API #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 @@ -1607,68 +1605,7 @@ PyType_Spec error_spec = { * MODULE */ -PyDoc_STRVAR(csv_module_doc, -"CSV parsing and writing.\n" -"\n" -"This module provides classes that assist in the reading and writing\n" -"of Comma Separated Value (CSV) files, and implements the interface\n" -"described by PEP 305. Although many CSV files are simple to parse,\n" -"the format is not formally defined by a stable specification and\n" -"is subtle enough that parsing lines of a CSV file with something\n" -"like line.split(\",\") is bound to fail. The module supports three\n" -"basic APIs: reading, writing, and registration of dialects.\n" -"\n" -"\n" -"DIALECT REGISTRATION:\n" -"\n" -"Readers and writers support a dialect argument, which is a convenient\n" -"handle on a group of settings. When the dialect argument is a string,\n" -"it identifies one of the dialects previously registered with the module.\n" -"If it is a class or instance, the attributes of the argument are used as\n" -"the settings for the reader or writer:\n" -"\n" -" class excel:\n" -" delimiter = ','\n" -" quotechar = '\"'\n" -" escapechar = None\n" -" doublequote = True\n" -" skipinitialspace = False\n" -" lineterminator = '\\r\\n'\n" -" quoting = QUOTE_MINIMAL\n" -"\n" -"SETTINGS:\n" -"\n" -" * quotechar - specifies a one-character string to use as the\n" -" quoting character. It defaults to '\"'.\n" -" * delimiter - specifies a one-character string to use as the\n" -" field separator. It defaults to ','.\n" -" * skipinitialspace - specifies how to interpret spaces which\n" -" immediately follow a delimiter. It defaults to False, which\n" -" means that spaces immediately following a delimiter is part\n" -" of the following field.\n" -" * lineterminator - specifies the character sequence which should\n" -" terminate rows.\n" -" * quoting - controls when quotes should be generated by the writer.\n" -" It can take on any of the following module constants:\n" -"\n" -" csv.QUOTE_MINIMAL means only when required, for example, when a\n" -" field contains either the quotechar or the delimiter\n" -" csv.QUOTE_ALL means that quotes are always placed around fields.\n" -" csv.QUOTE_NONNUMERIC means that quotes are always placed around\n" -" fields which do not parse as integers or floating point\n" -" numbers.\n" -" csv.QUOTE_STRINGS means that quotes are always placed around\n" -" fields which are strings. Note that the Python value None\n" -" is not a string.\n" -" csv.QUOTE_NOTNULL means that quotes are only placed around fields\n" -" that are not the Python value None.\n" -" csv.QUOTE_NONE means that quotes are never placed around fields.\n" -" * escapechar - specifies a one-character string used to escape\n" -" the delimiter when quoting is set to QUOTE_NONE.\n" -" * doublequote - controls the handling of quotes inside fields. When\n" -" True, two consecutive quotes are interpreted as one during read,\n" -" and when writing, each quote character embedded in the data is\n" -" written as two quotes\n"); +PyDoc_STRVAR(csv_module_doc, "CSV parsing and writing.\n"); PyDoc_STRVAR(csv_reader_doc, " csv_reader = reader(iterable [, dialect='excel']\n" @@ -1741,12 +1678,6 @@ csv_exec(PyObject *module) { return -1; } - /* Add version to the module. */ - if (PyModule_AddStringConstant(module, "__version__", - MODULE_VERSION) == -1) { - return -1; - } - /* Set the field limit */ module_state->field_limit = 128 * 1024; diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index fc16b9176fd1c0..b51a03b5497fed 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -876,20 +876,6 @@ CDataType_repeat(PyObject *self, Py_ssize_t length) return PyCArrayType_from_ctype(self, length); } -static PySequenceMethods CDataType_as_sequence = { - 0, /* inquiry sq_length; */ - 0, /* binaryfunc sq_concat; */ - CDataType_repeat, /* intargfunc sq_repeat; */ - 0, /* intargfunc sq_item; */ - 0, /* intintargfunc sq_slice; */ - 0, /* intobjargproc sq_ass_item; */ - 0, /* intintobjargproc sq_ass_slice; */ - 0, /* objobjproc sq_contains; */ - - 0, /* binaryfunc sq_inplace_concat; */ - 0, /* intargfunc sq_inplace_repeat; */ -}; - static int CDataType_clear(PyTypeObject *self) { @@ -903,8 +889,10 @@ static int CDataType_traverse(PyTypeObject *self, visitproc visit, void *arg) { StgDictObject *dict = PyType_stgdict((PyObject *)self); - if (dict) + if (dict) { Py_VISIT(dict->proto); + } + Py_VISIT(Py_TYPE(self)); return PyType_Type.tp_traverse((PyObject *)self, visit, arg); } @@ -935,91 +923,45 @@ UnionType_setattro(PyObject *self, PyObject *key, PyObject *value) return 0; } +static PyType_Slot pycstruct_type_slots[] = { + {Py_tp_setattro, PyCStructType_setattro}, + {Py_tp_doc, PyDoc_STR("metatype for the CData Objects")}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_clear, CDataType_clear}, + {Py_tp_methods, CDataType_methods}, + {Py_tp_new, PyCStructType_new}, -PyTypeObject PyCStructType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.PyCStructType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - PyCStructType_setattro, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - PyDoc_STR("metatype for the CData Objects"), /* tp_doc */ - (traverseproc)CDataType_traverse, /* tp_traverse */ - (inquiry)CDataType_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - CDataType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - PyCStructType_new, /* tp_new */ - 0, /* tp_free */ + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, }; -static PyTypeObject UnionType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.UnionType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - UnionType_setattro, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - PyDoc_STR("metatype for the CData Objects"), /* tp_doc */ - (traverseproc)CDataType_traverse, /* tp_traverse */ - (inquiry)CDataType_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - CDataType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - UnionType_new, /* tp_new */ - 0, /* tp_free */ +PyType_Spec pycstruct_type_spec = { + .name = "_ctypes.PyCStructType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = pycstruct_type_slots, +}; + +static PyType_Slot union_type_slots[] = { + {Py_tp_setattro, UnionType_setattro}, + {Py_tp_doc, PyDoc_STR("metatype for the Union Objects")}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_clear, CDataType_clear}, + {Py_tp_methods, CDataType_methods}, + {Py_tp_new, UnionType_new}, + + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, }; +static PyType_Spec union_type_spec = { + .name = "_ctypes.UnionType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = union_type_slots, +}; /******************************************************************/ @@ -1234,46 +1176,23 @@ static PyMethodDef PyCPointerType_methods[] = { { NULL, NULL }, }; -PyTypeObject PyCPointerType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.PyCPointerType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - PyDoc_STR("metatype for the Pointer Objects"), /* tp_doc */ - (traverseproc)CDataType_traverse, /* tp_traverse */ - (inquiry)CDataType_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - PyCPointerType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - PyCPointerType_new, /* tp_new */ - 0, /* tp_free */ +static PyType_Slot pycpointer_type_slots[] = { + {Py_tp_doc, PyDoc_STR("metatype for the Pointer Objects")}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_clear, CDataType_clear}, + {Py_tp_methods, PyCPointerType_methods}, + {Py_tp_new, PyCPointerType_new}, + + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, +}; + +static PyType_Spec pycpointer_type_spec = { + .name = "_ctypes.PyCPointerType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = pycpointer_type_slots, }; @@ -1606,48 +1525,24 @@ PyCArrayType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return NULL; } -PyTypeObject PyCArrayType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.PyCArrayType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */ - PyDoc_STR("metatype for the Array Objects"), /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - CDataType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - PyCArrayType_new, /* tp_new */ - 0, /* tp_free */ +static PyType_Slot pycarray_type_slots[] = { + {Py_tp_doc, PyDoc_STR("metatype for the Array Objects")}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_methods, CDataType_methods}, + {Py_tp_new, PyCArrayType_new}, + {Py_tp_clear, CDataType_clear}, + + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, }; +static PyType_Spec pycarray_type_spec = { + .name = "_ctypes.PyCArrayType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = pycarray_type_slots, +}; /******************************************************************/ /* @@ -2175,7 +2070,11 @@ PyCSimpleType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) } } - if (type == &PyCSimpleType_Type && fmt->setfunc_swapped && fmt->getfunc_swapped) { + ctypes_state *st = GLOBAL_STATE(); + if (type == st->PyCSimpleType_Type + && fmt->setfunc_swapped + && fmt->getfunc_swapped) + { PyObject *swapped = CreateSwappedType(type, args, kwds, proto, fmt); StgDictObject *sw_dict; @@ -2292,46 +2191,23 @@ static PyMethodDef PyCSimpleType_methods[] = { { NULL, NULL }, }; -PyTypeObject PyCSimpleType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.PyCSimpleType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */ - PyDoc_STR("metatype for the PyCSimpleType Objects"), /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - PyCSimpleType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - PyCSimpleType_new, /* tp_new */ - 0, /* tp_free */ +static PyType_Slot pycsimple_type_slots[] = { + {Py_tp_doc, PyDoc_STR("metatype for the PyCSimpleType Objects")}, + {Py_tp_methods, PyCSimpleType_methods}, + {Py_tp_new, PyCSimpleType_new}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_clear, CDataType_clear}, + + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, +}; + +PyType_Spec pycsimple_type_spec = { + .name = "_ctypes.PyCSimpleType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = pycsimple_type_slots, }; /******************************************************************/ @@ -2575,46 +2451,23 @@ PyCFuncPtrType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return (PyObject *)result; } -PyTypeObject PyCFuncPtrType_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_ctypes.PyCFuncPtrType", /* tp_name */ - 0, /* tp_basicsize */ - 0, /* tp_itemsize */ - 0, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - &CDataType_as_sequence, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /* tp_flags */ - PyDoc_STR("metatype for C function pointers"), /* tp_doc */ - (traverseproc)CDataType_traverse, /* tp_traverse */ - (inquiry)CDataType_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - CDataType_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - PyCFuncPtrType_new, /* tp_new */ - 0, /* tp_free */ +static PyType_Slot pycfuncptr_type_slots[] = { + {Py_tp_doc, PyDoc_STR("metatype for C function pointers")}, + {Py_tp_traverse, CDataType_traverse}, + {Py_tp_clear, CDataType_clear}, + {Py_tp_methods, CDataType_methods}, + {Py_tp_new, PyCFuncPtrType_new}, + + // Sequence protocol. + {Py_sq_repeat, CDataType_repeat}, + {0, NULL}, +}; + +static PyType_Spec pycfuncptr_type_spec = { + .name = "_ctypes.PyCFuncPtrType", + .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_IMMUTABLETYPE), + .slots = pycfuncptr_type_slots, }; @@ -4881,7 +4734,8 @@ PyCArrayType_from_ctype(PyObject *itemtype, Py_ssize_t length) ((PyTypeObject *)itemtype)->tp_name, (long)length); #endif - result = PyObject_CallFunction((PyObject *)&PyCArrayType_Type, + ctypes_state *st = GLOBAL_STATE(); + result = PyObject_CallFunction((PyObject *)st->PyCArrayType_Type, "s(O){s:n,s:O}", name, &PyCArray_Type, @@ -5690,24 +5544,29 @@ _ctypes_add_types(PyObject *mod) * * Metaclasses */ - TYPE_READY_BASE(&PyCStructType_Type, &PyType_Type); - TYPE_READY_BASE(&UnionType_Type, &PyType_Type); - TYPE_READY_BASE(&PyCPointerType_Type, &PyType_Type); - TYPE_READY_BASE(&PyCArrayType_Type, &PyType_Type); - TYPE_READY_BASE(&PyCSimpleType_Type, &PyType_Type); - TYPE_READY_BASE(&PyCFuncPtrType_Type, &PyType_Type); + CREATE_TYPE(mod, st->PyCStructType_Type, &pycstruct_type_spec, + &PyType_Type); + CREATE_TYPE(mod, st->UnionType_Type, &union_type_spec, &PyType_Type); + CREATE_TYPE(mod, st->PyCPointerType_Type, &pycpointer_type_spec, + &PyType_Type); + CREATE_TYPE(mod, st->PyCArrayType_Type, &pycarray_type_spec, + &PyType_Type); + CREATE_TYPE(mod, st->PyCSimpleType_Type, &pycsimple_type_spec, + &PyType_Type); + CREATE_TYPE(mod, st->PyCFuncPtrType_Type, &pycfuncptr_type_spec, + &PyType_Type); /************************************************* * * Classes using a custom metaclass */ - MOD_ADD_TYPE(&Struct_Type, &PyCStructType_Type, &PyCData_Type); - MOD_ADD_TYPE(&Union_Type, &UnionType_Type, &PyCData_Type); - MOD_ADD_TYPE(&PyCPointer_Type, &PyCPointerType_Type, &PyCData_Type); - MOD_ADD_TYPE(&PyCArray_Type, &PyCArrayType_Type, &PyCData_Type); - MOD_ADD_TYPE(&Simple_Type, &PyCSimpleType_Type, &PyCData_Type); - MOD_ADD_TYPE(&PyCFuncPtr_Type, &PyCFuncPtrType_Type, &PyCData_Type); + MOD_ADD_TYPE(&Struct_Type, st->PyCStructType_Type, &PyCData_Type); + MOD_ADD_TYPE(&Union_Type, st->UnionType_Type, &PyCData_Type); + MOD_ADD_TYPE(&PyCPointer_Type, st->PyCPointerType_Type, &PyCData_Type); + MOD_ADD_TYPE(&PyCArray_Type, st->PyCArrayType_Type, &PyCData_Type); + MOD_ADD_TYPE(&Simple_Type, st->PyCSimpleType_Type, &PyCData_Type); + MOD_ADD_TYPE(&PyCFuncPtr_Type, st->PyCFuncPtrType_Type, &PyCData_Type); /************************************************* * diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h index 8891a0a741de7b..55e9f777788079 100644 --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -41,6 +41,12 @@ typedef struct { PyTypeObject *PyComError_Type; #endif PyTypeObject *StructParam_Type; + PyTypeObject *PyCStructType_Type; + PyTypeObject *UnionType_Type; + PyTypeObject *PyCPointerType_Type; + PyTypeObject *PyCArrayType_Type; + PyTypeObject *PyCSimpleType_Type; + PyTypeObject *PyCFuncPtrType_Type; } ctypes_state; extern ctypes_state global_state; @@ -118,7 +124,7 @@ typedef struct { Py_ssize_t b_size; /* size of memory block in bytes */ Py_ssize_t b_length; /* number of references we need */ Py_ssize_t b_index; /* index of this object into base's - b_object list */ + b_object list */ PyObject *b_objects; /* list of references we need to keep */ union value b_value; /* end of tagCDataObject, additional fields follow */ @@ -155,9 +161,8 @@ extern PyTypeObject PyCData_Type; #define CDataObject_Check(v) PyObject_TypeCheck(v, &PyCData_Type) #define _CDataObject_HasExternalBuffer(v) ((v)->b_ptr != (char *)&(v)->b_value) -extern PyTypeObject PyCSimpleType_Type; -#define PyCSimpleTypeObject_CheckExact(v) Py_IS_TYPE(v, &PyCSimpleType_Type) -#define PyCSimpleTypeObject_Check(v) PyObject_TypeCheck(v, &PyCSimpleType_Type) +#define PyCSimpleTypeObject_CheckExact(v) Py_IS_TYPE(v, GLOBAL_STATE()->PyCSimpleType_Type) +#define PyCSimpleTypeObject_Check(v) PyObject_TypeCheck(v, GLOBAL_STATE()->PyCSimpleType_Type) extern struct fielddesc *_ctypes_get_fielddesc(const char *fmt); @@ -171,21 +176,17 @@ PyCField_FromDesc(PyObject *desc, Py_ssize_t index, extern PyObject *PyCData_AtAddress(PyObject *type, void *buf); extern PyObject *PyCData_FromBytes(PyObject *type, char *data, Py_ssize_t length); -extern PyTypeObject PyCArrayType_Type; extern PyTypeObject PyCArray_Type; -extern PyTypeObject PyCPointerType_Type; extern PyTypeObject PyCPointer_Type; extern PyTypeObject PyCFuncPtr_Type; -extern PyTypeObject PyCFuncPtrType_Type; -extern PyTypeObject PyCStructType_Type; -#define PyCArrayTypeObject_Check(v) PyObject_TypeCheck(v, &PyCArrayType_Type) +#define PyCArrayTypeObject_Check(v) PyObject_TypeCheck(v, GLOBAL_STATE()->PyCArrayType_Type) #define ArrayObject_Check(v) PyObject_TypeCheck(v, &PyCArray_Type) #define PointerObject_Check(v) PyObject_TypeCheck(v, &PyCPointer_Type) -#define PyCPointerTypeObject_Check(v) PyObject_TypeCheck(v, &PyCPointerType_Type) +#define PyCPointerTypeObject_Check(v) PyObject_TypeCheck(v, GLOBAL_STATE()->PyCPointerType_Type) #define PyCFuncPtrObject_Check(v) PyObject_TypeCheck(v, &PyCFuncPtr_Type) -#define PyCFuncPtrTypeObject_Check(v) PyObject_TypeCheck(v, &PyCFuncPtrType_Type) -#define PyCStructTypeObject_Check(v) PyObject_TypeCheck(v, &PyCStructType_Type) +#define PyCFuncPtrTypeObject_Check(v) PyObject_TypeCheck(v, GLOBAL_STATE()->PyCFuncPtrType_Type) +#define PyCStructTypeObject_Check(v) PyObject_TypeCheck(v, GLOBAL_STATE()->PyCStructType_Type) extern PyObject * PyCArrayType_from_ctype(PyObject *itemtype, Py_ssize_t length); diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index d0dd8f064e0395..aa1a300e4378dd 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -673,9 +673,10 @@ child_exec(char *const exec_array[], PyObject *preexec_fn, PyObject *preexec_fn_args_tuple) { - int i, saved_errno, reached_preexec = 0; + int i, saved_errno; PyObject *result; - const char* err_msg = ""; + /* Indicate to the parent that the error happened before exec(). */ + const char *err_msg = "noexec"; /* Buffer large enough to hold a hex integer. We can't malloc. */ char hex_errno[sizeof(saved_errno)*2+1]; @@ -735,8 +736,12 @@ child_exec(char *const exec_array[], /* We no longer manually close p2cread, c2pwrite, and errwrite here as * _close_open_fds takes care when it is not already non-inheritable. */ - if (cwd) - POSIX_CALL(chdir(cwd)); + if (cwd) { + if (chdir(cwd) == -1) { + err_msg = "noexec:chdir"; + goto error; + } + } if (child_umask >= 0) umask(child_umask); /* umask() always succeeds. */ @@ -784,7 +789,7 @@ child_exec(char *const exec_array[], #endif /* HAVE_SETREUID */ - reached_preexec = 1; + err_msg = ""; if (preexec_fn != Py_None && preexec_fn_args_tuple) { /* This is where the user has asked us to deadlock their program. */ result = PyObject_Call(preexec_fn, preexec_fn_args_tuple, NULL); @@ -842,16 +847,12 @@ child_exec(char *const exec_array[], } _Py_write_noraise(errpipe_write, cur, hex_errno + sizeof(hex_errno) - cur); _Py_write_noraise(errpipe_write, ":", 1); - if (!reached_preexec) { - /* Indicate to the parent that the error happened before exec(). */ - _Py_write_noraise(errpipe_write, "noexec", 6); - } /* We can't call strerror(saved_errno). It is not async signal safe. * The parent process will look the error message up. */ } else { _Py_write_noraise(errpipe_write, "SubprocessError:0:", 18); - _Py_write_noraise(errpipe_write, err_msg, strlen(err_msg)); } + _Py_write_noraise(errpipe_write, err_msg, strlen(err_msg)); } diff --git a/Objects/genobject.c b/Objects/genobject.c index f03919c75d70a5..e9aeb7ab9a9fa8 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -1628,12 +1628,12 @@ PyTypeObject PyAsyncGen_Type = { }; -#if _PyAsyncGen_MAXFREELIST > 0 +#ifdef WITH_FREELISTS static struct _Py_async_gen_state * get_async_gen_state(void) { - PyInterpreterState *interp = _PyInterpreterState_GET(); - return &interp->async_gen; + _PyFreeListState *state = _PyFreeListState_GET(); + return &state->async_gen_state; } #endif @@ -1656,36 +1656,36 @@ PyAsyncGen_New(PyFrameObject *f, PyObject *name, PyObject *qualname) void -_PyAsyncGen_ClearFreeLists(PyInterpreterState *interp) +_PyAsyncGen_ClearFreeLists(_PyFreeListState *freelist_state, int is_finalization) { -#if _PyAsyncGen_MAXFREELIST > 0 - struct _Py_async_gen_state *state = &interp->async_gen; +#ifdef WITH_FREELISTS + struct _Py_async_gen_state *state = &freelist_state->async_gen_state; - while (state->value_numfree) { + while (state->value_numfree > 0) { _PyAsyncGenWrappedValue *o; o = state->value_freelist[--state->value_numfree]; assert(_PyAsyncGenWrappedValue_CheckExact(o)); PyObject_GC_Del(o); } - while (state->asend_numfree) { + while (state->asend_numfree > 0) { PyAsyncGenASend *o; o = state->asend_freelist[--state->asend_numfree]; assert(Py_IS_TYPE(o, &_PyAsyncGenASend_Type)); PyObject_GC_Del(o); } + + if (is_finalization) { + state->value_numfree = -1; + state->asend_numfree = -1; + } #endif } void -_PyAsyncGen_Fini(PyInterpreterState *interp) +_PyAsyncGen_Fini(_PyFreeListState *state) { - _PyAsyncGen_ClearFreeLists(interp); -#if defined(Py_DEBUG) && _PyAsyncGen_MAXFREELIST > 0 - struct _Py_async_gen_state *state = &interp->async_gen; - state->value_numfree = -1; - state->asend_numfree = -1; -#endif + _PyAsyncGen_ClearFreeLists(state, 1); } @@ -1732,13 +1732,9 @@ async_gen_asend_dealloc(PyAsyncGenASend *o) _PyObject_GC_UNTRACK((PyObject *)o); Py_CLEAR(o->ags_gen); Py_CLEAR(o->ags_sendval); -#if _PyAsyncGen_MAXFREELIST > 0 +#ifdef WITH_FREELISTS struct _Py_async_gen_state *state = get_async_gen_state(); -#ifdef Py_DEBUG - // async_gen_asend_dealloc() must not be called after _PyAsyncGen_Fini() - assert(state->asend_numfree != -1); -#endif - if (state->asend_numfree < _PyAsyncGen_MAXFREELIST) { + if (state->asend_numfree >= 0 && state->asend_numfree < _PyAsyncGen_MAXFREELIST) { assert(PyAsyncGenASend_CheckExact(o)); _PyGC_CLEAR_FINALIZED((PyObject *)o); state->asend_freelist[state->asend_numfree++] = o; @@ -1906,13 +1902,9 @@ static PyObject * async_gen_asend_new(PyAsyncGenObject *gen, PyObject *sendval) { PyAsyncGenASend *o; -#if _PyAsyncGen_MAXFREELIST > 0 +#ifdef WITH_FREELISTS struct _Py_async_gen_state *state = get_async_gen_state(); -#ifdef Py_DEBUG - // async_gen_asend_new() must not be called after _PyAsyncGen_Fini() - assert(state->asend_numfree != -1); -#endif - if (state->asend_numfree) { + if (state->asend_numfree > 0) { state->asend_numfree--; o = state->asend_freelist[state->asend_numfree]; _Py_NewReference((PyObject *)o); @@ -1945,13 +1937,9 @@ async_gen_wrapped_val_dealloc(_PyAsyncGenWrappedValue *o) { _PyObject_GC_UNTRACK((PyObject *)o); Py_CLEAR(o->agw_val); -#if _PyAsyncGen_MAXFREELIST > 0 +#ifdef WITH_FREELISTS struct _Py_async_gen_state *state = get_async_gen_state(); -#ifdef Py_DEBUG - // async_gen_wrapped_val_dealloc() must not be called after _PyAsyncGen_Fini() - assert(state->value_numfree != -1); -#endif - if (state->value_numfree < _PyAsyncGen_MAXFREELIST) { + if (state->value_numfree >= 0 && state->value_numfree < _PyAsyncGen_MAXFREELIST) { assert(_PyAsyncGenWrappedValue_CheckExact(o)); state->value_freelist[state->value_numfree++] = o; OBJECT_STAT_INC(to_freelist); @@ -2022,13 +2010,9 @@ _PyAsyncGenValueWrapperNew(PyThreadState *tstate, PyObject *val) _PyAsyncGenWrappedValue *o; assert(val); -#if _PyAsyncGen_MAXFREELIST > 0 - struct _Py_async_gen_state *state = &tstate->interp->async_gen; -#ifdef Py_DEBUG - // _PyAsyncGenValueWrapperNew() must not be called after _PyAsyncGen_Fini() - assert(state->value_numfree != -1); -#endif - if (state->value_numfree) { +#ifdef WITH_FREELISTS + struct _Py_async_gen_state *state = get_async_gen_state(); + if (state->value_numfree > 0) { state->value_numfree--; o = state->value_freelist[state->value_numfree]; OBJECT_STAT_INC(from_freelist); diff --git a/Objects/listobject.c b/Objects/listobject.c index 79ef8f532fbb98..401d1026133f4e 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -816,13 +816,10 @@ static PyObject * list_insert_impl(PyListObject *self, Py_ssize_t index, PyObject *object) /*[clinic end generated code: output=7f35e32f60c8cb78 input=b1987ca998a4ae2d]*/ { - PyObject *ret = Py_None; - Py_BEGIN_CRITICAL_SECTION(self); - if (ins1(self, index, object) < 0) { - ret = NULL; + if (ins1(self, index, object) == 0) { + Py_RETURN_NONE; } - Py_END_CRITICAL_SECTION(); - return ret; + return NULL; } /*[clinic input] diff --git a/Objects/longobject.c b/Objects/longobject.c index fae70dd13bb18a..e655ba19e8f1c1 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -6171,7 +6171,7 @@ long_vectorcall(PyObject *type, PyObject * const*args, return long_new_impl(_PyType_CAST(type), args[0], args[1]); default: return PyErr_Format(PyExc_TypeError, - "int expected at most 2 argument%s, got %zd", + "int expected at most 2 arguments, got %zd", nargs); } } diff --git a/Objects/sliceobject.c b/Objects/sliceobject.c index 9ec8ea8e1b307f..440c1da30620c3 100644 --- a/Objects/sliceobject.c +++ b/Objects/sliceobject.c @@ -105,16 +105,20 @@ PyObject _Py_EllipsisObject = _PyObject_HEAD_INIT(&PyEllipsis_Type); void _PySlice_ClearCache(_PyFreeListState *state) { +#ifdef WITH_FREELISTS PySliceObject *obj = state->slice_state.slice_cache; if (obj != NULL) { state->slice_state.slice_cache = NULL; PyObject_GC_Del(obj); } +#endif } void _PySlice_Fini(_PyFreeListState *state) { +#ifdef WITH_FREELISTS _PySlice_ClearCache(state); +#endif } /* start, stop, and step are python objects with None indicating no @@ -125,15 +129,17 @@ static PySliceObject * _PyBuildSlice_Consume2(PyObject *start, PyObject *stop, PyObject *step) { assert(start != NULL && stop != NULL && step != NULL); - - _PyFreeListState *state = _PyFreeListState_GET(); PySliceObject *obj; +#ifdef WITH_FREELISTS + _PyFreeListState *state = _PyFreeListState_GET(); if (state->slice_state.slice_cache != NULL) { obj = state->slice_state.slice_cache; state->slice_state.slice_cache = NULL; _Py_NewReference((PyObject *)obj); } - else { + else +#endif + { obj = PyObject_GC_New(PySliceObject, &PySlice_Type); if (obj == NULL) { goto error; @@ -358,15 +364,18 @@ Create a slice object. This is used for extended slicing (e.g. a[0:10:2])."); static void slice_dealloc(PySliceObject *r) { - _PyFreeListState *state = _PyFreeListState_GET(); _PyObject_GC_UNTRACK(r); Py_DECREF(r->step); Py_DECREF(r->start); Py_DECREF(r->stop); +#ifdef WITH_FREELISTS + _PyFreeListState *state = _PyFreeListState_GET(); if (state->slice_state.slice_cache == NULL) { state->slice_state.slice_cache = r; } - else { + else +#endif + { PyObject_GC_Del(r); } } diff --git a/PC/layout/main.py b/PC/layout/main.py index accfd51dd978fb..d176b272f1c19d 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -41,7 +41,7 @@ VENV_DIRS_ONLY = FileNameSet("venv", "ensurepip") -EXCLUDE_FROM_PYDS = FileStemSet("python*", "pyshellext", "vcruntime*") +EXCLUDE_FROM_DLLS = FileStemSet("python*", "pyshellext", "vcruntime*") EXCLUDE_FROM_LIB = FileNameSet("*.pyc", "__pycache__", "*.pickle") EXCLUDE_FROM_PACKAGED_LIB = FileNameSet("readme.txt") EXCLUDE_FROM_COMPILE = FileNameSet("badsyntax_*", "bad_*") @@ -126,9 +126,9 @@ def in_build(f, dest="", new_name=None): n = new_name or n src = ns.build / f if ns.debug and src not in REQUIRED_DLLS: - if not src.stem.endswith("_d"): + if not "_d." in src.name: src = src.parent / (src.stem + "_d" + src.suffix) - if not n.endswith("_d"): + if "_d." not in f: n += "_d" f = n + "." + x yield dest + n + "." + x, src @@ -141,17 +141,45 @@ def in_build(f, dest="", new_name=None): if lib.is_file(): yield "libs/" + n + ".lib", lib + source = "python.exe" + sourcew = "pythonw.exe" + alias = [ + "python", + "python{}".format(VER_MAJOR) if ns.include_alias3 else "", + "python{}".format(VER_DOT) if ns.include_alias3x else "", + ] + aliasw = [ + "pythonw", + "pythonw{}".format(VER_MAJOR) if ns.include_alias3 else "", + "pythonw{}".format(VER_DOT) if ns.include_alias3x else "", + ] if ns.include_appxmanifest: - yield from in_build("python_uwp.exe", new_name="python{}".format(VER_DOT)) - yield from in_build("pythonw_uwp.exe", new_name="pythonw{}".format(VER_DOT)) - # For backwards compatibility, but we don't reference these ourselves. - yield from in_build("python_uwp.exe", new_name="python") - yield from in_build("pythonw_uwp.exe", new_name="pythonw") + source = "python_uwp.exe" + sourcew = "pythonw_uwp.exe" + elif ns.include_freethreaded: + source = "python{}t.exe".format(VER_DOT) + sourcew = "pythonw{}t.exe".format(VER_DOT) + if not ns.include_alias: + alias = [] + aliasw = [] + alias.extend([ + "python{}t".format(VER_DOT), + "python{}t".format(VER_MAJOR) if ns.include_alias3 else None, + ]) + aliasw.extend([ + "pythonw{}t".format(VER_DOT), + "pythonw{}t".format(VER_MAJOR) if ns.include_alias3 else None, + ]) + + for a in filter(None, alias): + yield from in_build(source, new_name=a) + for a in filter(None, aliasw): + yield from in_build(sourcew, new_name=a) + + if ns.include_freethreaded: + yield from in_build(FREETHREADED_PYTHON_DLL_NAME) else: - yield from in_build("python.exe", new_name="python") - yield from in_build("pythonw.exe", new_name="pythonw") - - yield from in_build(PYTHON_DLL_NAME) + yield from in_build(PYTHON_DLL_NAME) if ns.include_launchers and ns.include_appxmanifest: if ns.include_pip: @@ -160,7 +188,10 @@ def in_build(f, dest="", new_name=None): yield from in_build("pythonw_uwp.exe", new_name="idle{}".format(VER_DOT)) if ns.include_stable: - yield from in_build(PYTHON_STABLE_DLL_NAME) + if ns.include_freethreaded: + yield from in_build(FREETHREADED_PYTHON_STABLE_DLL_NAME) + else: + yield from in_build(PYTHON_STABLE_DLL_NAME) found_any = False for dest, src in rglob(ns.build, "vcruntime*.dll"): @@ -171,16 +202,28 @@ def in_build(f, dest="", new_name=None): yield "LICENSE.txt", ns.build / "LICENSE.txt" - for dest, src in rglob(ns.build, ("*.pyd", "*.dll")): - if src.stem.endswith("_d") != bool(ns.debug) and src not in REQUIRED_DLLS: - continue - if src in EXCLUDE_FROM_PYDS: - continue + for dest, src in rglob(ns.build, "*.pyd"): + if ns.include_freethreaded: + if not src.match("*.cp*t-win*.pyd"): + continue + if bool(src.match("*_d.cp*.pyd")) != bool(ns.debug): + continue + else: + if src.match("*.cp*t-win*.pyd"): + continue + if bool(src.match("*_d.pyd")) != bool(ns.debug): + continue if src in TEST_PYDS_ONLY and not ns.include_tests: continue if src in TCLTK_PYDS_ONLY and not ns.include_tcltk: continue + yield from in_build(src.name, dest="" if ns.flat_dlls else "DLLs/") + for dest, src in rglob(ns.build, "*.dll"): + if src.stem.endswith("_d") != bool(ns.debug) and src not in REQUIRED_DLLS: + continue + if src in EXCLUDE_FROM_DLLS: + continue yield from in_build(src.name, dest="" if ns.flat_dlls else "DLLs/") if ns.zip_lib: @@ -191,8 +234,12 @@ def in_build(f, dest="", new_name=None): yield "Lib/{}".format(dest), src if ns.include_venv: - yield from in_build("venvlauncher.exe", "Lib/venv/scripts/nt/", "python") - yield from in_build("venvwlauncher.exe", "Lib/venv/scripts/nt/", "pythonw") + if ns.include_freethreaded: + yield from in_build("venvlaunchert.exe", "Lib/venv/scripts/nt/") + yield from in_build("venvwlaunchert.exe", "Lib/venv/scripts/nt/") + else: + yield from in_build("venvlauncher.exe", "Lib/venv/scripts/nt/") + yield from in_build("venvwlauncher.exe", "Lib/venv/scripts/nt/") if ns.include_tools: @@ -208,7 +255,6 @@ def _c(d): yield PYTHON_PTH_NAME, ns.temp / PYTHON_PTH_NAME if ns.include_dev: - for dest, src in rglob(ns.source / "Include", "**/*.h"): yield "include/{}".format(dest), src yield "include/pyconfig.h", ns.build / "pyconfig.h" @@ -552,7 +598,6 @@ def main(): ns.source = ns.source or (Path(__file__).resolve().parent.parent.parent) ns.build = ns.build or Path(sys.executable).parent - ns.temp = ns.temp or Path(tempfile.mkdtemp()) ns.doc_build = ns.doc_build or (ns.source / "Doc" / "build") if not ns.source.is_absolute(): ns.source = (Path.cwd() / ns.source).resolve() @@ -565,7 +610,12 @@ def main(): if ns.include_cat and not ns.include_cat.is_absolute(): ns.include_cat = (Path.cwd() / ns.include_cat).resolve() if not ns.arch: - ns.arch = "amd64" if sys.maxsize > 2 ** 32 else "win32" + if sys.winver.endswith("-arm64"): + ns.arch = "arm64" + elif sys.winver.endswith("-32"): + ns.arch = "win32" + else: + ns.arch = "amd64" if ns.copy and not ns.copy.is_absolute(): ns.copy = (Path.cwd() / ns.copy).resolve() @@ -574,6 +624,14 @@ def main(): if ns.catalog and not ns.catalog.is_absolute(): ns.catalog = (Path.cwd() / ns.catalog).resolve() + if not ns.temp: + # Put temp on a Dev Drive for speed if we're copying to one. + # If not, the regular temp dir will have to do. + if ns.copy and getattr(os.path, "isdevdrive", lambda d: False)(ns.copy): + ns.temp = ns.copy.with_name(ns.copy.name + "_temp") + else: + ns.temp = Path(tempfile.mkdtemp()) + configure_logger(ns) log_info( @@ -602,6 +660,12 @@ def main(): log_warning("Assuming --include-tcltk to support --include-idle") ns.include_tcltk = True + if not (ns.include_alias or ns.include_alias3 or ns.include_alias3x): + if ns.include_freethreaded: + ns.include_alias3x = True + else: + ns.include_alias = True + try: generate_source_files(ns) files = list(get_layout(ns)) diff --git a/PC/layout/support/constants.py b/PC/layout/support/constants.py index 8195c3dc30cdc7..ae22aa16ebfa5d 100644 --- a/PC/layout/support/constants.py +++ b/PC/layout/support/constants.py @@ -39,3 +39,6 @@ def _get_suffix(field4): PYTHON_CHM_NAME = "python{}{}{}{}.chm".format( VER_MAJOR, VER_MINOR, VER_MICRO, VER_SUFFIX ) + +FREETHREADED_PYTHON_DLL_NAME = "python{}{}t.dll".format(VER_MAJOR, VER_MINOR) +FREETHREADED_PYTHON_STABLE_DLL_NAME = "python{}t.dll".format(VER_MAJOR) diff --git a/PC/layout/support/nuspec.py b/PC/layout/support/nuspec.py index dbcb713ef9d0c0..a87e0bea049427 100644 --- a/PC/layout/support/nuspec.py +++ b/PC/layout/support/nuspec.py @@ -24,6 +24,10 @@ amd64=("64-bit", "python", "Python"), arm32=("ARM", "pythonarm", "Python (ARM)"), arm64=("ARM64", "pythonarm64", "Python (ARM64)"), + win32t=("32-bit free-threaded", "pythonx86-freethreaded", "Python (32-bit, free-threaded)"), + amd64t=("64-bit free-threaded", "python-freethreaded", "Python (free-threaded)"), + arm32t=("ARM free-threaded", "pythonarm-freethreaded", "Python (ARM, free-threaded)"), + arm64t=("ARM64 free-threaded", "pythonarm64-freethreaded", "Python (ARM64, free-threaded)"), ) if not NUSPEC_DATA["PYTHON_VERSION"]: @@ -58,7 +62,10 @@ def _get_nuspec_data_overrides(ns): - for k, v in zip(NUSPEC_PLATFORM_DATA["_keys"], NUSPEC_PLATFORM_DATA[ns.arch]): + arch = ns.arch + if ns.include_freethreaded: + arch += "t" + for k, v in zip(NUSPEC_PLATFORM_DATA["_keys"], NUSPEC_PLATFORM_DATA[arch]): ev = os.getenv("PYTHON_NUSPEC_" + k) if ev: yield k, ev diff --git a/PC/layout/support/options.py b/PC/layout/support/options.py index 60256fb32fe329..f1a8eb0b317744 100644 --- a/PC/layout/support/options.py +++ b/PC/layout/support/options.py @@ -32,6 +32,10 @@ def public(f): "nuspec": {"help": "a python.nuspec file"}, "chm": {"help": "the CHM documentation"}, "html-doc": {"help": "the HTML documentation"}, + "freethreaded": {"help": "freethreaded binaries", "not-in-all": True}, + "alias": {"help": "aliased python.exe entry-point binaries"}, + "alias3": {"help": "aliased python3.exe entry-point binaries"}, + "alias3x": {"help": "aliased python3.x.exe entry-point binaries"}, } @@ -47,6 +51,8 @@ def public(f): "dev", "launchers", "appxmanifest", + "alias", + "alias3x", # XXX: Disabled for now "precompile", ], }, @@ -59,9 +65,10 @@ def public(f): "venv", "props", "nuspec", + "alias", ], }, - "iot": {"help": "Windows IoT Core", "options": ["stable", "pip"]}, + "iot": {"help": "Windows IoT Core", "options": ["alias", "stable", "pip"]}, "default": { "help": "development kit package", "options": [ @@ -74,11 +81,19 @@ def public(f): "dev", "symbols", "html-doc", + "alias", ], }, "embed": { "help": "embeddable package", - "options": ["stable", "zip-lib", "flat-dlls", "underpth", "precompile"], + "options": [ + "alias", + "stable", + "zip-lib", + "flat-dlls", + "underpth", + "precompile", + ], }, } diff --git a/PC/pyconfig.h.in b/PC/pyconfig.h.in index d8f0a6be69c21a..8bbf877a5bb5ed 100644 --- a/PC/pyconfig.h.in +++ b/PC/pyconfig.h.in @@ -94,6 +94,9 @@ WIN32 is still required for the locale module. #endif #endif /* Py_BUILD_CORE || Py_BUILD_CORE_BUILTIN || Py_BUILD_CORE_MODULE */ +/* Define to 1 if you want to disable the GIL */ +#undef Py_GIL_DISABLED + /* Compiler specific defines */ /* ------------------------------------------------------------------------*/ @@ -305,8 +308,16 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* not building the core - must be an ext */ # if defined(_MSC_VER) /* So MSVC users need not specify the .lib - file in their Makefile (other compilers are - generally taken care of by distutils.) */ + file in their Makefile */ +# if defined(Py_GIL_DISABLED) +# if defined(_DEBUG) +# pragma comment(lib,"python313t_d.lib") +# elif defined(Py_LIMITED_API) +# pragma comment(lib,"python3t.lib") +# else +# pragma comment(lib,"python313t.lib") +# endif /* _DEBUG */ +# else /* Py_GIL_DISABLED */ # if defined(_DEBUG) # pragma comment(lib,"python313_d.lib") # elif defined(Py_LIMITED_API) @@ -314,6 +325,7 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ # else # pragma comment(lib,"python313.lib") # endif /* _DEBUG */ +# endif /* Py_GIL_DISABLED */ # endif /* _MSC_VER */ # endif /* Py_BUILD_CORE */ #endif /* MS_COREDLL */ @@ -739,7 +751,4 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Define if libssl has X509_VERIFY_PARAM_set1_host and related function */ #define HAVE_X509_VERIFY_PARAM_SET1_HOST 1 -/* Define if you want to disable the GIL */ -#undef Py_GIL_DISABLED - #endif /* !Py_CONFIG_H */ diff --git a/PC/venvlauncher.c b/PC/venvlauncher.c new file mode 100644 index 00000000000000..fe97d32e93b5f6 --- /dev/null +++ b/PC/venvlauncher.c @@ -0,0 +1,510 @@ +/* + * venv redirector for Windows + * + * This launcher looks for a nearby pyvenv.cfg to find the correct home + * directory, and then launches the original Python executable from it. + * The name of this executable is passed as argv[0]. + */ + +#define __STDC_WANT_LIB_EXT1__ 1 + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define MS_WINDOWS +#include "patchlevel.h" + +#define MAXLEN PATHCCH_MAX_CCH +#define MSGSIZE 1024 + +#define RC_NO_STD_HANDLES 100 +#define RC_CREATE_PROCESS 101 +#define RC_NO_PYTHON 103 +#define RC_NO_MEMORY 104 +#define RC_NO_VENV_CFG 106 +#define RC_BAD_VENV_CFG 107 +#define RC_NO_COMMANDLINE 108 +#define RC_INTERNAL_ERROR 109 + +// This should always be defined when we build for real, +// but it's handy to have a definition for quick testing +#ifndef EXENAME +#define EXENAME L"python.exe" +#endif + +#ifndef CFGNAME +#define CFGNAME L"pyvenv.cfg" +#endif + +static FILE * log_fp = NULL; + +void +debug(wchar_t * format, ...) +{ + va_list va; + + if (log_fp != NULL) { + wchar_t buffer[MAXLEN]; + int r = 0; + va_start(va, format); + r = vswprintf_s(buffer, MAXLEN, format, va); + va_end(va); + + if (r <= 0) { + return; + } + fwprintf(log_fp, L"%ls\n", buffer); + while (r && isspace(buffer[r])) { + buffer[r--] = L'\0'; + } + if (buffer[0]) { + OutputDebugStringW(buffer); + } + } +} + + +void +formatWinerror(int rc, wchar_t * message, int size) +{ + FormatMessageW( + FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, rc, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), + message, size, NULL); +} + + +void +winerror(int err, wchar_t * format, ... ) +{ + va_list va; + wchar_t message[MSGSIZE]; + wchar_t win_message[MSGSIZE]; + int len; + + if (err == 0) { + err = GetLastError(); + } + + va_start(va, format); + len = _vsnwprintf_s(message, MSGSIZE, _TRUNCATE, format, va); + va_end(va); + + formatWinerror(err, win_message, MSGSIZE); + if (len >= 0) { + _snwprintf_s(&message[len], MSGSIZE - len, _TRUNCATE, L": %ls", + win_message); + } + +#if !defined(_WINDOWS) + fwprintf(stderr, L"%ls\n", message); +#else + MessageBoxW(NULL, message, L"Python venv launcher is sorry to say ...", + MB_OK); +#endif +} + + +void +error(wchar_t * format, ... ) +{ + va_list va; + wchar_t message[MSGSIZE]; + + va_start(va, format); + _vsnwprintf_s(message, MSGSIZE, _TRUNCATE, format, va); + va_end(va); + +#if !defined(_WINDOWS) + fwprintf(stderr, L"%ls\n", message); +#else + MessageBoxW(NULL, message, L"Python venv launcher is sorry to say ...", + MB_OK); +#endif +} + + +bool +isEnvVarSet(const wchar_t *name) +{ + /* only looking for non-empty, which means at least one character + and the null terminator */ + return GetEnvironmentVariableW(name, NULL, 0) >= 2; +} + + +bool +join(wchar_t *buffer, size_t bufferLength, const wchar_t *fragment) +{ + if (SUCCEEDED(PathCchCombineEx(buffer, bufferLength, buffer, fragment, PATHCCH_ALLOW_LONG_PATHS))) { + return true; + } + return false; +} + + +bool +split_parent(wchar_t *buffer, size_t bufferLength) +{ + return SUCCEEDED(PathCchRemoveFileSpec(buffer, bufferLength)); +} + + +/* + * Path calculation + */ + +int +calculate_pyvenvcfg_path(wchar_t *pyvenvcfg_path, size_t maxlen) +{ + if (!pyvenvcfg_path) { + error(L"invalid buffer provided"); + return RC_INTERNAL_ERROR; + } + if ((DWORD)maxlen != maxlen) { + error(L"path buffer is too large"); + return RC_INTERNAL_ERROR; + } + if (!GetModuleFileNameW(NULL, pyvenvcfg_path, (DWORD)maxlen)) { + winerror(GetLastError(), L"failed to read executable directory"); + return RC_NO_COMMANDLINE; + } + // Remove 'python.exe' from our path + if (!split_parent(pyvenvcfg_path, maxlen)) { + error(L"failed to remove segment from '%ls'", pyvenvcfg_path); + return RC_NO_COMMANDLINE; + } + // Replace with 'pyvenv.cfg' + if (!join(pyvenvcfg_path, maxlen, CFGNAME)) { + error(L"failed to append '%ls' to '%ls'", CFGNAME, pyvenvcfg_path); + return RC_NO_MEMORY; + } + // If it exists, return + if (GetFileAttributesW(pyvenvcfg_path) != INVALID_FILE_ATTRIBUTES) { + return 0; + } + // Otherwise, remove 'pyvenv.cfg' and (probably) 'Scripts' + if (!split_parent(pyvenvcfg_path, maxlen) || + !split_parent(pyvenvcfg_path, maxlen)) { + error(L"failed to remove segments from '%ls'", pyvenvcfg_path); + return RC_NO_COMMANDLINE; + } + // Replace 'pyvenv.cfg' + if (!join(pyvenvcfg_path, maxlen, CFGNAME)) { + error(L"failed to append '%ls' to '%ls'", CFGNAME, pyvenvcfg_path); + return RC_NO_MEMORY; + } + // If it exists, return + if (GetFileAttributesW(pyvenvcfg_path) != INVALID_FILE_ATTRIBUTES) { + return 0; + } + // Otherwise, we fail + winerror(GetLastError(), L"failed to locate %ls", CFGNAME); + return RC_NO_VENV_CFG; +} + + +/* + * pyvenv.cfg parsing + */ + +static int +find_home_value(const char *buffer, DWORD maxlen, const char **start, DWORD *length) +{ + if (!buffer || !start || !length) { + error(L"invalid find_home_value parameters()"); + return 0; + } + for (const char *s = strstr(buffer, "home"); + s && ((ptrdiff_t)s - (ptrdiff_t)buffer) < maxlen; + s = strstr(s + 1, "\nhome") + ) { + if (*s == '\n') { + ++s; + } + for (int i = 4; i > 0 && *s; --i, ++s); + + while (*s && iswspace(*s)) { + ++s; + } + if (*s != L'=') { + continue; + } + + do { + ++s; + } while (*s && iswspace(*s)); + + *start = s; + char *nl = strchr(s, '\n'); + if (nl) { + while (nl != s && iswspace(nl[-1])) { + --nl; + } + *length = (DWORD)((ptrdiff_t)nl - (ptrdiff_t)s); + } else { + *length = (DWORD)strlen(s); + } + return 1; + } + return 0; +} + + +int +read_home(const wchar_t *pyvenv_cfg, wchar_t *home_path, size_t maxlen) +{ + HANDLE hFile = CreateFileW(pyvenv_cfg, GENERIC_READ, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + NULL, OPEN_EXISTING, 0, NULL); + + if (hFile == INVALID_HANDLE_VALUE) { + winerror(GetLastError(), L"failed to open '%ls'", pyvenv_cfg); + return RC_BAD_VENV_CFG; + } + + // 8192 characters ought to be enough for anyone + // (doubled compared to the old implementation!) + char buffer[8192]; + DWORD len; + if (!ReadFile(hFile, buffer, sizeof(buffer) - 1, &len, NULL)) { + winerror(GetLastError(), L"failed to read '%ls'", pyvenv_cfg); + CloseHandle(hFile); + return RC_BAD_VENV_CFG; + } + CloseHandle(hFile); + // Ensure null termination + buffer[len] = '\0'; + + char *home; + DWORD home_len; + if (!find_home_value(buffer, sizeof(buffer), &home, &home_len)) { + error(L"no home= specified in '%ls'", pyvenv_cfg); + return RC_BAD_VENV_CFG; + } + + if ((DWORD)maxlen != maxlen) { + maxlen = 8192; + } + len = MultiByteToWideChar(CP_UTF8, 0, home, home_len, home_path, (DWORD)maxlen); + if (!len) { + winerror(GetLastError(), L"failed to decode home setting in '%ls'", pyvenv_cfg); + return RC_BAD_VENV_CFG; + } + home_path[len] = L'\0'; + + return 0; +} + + +int +locate_python(wchar_t *path, size_t maxlen) +{ + if (!join(path, maxlen, EXENAME)) { + error(L"failed to append %ls to '%ls'", EXENAME, path); + return RC_NO_MEMORY; + } + + if (GetFileAttributesW(path) == INVALID_FILE_ATTRIBUTES) { + winerror(GetLastError(), L"did not find executable at '%ls'", path); + return RC_NO_PYTHON; + } + + return 0; +} + + +int +smuggle_path() +{ + wchar_t buffer[MAXLEN]; + // We could use argv[0], but that may be wrong in certain rare cases (if the + // user is doing something weird like symlinks to venv redirectors), and + // what we _really_ want is the directory of the venv. We always copy the + // redirectors, so if we've made the venv, this will be correct. + DWORD len = GetModuleFileNameW(NULL, buffer, MAXLEN); + if (!len) { + winerror(GetLastError(), L"Failed to get own executable path"); + return RC_INTERNAL_ERROR; + } + buffer[len] = L'\0'; + debug(L"Setting __PYVENV_LAUNCHER__ = '%s'", buffer); + + if (!SetEnvironmentVariableW(L"__PYVENV_LAUNCHER__", buffer)) { + winerror(GetLastError(), L"Failed to set launcher environment"); + return RC_INTERNAL_ERROR; + } + + return 0; +} + +/* + * Process creation + */ + +static BOOL +safe_duplicate_handle(HANDLE in, HANDLE * pout, const wchar_t *name) +{ + BOOL ok; + HANDLE process = GetCurrentProcess(); + DWORD rc; + + *pout = NULL; + ok = DuplicateHandle(process, in, process, pout, 0, TRUE, + DUPLICATE_SAME_ACCESS); + if (!ok) { + rc = GetLastError(); + if (rc == ERROR_INVALID_HANDLE) { + debug(L"DuplicateHandle(%ls) returned ERROR_INVALID_HANDLE\n", name); + ok = TRUE; + } + else { + debug(L"DuplicateHandle(%ls) returned %d\n", name, rc); + } + } + return ok; +} + +static BOOL WINAPI +ctrl_c_handler(DWORD code) +{ + return TRUE; /* We just ignore all control events. */ +} + +static int +launch(const wchar_t *executable, wchar_t *cmdline) +{ + HANDLE job; + JOBOBJECT_EXTENDED_LIMIT_INFORMATION info; + DWORD rc; + BOOL ok; + STARTUPINFOW si; + PROCESS_INFORMATION pi; + +#if defined(_WINDOWS) + /* + When explorer launches a Windows (GUI) application, it displays + the "app starting" (the "pointer + hourglass") cursor for a number + of seconds, or until the app does something UI-ish (eg, creating a + window, or fetching a message). As this launcher doesn't do this + directly, that cursor remains even after the child process does these + things. We avoid that by doing a simple post+get message. + See http://bugs.python.org/issue17290 + */ + MSG msg; + + PostMessage(0, 0, 0, 0); + GetMessage(&msg, 0, 0, 0); +#endif + + debug(L"run_child: about to run '%ls' with '%ls'\n", executable, cmdline); + job = CreateJobObject(NULL, NULL); + ok = QueryInformationJobObject(job, JobObjectExtendedLimitInformation, + &info, sizeof(info), &rc); + if (!ok || (rc != sizeof(info)) || !job) { + winerror(GetLastError(), L"Job information querying failed"); + return RC_CREATE_PROCESS; + } + info.BasicLimitInformation.LimitFlags |= JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE | + JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK; + ok = SetInformationJobObject(job, JobObjectExtendedLimitInformation, &info, + sizeof(info)); + if (!ok) { + winerror(GetLastError(), L"Job information setting failed"); + return RC_CREATE_PROCESS; + } + memset(&si, 0, sizeof(si)); + GetStartupInfoW(&si); + ok = safe_duplicate_handle(GetStdHandle(STD_INPUT_HANDLE), &si.hStdInput, L"stdin"); + if (!ok) { + return RC_NO_STD_HANDLES; + } + ok = safe_duplicate_handle(GetStdHandle(STD_OUTPUT_HANDLE), &si.hStdOutput, L"stdout"); + if (!ok) { + return RC_NO_STD_HANDLES; + } + ok = safe_duplicate_handle(GetStdHandle(STD_ERROR_HANDLE), &si.hStdError, L"stderr"); + if (!ok) { + return RC_NO_STD_HANDLES; + } + + ok = SetConsoleCtrlHandler(ctrl_c_handler, TRUE); + if (!ok) { + winerror(GetLastError(), L"control handler setting failed"); + return RC_CREATE_PROCESS; + } + + si.dwFlags = STARTF_USESTDHANDLES; + ok = CreateProcessW(executable, cmdline, NULL, NULL, TRUE, + 0, NULL, NULL, &si, &pi); + if (!ok) { + winerror(GetLastError(), L"Unable to create process using '%ls'", cmdline); + return RC_CREATE_PROCESS; + } + AssignProcessToJobObject(job, pi.hProcess); + CloseHandle(pi.hThread); + WaitForSingleObjectEx(pi.hProcess, INFINITE, FALSE); + ok = GetExitCodeProcess(pi.hProcess, &rc); + if (!ok) { + winerror(GetLastError(), L"Failed to get exit code of process"); + return RC_CREATE_PROCESS; + } + debug(L"child process exit code: %d", rc); + return rc; +} + + +int +process(int argc, wchar_t ** argv) +{ + int exitCode; + wchar_t pyvenvcfg_path[MAXLEN]; + wchar_t home_path[MAXLEN]; + + if (isEnvVarSet(L"PYLAUNCHER_DEBUG")) { + setvbuf(stderr, (char *)NULL, _IONBF, 0); + log_fp = stderr; + } + + exitCode = calculate_pyvenvcfg_path(pyvenvcfg_path, MAXLEN); + if (exitCode) return exitCode; + + exitCode = read_home(pyvenvcfg_path, home_path, MAXLEN); + if (exitCode) return exitCode; + + exitCode = locate_python(home_path, MAXLEN); + if (exitCode) return exitCode; + + // We do not update argv[0] to point at the target runtime, and so we do not + // pass through our original argv[0] in an environment variable. + //exitCode = smuggle_path(); + //if (exitCode) return exitCode; + + exitCode = launch(home_path, GetCommandLineW()); + return exitCode; +} + + +#if defined(_WINDOWS) + +int WINAPI wWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, + LPWSTR lpstrCmd, int nShow) +{ + return process(__argc, __wargv); +} + +#else + +int cdecl wmain(int argc, wchar_t ** argv) +{ + return process(argc, argv); +} + +#endif diff --git a/PCbuild/_asyncio.vcxproj b/PCbuild/_asyncio.vcxproj index ed1e1bc0a420dc..76b0ffd660dba0 100644 --- a/PCbuild/_asyncio.vcxproj +++ b/PCbuild/_asyncio.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_bz2.vcxproj b/PCbuild/_bz2.vcxproj index 3fe95fbf83993a..e0dc6ec187a08d 100644 --- a/PCbuild/_bz2.vcxproj +++ b/PCbuild/_bz2.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_ctypes.vcxproj b/PCbuild/_ctypes.vcxproj index 253da31e9ce182..63d5fa49cd4e17 100644 --- a/PCbuild/_ctypes.vcxproj +++ b/PCbuild/_ctypes.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_ctypes_test.vcxproj b/PCbuild/_ctypes_test.vcxproj index 8a01e743a4d86f..97354739c09834 100644 --- a/PCbuild/_ctypes_test.vcxproj +++ b/PCbuild/_ctypes_test.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_decimal.vcxproj b/PCbuild/_decimal.vcxproj index 0916f1a2d37887..490d7df87eb1c6 100644 --- a/PCbuild/_decimal.vcxproj +++ b/PCbuild/_decimal.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_elementtree.vcxproj b/PCbuild/_elementtree.vcxproj index 8da5244bac0cb6..8c9c0e42f7fe3e 100644 --- a/PCbuild/_elementtree.vcxproj +++ b/PCbuild/_elementtree.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_hashlib.vcxproj b/PCbuild/_hashlib.vcxproj index 6dad8183c57ae3..2cd205224bc089 100644 --- a/PCbuild/_hashlib.vcxproj +++ b/PCbuild/_hashlib.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_lzma.vcxproj b/PCbuild/_lzma.vcxproj index fe076a6fc57168..40107d4b76cd53 100644 --- a/PCbuild/_lzma.vcxproj +++ b/PCbuild/_lzma.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_multiprocessing.vcxproj b/PCbuild/_multiprocessing.vcxproj index 77b6bfc8e1e483..a65397f532aa86 100644 --- a/PCbuild/_multiprocessing.vcxproj +++ b/PCbuild/_multiprocessing.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_overlapped.vcxproj b/PCbuild/_overlapped.vcxproj index 9e60d3b5db336c..224bf05d5303a0 100644 --- a/PCbuild/_overlapped.vcxproj +++ b/PCbuild/_overlapped.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_queue.vcxproj b/PCbuild/_queue.vcxproj index 8065b235851686..80a1c3c6a4ad3e 100644 --- a/PCbuild/_queue.vcxproj +++ b/PCbuild/_queue.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_socket.vcxproj b/PCbuild/_socket.vcxproj index 78fa4d6729abb9..41af0895921bbb 100644 --- a/PCbuild/_socket.vcxproj +++ b/PCbuild/_socket.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_sqlite3.vcxproj b/PCbuild/_sqlite3.vcxproj index 57c7413671e54e..9ae0a0fc3a009d 100644 --- a/PCbuild/_sqlite3.vcxproj +++ b/PCbuild/_sqlite3.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_ssl.vcxproj b/PCbuild/_ssl.vcxproj index 226ff506f8c62b..d4e1affab031d7 100644 --- a/PCbuild/_ssl.vcxproj +++ b/PCbuild/_ssl.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testbuffer.vcxproj b/PCbuild/_testbuffer.vcxproj index 917d7ae50feb14..4e721e8ce09f0c 100644 --- a/PCbuild/_testbuffer.vcxproj +++ b/PCbuild/_testbuffer.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testcapi.vcxproj b/PCbuild/_testcapi.vcxproj index 1c15541d3ec735..6911aacab29b97 100644 --- a/PCbuild/_testcapi.vcxproj +++ b/PCbuild/_testcapi.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testclinic.vcxproj b/PCbuild/_testclinic.vcxproj index e319b3c0f42e0f..ef981332c6ab03 100644 --- a/PCbuild/_testclinic.vcxproj +++ b/PCbuild/_testclinic.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) @@ -107,4 +107,4 @@ - \ No newline at end of file + diff --git a/PCbuild/_testclinic_limited.vcxproj b/PCbuild/_testclinic_limited.vcxproj index b00b2be491b423..183a55080e8693 100644 --- a/PCbuild/_testclinic_limited.vcxproj +++ b/PCbuild/_testclinic_limited.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testconsole.vcxproj b/PCbuild/_testconsole.vcxproj index 5d7e14eff10294..69d312b17a5a62 100644 --- a/PCbuild/_testconsole.vcxproj +++ b/PCbuild/_testconsole.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testimportmultiple.vcxproj b/PCbuild/_testimportmultiple.vcxproj index 6d80d5779f24d8..c35ac83c1c739f 100644 --- a/PCbuild/_testimportmultiple.vcxproj +++ b/PCbuild/_testimportmultiple.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testinternalcapi.vcxproj b/PCbuild/_testinternalcapi.vcxproj index 558f66ca95cd33..a825cac9138674 100644 --- a/PCbuild/_testinternalcapi.vcxproj +++ b/PCbuild/_testinternalcapi.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testmultiphase.vcxproj b/PCbuild/_testmultiphase.vcxproj index 430eb528cc3927..e730fe308ab835 100644 --- a/PCbuild/_testmultiphase.vcxproj +++ b/PCbuild/_testmultiphase.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_testsinglephase.vcxproj b/PCbuild/_testsinglephase.vcxproj index fb4bcd953923f8..bf4dabf66c1040 100644 --- a/PCbuild/_testsinglephase.vcxproj +++ b/PCbuild/_testsinglephase.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_tkinter.vcxproj b/PCbuild/_tkinter.vcxproj index 30cedcbb43de76..117488a01621cc 100644 --- a/PCbuild/_tkinter.vcxproj +++ b/PCbuild/_tkinter.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_uuid.vcxproj b/PCbuild/_uuid.vcxproj index 2437b7eb2d9399..50d81cc7916dbd 100644 --- a/PCbuild/_uuid.vcxproj +++ b/PCbuild/_uuid.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_wmi.vcxproj b/PCbuild/_wmi.vcxproj index c1914a3fa5a1bf..22fa8960982528 100644 --- a/PCbuild/_wmi.vcxproj +++ b/PCbuild/_wmi.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/_zoneinfo.vcxproj b/PCbuild/_zoneinfo.vcxproj index 6e6389c3773397..47b5bfa5b8815a 100644 --- a/PCbuild/_zoneinfo.vcxproj +++ b/PCbuild/_zoneinfo.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/pyexpat.vcxproj b/PCbuild/pyexpat.vcxproj index 001f8afd89b9e9..dc9161a8b290f9 100644 --- a/PCbuild/pyexpat.vcxproj +++ b/PCbuild/pyexpat.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props index 16ad91ef0278c8..fd5fbc9e910eee 100644 --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -24,6 +24,13 @@ false + + $(TargetName)$(TargetExt) + <_TargetNameSep>$(TargetNameExt.LastIndexOf(`.`)) + $(TargetNameExt.Substring(0, $(_TargetNameSep))) + $(TargetNameExt.Substring($(_TargetNameSep))) + + <_VCToolsVersion>$([System.Version]::Parse(`$(VCToolsVersion)`).Major).$([System.Version]::Parse(`$(VCToolsVersion)`).Minor) @@ -38,7 +45,7 @@ <_PlatformPreprocessorDefinition>_WIN32; <_PlatformPreprocessorDefinition Condition="$(Platform) == 'x64'">_WIN64; <_PlatformPreprocessorDefinition Condition="$(Platform) == 'x64' and $(PlatformToolset) != 'ClangCL'">_M_X64;$(_PlatformPreprocessorDefinition) - <_Py3NamePreprocessorDefinition>PY3_DLLNAME=L"$(Py3DllName)"; + <_Py3NamePreprocessorDefinition>PY3_DLLNAME=L"$(Py3DllName)$(PyDebugExt)"; @@ -158,8 +165,8 @@ public override bool Execute() { - - + + $([System.IO.Path]::GetFullPath($(MSBuildThisFileDirectory)\..\)) $(PySourcePath)\ - + $(PySourcePath)PCbuild\win32\ $(Py_OutDir)\win32\ @@ -52,7 +52,7 @@ $(PySourcePath)PCbuild\$(ArchName)\ $(BuildPath)\ $(BuildPath)instrumented\ - + ..\\.. ..\\..\\.. @@ -84,22 +84,19 @@ _d - + -test - + -32 -arm32 -arm64 - - - $(BuildPath)python$(PyDebugExt).exe true - + true @@ -141,7 +138,7 @@ @@ -223,22 +220,55 @@ )) )) $([msbuild]::Add($(Field3Value), 9000)) - + + + python$(MajorVersionNumber).$(MinorVersionNumber)t + python + $(BuildPath)$(PyExeName)$(PyDebugExt).exe + pythonw$(MajorVersionNumber).$(MinorVersionNumber)t + pythonw + - python$(MajorVersionNumber)$(MinorVersionNumber)$(PyDebugExt) + python$(MajorVersionNumber)$(MinorVersionNumber)t$(PyDebugExt) + python$(MajorVersionNumber)$(MinorVersionNumber)$(PyDebugExt) - python3$(PyDebugExt) + python3t + python3 .cp$(MajorVersionNumber)$(MinorVersionNumber)-win32 .cp$(MajorVersionNumber)$(MinorVersionNumber)-win_arm32 .cp$(MajorVersionNumber)$(MinorVersionNumber)-win_arm64 .cp$(MajorVersionNumber)$(MinorVersionNumber)-win_amd64 - + $(MajorVersionNumber).$(MinorVersionNumber)$(PyArchExt)$(PyTestExt) + + + .cp$(MajorVersionNumber)$(MinorVersionNumber)t-win32 + .cp$(MajorVersionNumber)$(MinorVersionNumber)t-win_arm32 + .cp$(MajorVersionNumber)$(MinorVersionNumber)t-win_arm64 + .cp$(MajorVersionNumber)$(MinorVersionNumber)t-win_amd64 + + + $(MajorVersionNumber).$(MinorVersionNumber)t$(PyArchExt)$(PyTestExt) + + + + + .pyd - + + + $(FreethreadedPydTag) + + + $(PydTag).pyd + + + $(FreethreadedSysWinVer) + + diff --git a/PCbuild/python.vcxproj b/PCbuild/python.vcxproj index fdf573aa5bf983..1e5ab877488e4a 100644 --- a/PCbuild/python.vcxproj +++ b/PCbuild/python.vcxproj @@ -72,6 +72,7 @@ + $(PyExeName) Application false MultiByte @@ -132,7 +133,7 @@ +"$(OutDir)$(PyExeName)$(PyDebugExt).exe" "$(PySourcePath)PC\validate_ucrtbase.py" $(UcrtName)' ContinueOnError="true" /> @@ -145,7 +146,7 @@ set PYTHONPATH=$(PySourcePath)Lib @echo Running $(Configuration)^|$(Platform) interpreter... @setlocal @set PYTHONHOME=$(PySourcePath) -@"$(OutDir)python$(PyDebugExt).exe" %* +@"$(OutDir)$(PyExeName)$(PyDebugExt).exe" %* <_ExistingContent Condition="Exists('$(PySourcePath)python.bat')">$([System.IO.File]::ReadAllText('$(PySourcePath)python.bat')) diff --git a/PCbuild/python3dll.vcxproj b/PCbuild/python3dll.vcxproj index ec22e6fc76e584..235ea1cf9d33fb 100644 --- a/PCbuild/python3dll.vcxproj +++ b/PCbuild/python3dll.vcxproj @@ -70,12 +70,12 @@ {885D4898-D08D-4091-9C40-C700CFE3FC5A} python3dll Win32Proj - python3 false + $(Py3DllName) DynamicLibrary diff --git a/PCbuild/pythonw.vcxproj b/PCbuild/pythonw.vcxproj index 31f21308e25fb3..d6cf0c97dedb09 100644 --- a/PCbuild/pythonw.vcxproj +++ b/PCbuild/pythonw.vcxproj @@ -73,6 +73,7 @@ + $(PyWExeName) Application false diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat index 332ba5edcf4082..293f99ae135faa 100644 --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -9,6 +9,7 @@ rem -q runs the tests just once, and without deleting .pyc files. rem -p or -win32, -x64, -arm32, -arm64 rem Run the specified architecture of python (or python_d if -d rem was specified). If omitted, uses %PREFIX% if set or 64-bit. +rem --disable-gil Run free-threaded build. rem All leading instances of these switches are shifted off, and rem whatever remains (up to 9 arguments) is passed to regrtest.py. rem For example, @@ -29,6 +30,7 @@ rem rt -u "network,largefile" setlocal set pcbuild=%~dp0 +set pyname=python set suffix= set qmode= set dashO= @@ -39,15 +41,18 @@ set exe= if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts +rem HACK: Need some way to infer the version number in this script +if "%1"=="--disable-gil" (set pyname=python3.13t) & shift & goto CheckOpts if "%1"=="-win32" (set prefix=%pcbuild%win32) & shift & goto CheckOpts if "%1"=="-x64" (set prefix=%pcbuild%amd64) & shift & goto CheckOpts +if "%1"=="-amd64" (set prefix=%pcbuild%amd64) & shift & goto CheckOpts if "%1"=="-arm64" (set prefix=%pcbuild%arm64) & shift & goto CheckOpts if "%1"=="-arm32" (set prefix=%pcbuild%arm32) & shift & goto CheckOpts if "%1"=="-p" (call :SetPlatform %~2) & shift & shift & goto CheckOpts if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto CheckOpts if not defined prefix set prefix=%pcbuild%amd64 -set exe=%prefix%\python%suffix%.exe +set exe=%prefix%\%pyname%%suffix%.exe set cmd="%exe%" %dashO% -m test %regrtestargs% if defined qmode goto Qmode diff --git a/PCbuild/select.vcxproj b/PCbuild/select.vcxproj index 750a713949919a..d7448fd4d72380 100644 --- a/PCbuild/select.vcxproj +++ b/PCbuild/select.vcxproj @@ -78,7 +78,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/sqlite3.vcxproj b/PCbuild/sqlite3.vcxproj index c502d51833b91a..6bcc4e913c8e77 100644 --- a/PCbuild/sqlite3.vcxproj +++ b/PCbuild/sqlite3.vcxproj @@ -69,12 +69,12 @@ {A1A295E5-463C-437F-81CA-1F32367685DA} sqlite3 - .pyd false + $(PyStdlibPydExt) DynamicLibrary NotSet diff --git a/PCbuild/unicodedata.vcxproj b/PCbuild/unicodedata.vcxproj index addef753359ed6..781f938e2ab78e 100644 --- a/PCbuild/unicodedata.vcxproj +++ b/PCbuild/unicodedata.vcxproj @@ -79,7 +79,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/venvlauncher.vcxproj b/PCbuild/venvlauncher.vcxproj index 123e84ec4e3682..1193e032245c94 100644 --- a/PCbuild/venvlauncher.vcxproj +++ b/PCbuild/venvlauncher.vcxproj @@ -69,12 +69,13 @@ {494BAC80-A60C-43A9-99E7-ACB691CE2C4D} venvlauncher - venvlauncher false + venvlauncher + $(TargetName)t Application MultiByte @@ -91,19 +92,19 @@ - _CONSOLE;VENV_REDIRECT;%(PreprocessorDefinitions) + EXENAME=L"$(PyExeName)$(PyDebugExt).exe";_CONSOLE;%(PreprocessorDefinitions) MultiThreaded PY_ICON;%(PreprocessorDefinitions) - version.lib;%(AdditionalDependencies) + pathcch.lib;%(AdditionalDependencies) Console - + diff --git a/PCbuild/venvlauncher.vcxproj.filters b/PCbuild/venvlauncher.vcxproj.filters index ec13936bf6cb7e..56a0f005a3fa2a 100644 --- a/PCbuild/venvlauncher.vcxproj.filters +++ b/PCbuild/venvlauncher.vcxproj.filters @@ -19,7 +19,7 @@ - + Source Files diff --git a/PCbuild/venvwlauncher.vcxproj b/PCbuild/venvwlauncher.vcxproj index b8504d5d08e52f..1b61718201367f 100644 --- a/PCbuild/venvwlauncher.vcxproj +++ b/PCbuild/venvwlauncher.vcxproj @@ -69,12 +69,13 @@ {FDB84CBB-2FB6-47C8-A2D6-091E0833239D} venvwlauncher - venvwlauncher false + venvwlauncher + $(TargetName)t Application MultiByte @@ -91,19 +92,19 @@ - _WINDOWS;VENV_REDIRECT;%(PreprocessorDefinitions) + EXENAME=L"$(PyExeName)$(PyDebugExt).exe";_WINDOWS;%(PreprocessorDefinitions) MultiThreaded PYW_ICON;%(PreprocessorDefinitions) - version.lib;%(AdditionalDependencies) + pathcch.lib;%(AdditionalDependencies) Windows - + diff --git a/PCbuild/venvwlauncher.vcxproj.filters b/PCbuild/venvwlauncher.vcxproj.filters index 8addc13e977e7a..61a514395e82dc 100644 --- a/PCbuild/venvwlauncher.vcxproj.filters +++ b/PCbuild/venvwlauncher.vcxproj.filters @@ -9,7 +9,7 @@ - + Source Files diff --git a/PCbuild/winsound.vcxproj b/PCbuild/winsound.vcxproj index 32cedc9b444902..c26029b15a339f 100644 --- a/PCbuild/winsound.vcxproj +++ b/PCbuild/winsound.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/xxlimited.vcxproj b/PCbuild/xxlimited.vcxproj index 1c776fb0da3e72..093e6920c0b76c 100644 --- a/PCbuild/xxlimited.vcxproj +++ b/PCbuild/xxlimited.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/PCbuild/xxlimited_35.vcxproj b/PCbuild/xxlimited_35.vcxproj index dd830b3b6aaa91..3f4d4463f24af0 100644 --- a/PCbuild/xxlimited_35.vcxproj +++ b/PCbuild/xxlimited_35.vcxproj @@ -80,7 +80,7 @@ - .pyd + $(PyStdlibPydExt) diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h index a3606b17b71c62..c2550f53ad6eaa 100644 --- a/Python/ceval_macros.h +++ b/Python/ceval_macros.h @@ -382,9 +382,6 @@ static inline void _Py_LeaveRecursiveCallPy(PyThreadState *tstate) { /* There's no STORE_IP(), it's inlined by the code generator. */ -#define STORE_SP() \ -_PyFrame_SetStackPointer(frame, stack_pointer) - #define LOAD_SP() \ stack_pointer = _PyFrame_GetStackPointer(frame); diff --git a/Python/gc_free_threading.c b/Python/gc_free_threading.c index b1511eb5a70e7e..207a43b68d21f5 100644 --- a/Python/gc_free_threading.c +++ b/Python/gc_free_threading.c @@ -15,7 +15,6 @@ void _PyGC_ClearAllFreeLists(PyInterpreterState *interp) { _PyDict_ClearFreeList(interp); - _PyAsyncGen_ClearFreeLists(interp); HEAD_LOCK(&_PyRuntime); _PyThreadStateImpl *tstate = (_PyThreadStateImpl *)interp->threads.head; diff --git a/Python/gc_gil.c b/Python/gc_gil.c index edf84176f79e0d..04c1c184250c60 100644 --- a/Python/gc_gil.c +++ b/Python/gc_gil.c @@ -12,7 +12,6 @@ void _PyGC_ClearAllFreeLists(PyInterpreterState *interp) { _PyDict_ClearFreeList(interp); - _PyAsyncGen_ClearFreeLists(interp); _Py_ClearFreeLists(&interp->freelist_state, 0); } diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 598cd68806e9be..0d5eec06e9b458 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1735,7 +1735,6 @@ finalize_interp_types(PyInterpreterState *interp) _PySys_FiniTypes(interp); _PyXI_FiniTypes(interp); _PyExc_Fini(interp); - _PyAsyncGen_Fini(interp); _PyFloat_FiniType(interp); _PyLong_FiniTypes(interp); _PyThread_FiniType(interp); @@ -1759,6 +1758,7 @@ finalize_interp_types(PyInterpreterState *interp) _PyFloat_Fini(state); _PySlice_Fini(state); _PyContext_Fini(state); + _PyAsyncGen_Fini(state); #ifdef Py_DEBUG _PyStaticObjects_CheckRefcnt(interp); diff --git a/Python/pystate.c b/Python/pystate.c index 8374223fbf3b19..999976283da675 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1462,6 +1462,7 @@ _Py_ClearFreeLists(_PyFreeListState *state, int is_finalization) _PyTuple_ClearFreeList(state, is_finalization); _PyList_ClearFreeList(state, is_finalization); _PyContext_ClearFreeList(state, is_finalization); + _PyAsyncGen_ClearFreeLists(state, is_finalization); } void @@ -1549,7 +1550,7 @@ PyThreadState_Clear(PyThreadState *tstate) #ifdef Py_GIL_DISABLED // Each thread should clear own freelists in free-threading builds. _PyFreeListState *freelist_state = &((_PyThreadStateImpl*)tstate)->freelist_state; - _Py_ClearFreeLists(freelist_state, 0); + _Py_ClearFreeLists(freelist_state, 1); _PySlice_ClearCache(freelist_state); #endif diff --git a/Tools/build/generate_sbom.py b/Tools/build/generate_sbom.py index 282ee20cc402b0..317d48fee3a9d4 100644 --- a/Tools/build/generate_sbom.py +++ b/Tools/build/generate_sbom.py @@ -106,6 +106,7 @@ def filter_gitignored_paths(paths: list[str]) -> list[str]: # Non-matching files show up as '::' git_check_ignore_proc = subprocess.run( ["git", "check-ignore", "--verbose", "--non-matching", *paths], + cwd=CPYTHON_ROOT_DIR, check=False, stdout=subprocess.PIPE, ) diff --git a/Tools/msi/build.bat b/Tools/msi/build.bat index b9aab887c4939b..2fe8a475e7e3a3 100644 --- a/Tools/msi/build.bat +++ b/Tools/msi/build.bat @@ -22,6 +22,9 @@ if "%~1" EQU "--no-test-marker" (set BUILDTEST=) && shift && goto CheckOpts if "%~1" EQU "--test-marker" (set BUILDTEST=--test-marker) && shift && goto CheckOpts if "%~1" EQU "--pack" (set BUILDPACK=1) && shift && goto CheckOpts if "%~1" EQU "-r" (set REBUILD=-r) && shift && goto CheckOpts +rem %IncludeFreethreaded% is recognised by the MSI build, but not the regular build. +rem We use it to build twice and then build the installer with its extra option +if /I "%~1" EQU "--disable-gil" (set IncludeFreethreaded=true) && shift && goto CheckOpts if not defined BUILDX86 if not defined BUILDX64 if not defined BUILDARM64 (set BUILDX86=1) && (set BUILDX64=1) @@ -44,6 +47,20 @@ if errorlevel 1 exit /B %ERRORLEVEL% if defined BUILDARM64 call "%PCBUILD%build.bat" -p ARM64 -e %REBUILD% %BUILDTEST% if errorlevel 1 exit /B %ERRORLEVEL% +if /I "%IncludeFreethreaded%"=="true" ( + rem Cannot "exit /B" inside an if block because %ERRORLEVEL% will be wrong. + rem We just skip everything after the first "errorlevel 1" and then exit after + if defined BUILDX86 call "%PCBUILD%build.bat" -p Win32 -d -e %REBUILD% %BUILDTEST% --disable-gil + if not errorlevel 1 if defined BUILDX86 call "%PCBUILD%build.bat" -p Win32 -e %REBUILD% %BUILDTEST% --disable-gil + + if not errorlevel 1 if defined BUILDX64 call "%PCBUILD%build.bat" -p x64 -d -e %REBUILD% %BUILDTEST% --disable-gil + if not errorlevel 1 if defined BUILDX64 call "%PCBUILD%build.bat" -p x64 -e %REBUILD% %BUILDTEST% --disable-gil + + if not errorlevel 1 if defined BUILDARM64 call "%PCBUILD%build.bat" -p ARM64 -d -e %REBUILD% %BUILDTEST% --disable-gil + if not errorlevel 1 if defined BUILDARM64 call "%PCBUILD%build.bat" -p ARM64 -e %REBUILD% %BUILDTEST% --disable-gil +) +if errorlevel 1 exit /B %ERRORLEVEL% + if defined BUILDDOC call "%PCBUILD%..\Doc\make.bat" html if errorlevel 1 exit /B %ERRORLEVEL% diff --git a/Tools/msi/bundle/Default.thm b/Tools/msi/bundle/Default.thm index d1b0f5bd9c1d4a..471d37acc33b3d 100644 --- a/Tools/msi/bundle/Default.thm +++ b/Tools/msi/bundle/Default.thm @@ -83,6 +83,7 @@ #(loc.PrecompileLabel) #(loc.Include_symbolsLabel) #(loc.Include_debugLabel) + #(loc.Include_freethreadedLabel) #(loc.CustomLocationLabel) diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl index 6f8befba3a2523..1540f050159a54 100644 --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -91,6 +91,7 @@ Select Customize to review current options. &Precompile standard library Download debugging &symbols Download debu&g binaries (requires VS 2017 or later) + Download &free-threaded binaries (experimental) [ActionLikeInstallation] Progress [ActionLikeInstalling]: diff --git a/Tools/msi/bundle/bundle.targets b/Tools/msi/bundle/bundle.targets index 9c7410fe514d19..cb3effb4434843 100644 --- a/Tools/msi/bundle/bundle.targets +++ b/Tools/msi/bundle/bundle.targets @@ -72,6 +72,7 @@ + diff --git a/Tools/msi/bundle/bundle.wxs b/Tools/msi/bundle/bundle.wxs index 8b12baae31105e..9b4f072152d5c0 100644 --- a/Tools/msi/bundle/bundle.wxs +++ b/Tools/msi/bundle/bundle.wxs @@ -82,6 +82,13 @@ + + + + + + + @@ -104,6 +111,9 @@ + + + diff --git a/Tools/msi/bundle/packagegroups/freethreaded.wxs b/Tools/msi/bundle/packagegroups/freethreaded.wxs new file mode 100644 index 00000000000000..121ca34ab66157 --- /dev/null +++ b/Tools/msi/bundle/packagegroups/freethreaded.wxs @@ -0,0 +1,62 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/Tools/msi/freethreaded/freethreaded.wixproj b/Tools/msi/freethreaded/freethreaded.wixproj new file mode 100644 index 00000000000000..0b4bd055d77977 --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded.wixproj @@ -0,0 +1,20 @@ + + + + {1B4502D5-B627-4F50-ABEA-4CC5A8E88265} + 2.0 + freethreaded + Package + + + + + + + + + + + + + \ No newline at end of file diff --git a/Tools/msi/freethreaded/freethreaded.wxs b/Tools/msi/freethreaded/freethreaded.wxs new file mode 100644 index 00000000000000..063aa28bf09fce --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded.wxs @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + + + + diff --git a/Tools/msi/freethreaded/freethreaded_d.wixproj b/Tools/msi/freethreaded/freethreaded_d.wixproj new file mode 100644 index 00000000000000..e1563d4f907126 --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_d.wixproj @@ -0,0 +1,20 @@ + + + + {D3677DCF-098A-4398-9FA5-8E74AC37E0DF} + 2.0 + freethreaded_d + Package + + + + + + + + + + + + + \ No newline at end of file diff --git a/Tools/msi/freethreaded/freethreaded_d.wxs b/Tools/msi/freethreaded/freethreaded_d.wxs new file mode 100644 index 00000000000000..cddf22a6c803d3 --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_d.wxs @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + diff --git a/Tools/msi/freethreaded/freethreaded_en-US.wxl_template b/Tools/msi/freethreaded/freethreaded_en-US.wxl_template new file mode 100644 index 00000000000000..b9747eb256d24b --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_en-US.wxl_template @@ -0,0 +1,8 @@ + + + Freethreaded Interpreter + freethreaded + Python {{ShortVersion}} ({{Bitness}}, freethreaded) + Launches the !(loc.ProductName) freethreaded interpreter. + https://www.python.org/ + diff --git a/Tools/msi/freethreaded/freethreaded_files.wxs b/Tools/msi/freethreaded/freethreaded_files.wxs new file mode 100644 index 00000000000000..adaf63c69d5ade --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_files.wxs @@ -0,0 +1,175 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Tools/msi/freethreaded/freethreaded_pdb.wixproj b/Tools/msi/freethreaded/freethreaded_pdb.wixproj new file mode 100644 index 00000000000000..789a4f55ae5191 --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_pdb.wixproj @@ -0,0 +1,20 @@ + + + + {E98E7539-64E7-4DCE-AACD-01E3ADE40EFD} + 2.0 + freethreaded_pdb + Package + + + + + + + + + + + + + \ No newline at end of file diff --git a/Tools/msi/freethreaded/freethreaded_pdb.wxs b/Tools/msi/freethreaded/freethreaded_pdb.wxs new file mode 100644 index 00000000000000..302ac416fe9275 --- /dev/null +++ b/Tools/msi/freethreaded/freethreaded_pdb.wxs @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/Tools/msi/lib/lib_files.wxs b/Tools/msi/lib/lib_files.wxs index a82cad596d47a6..b8e16b5fe238a0 100644 --- a/Tools/msi/lib/lib_files.wxs +++ b/Tools/msi/lib/lib_files.wxs @@ -26,10 +26,10 @@ - + - + @@ -63,10 +63,10 @@ - + - + @@ -100,10 +100,10 @@ - + - + diff --git a/Tools/msi/msi.props b/Tools/msi/msi.props index cfb3ca9e76e24c..372c4823bce07f 100644 --- a/Tools/msi/msi.props +++ b/Tools/msi/msi.props @@ -24,14 +24,14 @@ This URI is used to generate the various GUIDs used by the installer. Installers built with the same URI will upgrade each other or block when attempting to downgrade. - + By default, this is the local computer name, which will produce installers that do not interfere with other installers. Products that intend to bundle Python should rebuild these modules with their own URI to avoid conflicting with the official releases. - + The official releases use "https://www.python.org/$(ArchName)" - + This is not the same as the DownloadUrl property used in the bundle projects. --> @@ -39,7 +39,7 @@ $(ReleaseUri)/ - + @@ -63,13 +63,17 @@ $(MajorVersionNumber).$(MinorVersionNumber).$(Field3Value).0 - + + + false + + $([System.Math]::Floor($([System.DateTime]::Now.Subtract($([System.DateTime]::new(2001, 1, 1))).TotalDays))) $(MajorVersionNumber).$(MinorVersionNumber).$(MicroVersionNumber)dev$(RevisionNumber) $(MajorVersionNumber).$(MinorVersionNumber).$(RevisionNumber).0 - + 32-bit 64-bit @@ -91,9 +95,12 @@ PyDebugExt=$(PyDebugExt); PyArchExt=$(PyArchExt); PyTestExt=$(PyTestExt); + PydTag=$(PydTag); + FreethreadedPydTag=$(FreethreadedPydTag); OptionalFeatureName=$(OutputName); ssltag=$(OpenSSLDLLSuffix); Suffix32=$(PyArchExt); + IncludeFreethreaded=$(IncludeFreethreaded); $(DefineConstants);CRTRedist=$(CRTRedist); @@ -139,7 +146,7 @@ - + <_Uuid Include="CoreUpgradeCode"> upgradecode @@ -162,6 +169,12 @@ <_Uuid Include="PythonRegComponentGuid"> registry/$(OutputName) + <_Uuid Include="FreethreadedPythonExeComponentGuid" Condition="$(IncludeFreethreaded)"> + freethreaded/python.exe + + <_Uuid Include="FreethreadedPythonwExeComponentGuid" Condition="$(IncludeFreethreaded)"> + freethreaded/pythonw.exe + @(_Uuid->'("%(Identity)", "$(MajorVersionNumber).$(MinorVersionNumber)/%(Uri)")',',') <_GenerateCommand>import uuid; print('\n'.join('{}={}'.format(i, uuid.uuid5(uuid.UUID('c8d9733e-a70c-43ff-ab0c-e26456f11083'), '$(ReleaseUri.Replace(`{arch}`, `$(ArchName)`))' + j)) for i,j in [$(_Uuids.Replace(`"`,`'`))])) - + - + - + $(DefineConstants);@(_UuidValue,';');