Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add notes on runtime version access #1607

Closed
wants to merge 15 commits into from
32 changes: 20 additions & 12 deletions source/discussions/single-source-version.rst
Original file line number Diff line number Diff line change
@@ -1,31 +1,39 @@
.. _`Single sourcing the version discussion`:
.. _single-source-version:

===================================
Single-sourcing the Project Version
===================================

:Page Status: Complete
:Last Reviewed: 2024-08-24
:Last Reviewed: 2024-10-02

One of the challenges in building packages is that the version string can be required in multiple places.
Many Python :term:`distribution packages <Distribution Package>` publish a single
Python :term:`import package <Import Package>` where it is desired that the runtime
``__version__`` attribute on the import package report the same version specifier
as ``importlib.metadata.version`` reports for the distribution package
(as described in :ref:`runtime-version-access`).

* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`)
This will make it available in the installed package’s metadata, from where it will be accessible at runtime using ``importlib.metadata.version("distribution_name")``.
It is also frequently desired that this version information be derived from a version
control system *tag* (such as ``v1.2.3``) rather than being manually updated in the
source code.

* A package may set a module attribute (e.g., ``__version__``) to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of the attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way <Build system version handling>`.
To ensure that version numbers do not get out of sync, it may be sufficient to add
an automated test case that ensure ``package.__version__`` and
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
``importlib.metadata.version("package")`` report the same value.

* If the code is in in a version control system (VCS), e.g. Git, the version may appear in a *tag* such as ``v1.2.3``.

To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number.
Alternatively, a project's chosen build system mar offer a way to define a single
source of truth for the version number.

In general, the options are:

1) If the code is in a version control system (VCS), e.g. Git, then the version can be extracted from the VCS.

2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required.

3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in a module, such as :file:`__init__.py`, and the build system can extract it at build time.
2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it
into other locations it may be required.

3) The version string can be hard-coded into the source code -- either in a special purpose file,
such as :file:`_version.txt`, or as an attribute in a module, such as :file:`__init__.py`, and the build
system can extract it at build time.
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved

Consult your build system's documentation for their recommended method.

Expand Down
44 changes: 43 additions & 1 deletion source/discussions/versioning.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,6 @@ numbering scheme that readily conveys the approximate age of a release, but
doesn't otherwise commit to a particular release cadence within the year.



Local version identifiers
=========================

Expand All @@ -172,6 +171,49 @@ since the latest release, setuptools-scm generates a version like
"0.5.dev1+gd00980f", or if the repository has untracked changes, like
"0.5.dev1+gd00980f.d20231217".

.. _runtime-version-access:

Accessing version information at runtime
========================================

Version information for all :term:`distribution packages <Distribution Package>`
that are locally available in the current environment can be obtained at runtime
using the standard library's ``importlib.metadata.version`` function::

>>> importlib.metadata.version("pip")
'23.3.2'

Many libraries also choose to version their top level
:term:`import packages <Import Package>` by providing a package level
``__version__`` attribute::

>>> import pip
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
>>> pip.__version__
'23.3.2'
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved

Import packages are *not* required to be versioned independently of their
distribution package version information (see the rejected proposal in
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
:pep:`PEP 396 <396>`), so this approach to retrieving runtime version
information should only be used with libraries that are known to provide it.

Library publishers wishing to ensure their reported distribution package and
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
import package versions are consistent with each other can review the
:ref:`single-source-version` discussion for potential approaches to doing so.

Some libraries may need to publish version information for external APIs
that don't meet the requirements for Python distribution package
:ref:`version specifiers <version-specifiers>`. Such libraries should
define their own library-specific ways of obtaining the relevant information
at runtime. For example, the standard library's :mod:`ssl` module offers
multiple ways to access the underlying OpenSSL library version::

>>> ssl.OPENSSL_VERSION
'OpenSSL 3.2.2 4 Jun 2024'
>>> ssl.OPENSSL_VERSION_INFO
(3, 2, 0, 2, 0)
>>> ssl.OPENSSL_VERSION_NUMBER
807403552


--------------------------------------------------------------------------------

Expand Down
1 change: 0 additions & 1 deletion source/guides/section-build-and-publish.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ Building and Publishing

writing-pyproject-toml
distributing-packages-using-setuptools
single-sourcing-package-version
dropping-older-python-versions
packaging-binary-extensions
packaging-namespace-packages
Expand Down
175 changes: 5 additions & 170 deletions source/guides/single-sourcing-package-version.rst
Original file line number Diff line number Diff line change
@@ -1,173 +1,8 @@
.. _`Single sourcing the version`:
:orphan:

===================================
Single-sourcing the package version
===================================
.. meta::
:http-equiv=refresh: 0; url=../../discussions/single-source-version/

.. todo:: Update this page for build backends other than setuptools.
Redirecting stale single-source package version link...

There are many techniques to maintain a single source of truth for the version
number of your project:

#. Read the file in :file:`setup.py` and get the version. Example (from `pip setup.py
<https://github.com/pypa/pip/blob/003c7ac/setup.py>`_)::

import codecs
import os.path

def read(rel_path):
here = os.path.abspath(os.path.dirname(__file__))
with codecs.open(os.path.join(here, rel_path), 'r') as fp:
return fp.read()

def get_version(rel_path):
for line in read(rel_path).splitlines():
if line.startswith('__version__'):
delim = '"' if '"' in line else "'"
return line.split(delim)[1]
else:
raise RuntimeError("Unable to find version string.")

setup(
...
version=get_version("package/__init__.py")
...
)

.. note::

As of the release of setuptools 46.4.0, one can accomplish the same
thing by instead placing the following in the project's
:file:`setup.cfg` file (replacing "package" with the import name of the
package):

.. code-block:: ini

[metadata]
version = attr: package.__version__

As of the release of setuptools 61.0.0, one can specify the
version dynamically in the project's :file:`pyproject.toml` file.

.. code-block:: toml

[project]
name = "package"
dynamic = ["version"]

[tool.setuptools.dynamic]
version = {attr = "package.__version__"}

Please be aware that declarative config indicators, including the
``attr:`` directive, are not supported in parameters to
:file:`setup.py`.

#. Use an external build tool that either manages updating both locations, or
offers an API that both locations can use.

Few tools you could use, in no particular order, and not necessarily complete:
`bump2version <https://pypi.org/project/bump2version>`_,
`changes <https://pypi.org/project/changes>`_,
`commitizen <https://pypi.org/project/commitizen>`_,
`zest.releaser <https://pypi.org/project/zest.releaser>`_.


#. Set the value to a ``__version__`` global variable in a dedicated module in
your project (e.g. :file:`version.py`), then have :file:`setup.py` read and
``exec`` the value into a variable.

::

version = {}
with open("...sample/version.py") as fp:
exec(fp.read(), version)
# later on we use: version['__version__']

Example using this technique: `warehouse <https://github.com/pypa/warehouse/blob/64ca42e42d5613c8339b3ec5e1cb7765c6b23083/warehouse/__about__.py>`_.

#. Place the value in a simple ``VERSION`` text file and have both
:file:`setup.py` and the project code read it.

::

with open(os.path.join(mypackage_root_dir, 'VERSION')) as version_file:
version = version_file.read().strip()

An advantage with this technique is that it's not specific to Python. Any
tool can read the version.

.. warning::

With this approach you must make sure that the ``VERSION`` file is included in
all your source and binary distributions (e.g. add ``include VERSION`` to your
:file:`MANIFEST.in`).

#. Set the value in :file:`setup.py`, and have the project code use the
``importlib.metadata`` API to fetch the value at runtime.
(``importlib.metadata`` was introduced in Python 3.8 and is available to
older versions as the ``importlib-metadata`` project.) An installed
project's version can be fetched with the API as follows::

import sys

if sys.version_info >= (3, 8):
from importlib import metadata
else:
import importlib_metadata as metadata

assert metadata.version('pip') == '1.2.0'

Be aware that the ``importlib.metadata`` API only knows about what's in the
installation metadata, which is not necessarily the code that's currently
imported.

If a project uses this method to fetch its version at runtime, then its
``install_requires`` value needs to be edited to install
``importlib-metadata`` on pre-3.8 versions of Python like so::

setup(
...
install_requires=[
...
'importlib-metadata >= 1.0 ; python_version < "3.8"',
...
],
...
)

An older (and less efficient) alternative to ``importlib.metadata`` is the
``pkg_resources`` API provided by ``setuptools``::

import pkg_resources
assert pkg_resources.get_distribution('pip').version == '1.2.0'

If a project uses ``pkg_resources`` to fetch its own version at runtime,
then ``setuptools`` must be added to the project's ``install_requires``
list.

Example using this technique: `setuptools <https://github.com/pypa/setuptools/blob/main/setuptools/version.py>`_.


#. Set the value to ``__version__`` in ``sample/__init__.py`` and import
``sample`` in :file:`setup.py`.

::

import sample
setup(
...
version=sample.__version__
...
)

.. warning::

Although this technique is common, beware that it will fail if
``sample/__init__.py`` imports packages from ``install_requires``
dependencies, which will very likely not be installed yet when
:file:`setup.py` is run.


#. Keep the version number in the tags of a version control system (Git, Mercurial, etc)
instead of in the code, and automatically extract it from there using
`setuptools_scm <https://pypi.org/project/setuptools-scm/>`_.
If the page doesn't automatically refresh, click :ref:`here <single-source-version>`.
4 changes: 2 additions & 2 deletions source/guides/writing-pyproject-toml.rst
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,8 @@ This field is required, although it is often marked as dynamic using
dynamic = ["version"]

This allows use cases such as filling the version from a ``__version__``
attribute or a Git tag. Consult :ref:`Single sourcing the version` for more
details.
attribute or a Git tag. Consult the :ref:`single-source-version`
discussion for more details.
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved


Dependencies and requirements
Expand Down
Loading