Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add notes on runtime version access #1612

Merged
merged 1 commit into from
Oct 15, 2024
Merged

Add notes on runtime version access #1612

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions source/discussions/single-source-version.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,49 @@
.. _`Single sourcing the version discussion`:
.. _single-source-version:

===================================
Single-sourcing the Project Version
===================================

:Page Status: Complete
:Last Reviewed: 2024-08-24
:Last Reviewed: 2024-10-07

One of the challenges in building packages is that the version string can be required in multiple places.
Many Python :term:`distribution packages <Distribution Package>` publish a single
Python :term:`import package <Import Package>` where it is desired that the runtime
``__version__`` attribute on the import package report the same version specifier
as :func:`importlib.metadata.version` reports for the distribution package
(as described in :ref:`runtime-version-access`).

* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`)
This will make it available in the installed package’s metadata, from where it will be accessible at runtime using ``importlib.metadata.version("distribution_name")``.
It is also frequently desired that this version information be derived from a version
control system *tag* (such as ``v1.2.3``) rather than being manually updated in the
source code.

* A package may set a module attribute (e.g., ``__version__``) to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of the attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way <Build system version handling>`.
Some projects may choose to simply live with the data entry duplication, and rely
on automated testing to ensure the different values do not diverge.

* If the code is in in a version control system (VCS), e.g. Git, the version may appear in a *tag* such as ``v1.2.3``.

To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number.
Alternatively, a project's chosen build system may offer a way to define a single
source of truth for the version number.

In general, the options are:

1) If the code is in a version control system (VCS), e.g. Git, then the version can be extracted from the VCS.

2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required.
1) If the code is in a version control system (VCS), such as Git, then the version can be extracted from the VCS.

3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in a module, such as :file:`__init__.py`, and the build system can extract it at build time.
2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it
into other locations it may be required.

3) The version string can be hard-coded into the source code -- either in a special purpose file,
such as :file:`_version.txt` (which must then be shipped as part of the project's source distribution
package), or as an attribute in a particular module, such as :file:`__init__.py`. The build
system can then extract it from the runtime location at build time.

Consult your build system's documentation for their recommended method.

When the intention is that a distribution package and its associated import package
share the same version, it is recommended that the project include an automated test
case that ensures ``import_name.__version__`` and ``importlib.metadata.version("dist-name")``
report the same value (note: for many projects, ``import_name`` and ``dist-name`` will
be the same name).


.. _Build system version handling:

Build System Version Handling
Expand Down
59 changes: 58 additions & 1 deletion source/discussions/versioning.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,6 @@ numbering scheme that readily conveys the approximate age of a release, but
doesn't otherwise commit to a particular release cadence within the year.



Local version identifiers
=========================

Expand All @@ -172,6 +171,55 @@ since the latest release, setuptools-scm generates a version like
"0.5.dev1+gd00980f", or if the repository has untracked changes, like
"0.5.dev1+gd00980f.d20231217".

.. _runtime-version-access:

Accessing version information at runtime
========================================

Version information for all :term:`distribution packages <Distribution Package>`
that are locally available in the current environment can be obtained at runtime
using the standard library's :func:`importlib.metadata.version` function::

>>> importlib.metadata.version("cryptography")
'41.0.7'

Many projects also choose to version their top level
:term:`import packages <Import Package>` by providing a package level
``__version__`` attribute::

>>> import cryptography
>>> cryptography.__version__
'41.0.7'

This technique can be particularly valuable for CLI applications which want
to ensure that version query invocations (such as ``pip -V``) run as quickly
as possible.

Package publishers wishing to ensure their reported distribution package and
import package versions are consistent with each other can review the
:ref:`single-source-version` discussion for potential approaches to doing so.

As import packages and modules are not *required* to publish runtime
version information in this way (see the rejected proposal in
:pep:`PEP 396 <396>`), the ``__version__`` attribute should either only be
queried with interfaces that are known to provide it (such as a project
querying its own version or the version of one of its direct dependencies),
or else the querying code should be designed to handle the case where the
attribute is missing [#fallback-to-dist-version]_.

Some projects may need to publish version information for external APIs
that don't meet the requirements for Python distribution package
:ref:`version specifiers <version-specifiers>`. Such projects should
define their own project-specific ways of obtaining the relevant information
at runtime. For example, the standard library's :mod:`ssl` module offers
multiple ways to access the underlying OpenSSL library version::

>>> ssl.OPENSSL_VERSION
'OpenSSL 3.2.2 4 Jun 2024'
>>> ssl.OPENSSL_VERSION_INFO
(3, 2, 0, 2, 0)
>>> hex(ssl.OPENSSL_VERSION_NUMBER)
'0x30200020'

--------------------------------------------------------------------------------

Expand All @@ -184,6 +232,15 @@ since the latest release, setuptools-scm generates a version like
Brett Cannon <semver-brett-cannon_>`_. For a humoristic take, read about
ZeroVer_.

.. [#fallback-to-dist-version] A full list mapping the top level names available
for import to the distribution packages that provide those import packages and
modules may be obtained through the standard library's
:func:`importlib.metadata.packages_distributions` function. This means that
even code that is attempting to infer a version to report for all importable
top-level names has a means to fall back to reporting the distribution
version information if no ``__version__`` attribute is defined. Only standard
library modules, and modules added via means other than Python package
installation would fail to have version information reported in that case.


.. _zerover: https://0ver.org
Expand Down
1 change: 0 additions & 1 deletion source/guides/section-build-and-publish.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ Building and Publishing

writing-pyproject-toml
distributing-packages-using-setuptools
single-sourcing-package-version
dropping-older-python-versions
packaging-binary-extensions
packaging-namespace-packages
Expand Down
175 changes: 5 additions & 170 deletions source/guides/single-sourcing-package-version.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,173 +1,8 @@
.. _`Single sourcing the version`:
:orphan:

===================================
Single-sourcing the package version
===================================
.. meta::
:http-equiv=refresh: 0; url=../../discussions/single-source-version/

.. todo:: Update this page for build backends other than setuptools.
Redirecting stale single-source package version link...

There are many techniques to maintain a single source of truth for the version
number of your project:

#. Read the file in :file:`setup.py` and get the version. Example (from `pip setup.py
<https://github.com/pypa/pip/blob/003c7ac/setup.py>`_)::

import codecs
import os.path

def read(rel_path):
here = os.path.abspath(os.path.dirname(__file__))
with codecs.open(os.path.join(here, rel_path), 'r') as fp:
return fp.read()

def get_version(rel_path):
for line in read(rel_path).splitlines():
if line.startswith('__version__'):
delim = '"' if '"' in line else "'"
return line.split(delim)[1]
else:
raise RuntimeError("Unable to find version string.")

setup(
...
version=get_version("package/__init__.py")
...
)

.. note::

As of the release of setuptools 46.4.0, one can accomplish the same
thing by instead placing the following in the project's
:file:`setup.cfg` file (replacing "package" with the import name of the
package):

.. code-block:: ini

[metadata]
version = attr: package.__version__

As of the release of setuptools 61.0.0, one can specify the
version dynamically in the project's :file:`pyproject.toml` file.

.. code-block:: toml

[project]
name = "package"
dynamic = ["version"]

[tool.setuptools.dynamic]
version = {attr = "package.__version__"}

Please be aware that declarative config indicators, including the
``attr:`` directive, are not supported in parameters to
:file:`setup.py`.

#. Use an external build tool that either manages updating both locations, or
offers an API that both locations can use.

Few tools you could use, in no particular order, and not necessarily complete:
`bump2version <https://pypi.org/project/bump2version>`_,
`changes <https://pypi.org/project/changes>`_,
`commitizen <https://pypi.org/project/commitizen>`_,
`zest.releaser <https://pypi.org/project/zest.releaser>`_.


#. Set the value to a ``__version__`` global variable in a dedicated module in
your project (e.g. :file:`version.py`), then have :file:`setup.py` read and
``exec`` the value into a variable.

::

version = {}
with open("...sample/version.py") as fp:
exec(fp.read(), version)
# later on we use: version['__version__']

Example using this technique: `warehouse <https://github.com/pypa/warehouse/blob/64ca42e42d5613c8339b3ec5e1cb7765c6b23083/warehouse/__about__.py>`_.

#. Place the value in a simple ``VERSION`` text file and have both
:file:`setup.py` and the project code read it.

::

with open(os.path.join(mypackage_root_dir, 'VERSION')) as version_file:
version = version_file.read().strip()

An advantage with this technique is that it's not specific to Python. Any
tool can read the version.

.. warning::

With this approach you must make sure that the ``VERSION`` file is included in
all your source and binary distributions (e.g. add ``include VERSION`` to your
:file:`MANIFEST.in`).

#. Set the value in :file:`setup.py`, and have the project code use the
``importlib.metadata`` API to fetch the value at runtime.
(``importlib.metadata`` was introduced in Python 3.8 and is available to
older versions as the ``importlib-metadata`` project.) An installed
project's version can be fetched with the API as follows::

import sys

if sys.version_info >= (3, 8):
from importlib import metadata
else:
import importlib_metadata as metadata

assert metadata.version('pip') == '1.2.0'

Be aware that the ``importlib.metadata`` API only knows about what's in the
installation metadata, which is not necessarily the code that's currently
imported.

If a project uses this method to fetch its version at runtime, then its
``install_requires`` value needs to be edited to install
``importlib-metadata`` on pre-3.8 versions of Python like so::

setup(
...
install_requires=[
...
'importlib-metadata >= 1.0 ; python_version < "3.8"',
...
],
...
)

An older (and less efficient) alternative to ``importlib.metadata`` is the
``pkg_resources`` API provided by ``setuptools``::

import pkg_resources
assert pkg_resources.get_distribution('pip').version == '1.2.0'

If a project uses ``pkg_resources`` to fetch its own version at runtime,
then ``setuptools`` must be added to the project's ``install_requires``
list.

Example using this technique: `setuptools <https://github.com/pypa/setuptools/blob/main/setuptools/version.py>`_.


#. Set the value to ``__version__`` in ``sample/__init__.py`` and import
``sample`` in :file:`setup.py`.

::

import sample
setup(
...
version=sample.__version__
...
)

.. warning::

Although this technique is common, beware that it will fail if
``sample/__init__.py`` imports packages from ``install_requires``
dependencies, which will very likely not be installed yet when
:file:`setup.py` is run.


#. Keep the version number in the tags of a version control system (Git, Mercurial, etc)
instead of in the code, and automatically extract it from there using
`setuptools_scm <https://pypi.org/project/setuptools-scm/>`_.
If the page doesn't automatically refresh, see :ref:`single-source-version`.
4 changes: 2 additions & 2 deletions source/guides/writing-pyproject-toml.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,8 @@ This field is required, although it is often marked as dynamic using
dynamic = ["version"]

This allows use cases such as filling the version from a ``__version__``
attribute or a Git tag. Consult :ref:`Single sourcing the version` for more
details.
attribute or a Git tag. Consult the :ref:`single-source-version`
discussion for more details.


Dependencies and requirements
Expand Down
Loading