From 9d7a77e03bbedb8b25d13e983912262ba27225eb Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Fri, 1 Dec 2023 11:31:42 +0100 Subject: [PATCH 1/9] Update "install_requires vs. requirements files" discussion Use pyproject.toml metadata instead of referring to setuptools' install_requires setup() keyword. --- .../install-requires-vs-requirements.rst | 85 ++++++++++--------- ...distributing-packages-using-setuptools.rst | 2 - source/guides/writing-pyproject-toml.rst | 2 + 3 files changed, 45 insertions(+), 44 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 99e1552b8..1481ad0fd 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -1,89 +1,90 @@ -.. _`install_requires vs requirements files`: +========================================== +Package dependencies vs requirements files +========================================== -====================================== -install_requires vs requirements files -====================================== +Package dependencies +==================== -install_requires ----------------- +Packages can declare dependencies, i.e., other packages that they need +to function. The standard method to do so is to set the +:ref:`dependencies key ` in the +``[project]`` section of a ``pyproject.toml`` file. When installing a +package, installers like :ref:`pip` will automatically install the +dependencies. This should be used for packages that the project +**minimally** needs to run correctly. -``install_requires`` is a :ref:`setuptools` :file:`setup.py` keyword that -should be used to specify what a project **minimally** needs to run correctly. -When the project is installed by :ref:`pip`, this is the specification that is -used to install its dependencies. - -For example, if the project requires A and B, your ``install_requires`` would be +For example, if the project requires A and B, your ``dependencies`` would be like so: -:: +.. code-block:: toml - install_requires=[ - 'A', - 'B' - ] + [project] + dependencies = ["A", "B"] Additionally, it's best practice to indicate any known lower or upper bounds. For example, it may be known, that your project requires at least v1 of 'A', and v2 of 'B', so it would be like so: -:: +.. code-block:: toml - install_requires=[ - 'A>=1', - 'B>=2' - ] + [project] + dependencies = [ + "A>=1", + "B>=2" + ] It may also be known that project 'A' introduced a change in its v2 that breaks the compatibility of your project with v2 of 'A' and later, so it makes sense to not allow v2: -:: +.. code-block:: toml - install_requires=[ - 'A>=1,<2', - 'B>=2' - ] + [project] + dependencies = [ + "A>=1,<2", + "B>=2" + ] -It is not considered best practice to use ``install_requires`` to pin -dependencies to specific versions, or to specify sub-dependencies +It is not considered best practice to use ``dependencies`` to pin +dependencies to specific versions, or to specify transitive dependencies (i.e. dependencies of your dependencies). This is overly-restrictive, and prevents the user from gaining the benefit of dependency upgrades. -Lastly, it's important to understand that ``install_requires`` is a listing of -"Abstract" requirements, i.e just names and version restrictions that don't -determine where the dependencies will be fulfilled from (i.e. from what -index or source). The where (i.e. how they are to be made "Concrete") is to -be determined at install time using :ref:`pip` options. [1]_ +Lastly, it's important to understand that ``dependencies`` is a listing of +"abstract" requirements, i.e, it just names and version restrictions, but doesn't +determine where the dependencies will be fulfilled from (from what package +index or source). The where (i.e. how they are to be made "concrete") is to +be determined at install time, e.g., using :ref:`pip` options. [1]_ Requirements files ------------------- +================== -:ref:`Requirements Files ` described most simply, are +:ref:`Requirements Files `, described most simply, are just a list of :ref:`pip:pip install` arguments placed into a file. -Whereas ``install_requires`` defines the dependencies for a single project, +Whereas ``dependencies`` defines the dependencies for a single project, :ref:`Requirements Files ` are often used to define the requirements for a complete Python environment. -Whereas ``install_requires`` requirements are minimal, requirements files +Whereas ``dependencies`` requirements are minimal, requirements files often contain an exhaustive listing of pinned versions for the purpose of achieving :ref:`repeatable installations ` of a complete environment. -Whereas ``install_requires`` requirements are "Abstract", i.e. not associated +Whereas ``dependencies`` requirements are "abstract", i.e., not associated with any particular index, requirements files often contain pip options like ``--index-url`` or ``--find-links`` to make requirements -"Concrete", i.e. associated with a particular index or directory of +"concrete", i.e., associated with a particular index or directory of packages. [1]_ -Whereas ``install_requires`` metadata is automatically analyzed by pip during an +Whereas ``dependencies`` metadata is automatically analyzed by pip during an install, requirements files are not, and only are used when a user specifically installs them using ``python -m pip install -r``. ---- -.. [1] For more on "Abstract" vs "Concrete" requirements, see +.. [1] For more on "abstract" vs "concrete" requirements, see https://caremad.io/posts/2013/07/setup-vs-requirement/. diff --git a/source/guides/distributing-packages-using-setuptools.rst b/source/guides/distributing-packages-using-setuptools.rst index 4b570e17b..7b8fc7b03 100644 --- a/source/guides/distributing-packages-using-setuptools.rst +++ b/source/guides/distributing-packages-using-setuptools.rst @@ -394,8 +394,6 @@ package, set ``py_modules`` to a list of the names of the modules (minus the minimally needs to run. When the project is installed by :ref:`pip`, this is the specification that is used to install its dependencies. -For more on using "install_requires" see :ref:`install_requires vs Requirements files`. - .. _python_requires: diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 41456da08..6dbb349c3 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -154,6 +154,8 @@ This field is required, although it is often marked as dynamic using Dependencies and requirements ============================= +.. _writing-pyproject-toml-dependencies: + ``dependencies``/``optional-dependencies`` ------------------------------------------ From 24bed62fe18eb0ff79be2b5a3b5d43c86c63b89e Mon Sep 17 00:00:00 2001 From: Jean Abou-Samra Date: Fri, 1 Dec 2023 16:21:30 +0100 Subject: [PATCH 2/9] Update source/discussions/install-requires-vs-requirements.rst Co-authored-by: sinoroc <5529267+sinoroc@users.noreply.github.com> --- source/discussions/install-requires-vs-requirements.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 1481ad0fd..5b2494a58 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -53,7 +53,7 @@ dependencies to specific versions, or to specify transitive dependencies prevents the user from gaining the benefit of dependency upgrades. Lastly, it's important to understand that ``dependencies`` is a listing of -"abstract" requirements, i.e, it just names and version restrictions, but doesn't +"abstract" requirements, i.e., just names and version restrictions, but doesn't determine where the dependencies will be fulfilled from (from what package index or source). The where (i.e. how they are to be made "concrete") is to be determined at install time, e.g., using :ref:`pip` options. [1]_ From a32e1cc9aa59cc615872d3e4d3995e6c3d7edbaf Mon Sep 17 00:00:00 2001 From: Jean Abou-Samra Date: Wed, 6 Dec 2023 18:47:38 +0100 Subject: [PATCH 3/9] Add spaces Co-authored-by: Sviatoslav Sydorenko --- source/discussions/install-requires-vs-requirements.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 5b2494a58..cfb011c32 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -43,8 +43,8 @@ so it makes sense to not allow v2: [project] dependencies = [ - "A>=1,<2", - "B>=2" + "A >= 1, < 2", + "B >= 2" ] It is not considered best practice to use ``dependencies`` to pin From 59498a7f7a761da1a1f50e308a56ce936c6cb224 Mon Sep 17 00:00:00 2001 From: Jean Abou-Samra Date: Wed, 6 Dec 2023 18:47:57 +0100 Subject: [PATCH 4/9] More spaces Co-authored-by: Sviatoslav Sydorenko --- source/discussions/install-requires-vs-requirements.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index cfb011c32..4e48ef79e 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -31,8 +31,8 @@ v2 of 'B', so it would be like so: [project] dependencies = [ - "A>=1", - "B>=2" + "A >= 1", + "B >= 2" ] It may also be known that project 'A' introduced a change in its v2 From 79f5dbd6976669792ed957c6b140614b3f0cd0de Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Wed, 6 Dec 2023 19:28:07 +0100 Subject: [PATCH 5/9] Rewrite to use "metadata dependencies" instead of "``dependencies``" --- .../install-requires-vs-requirements.rst | 80 +++++++++++-------- 1 file changed, 45 insertions(+), 35 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 4e48ef79e..9ad048f9a 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -1,21 +1,32 @@ -========================================== -Package dependencies vs requirements files -========================================== - - -Package dependencies -==================== - -Packages can declare dependencies, i.e., other packages that they need -to function. The standard method to do so is to set the -:ref:`dependencies key ` in the -``[project]`` section of a ``pyproject.toml`` file. When installing a -package, installers like :ref:`pip` will automatically install the -dependencies. This should be used for packages that the project +============================================ +Metadata dependencies vs. requirements files +============================================ + +There are two main places where you will find list of "needed packages to +install", perhaps with version constraints, like ``requests`` or +``requests==2.31.0``. These are metadata dependencies, typically in a +``pyproject.toml`` (or ``setup.py``) file, and requirements files, often called +``requirements.txt``. This page breaks down the differences. + +Metadata dependencies +===================== + +Packages can declare dependencies, i.e., other packages that they need to +function. The standard method to do so is to set the :ref:`dependencies key +` in the ``[project]`` section of a +``pyproject.toml`` file -- although other :term:`build backends ` +may use different methods. There can also be groups of optional dependencies, +also called "extras", which are typically specified in the +``optional-dependencies`` key of the ``[project]`` table. Both dependencies and +extras are ultimately written by the build backend to the package's distribution +metadata. On this page, we'll refer to these as "metadata dependencies". + +When installing a package, installers like :ref:`pip` will automatically install +the metadata dependencies. They should be used for packages that the project **minimally** needs to run correctly. -For example, if the project requires A and B, your ``dependencies`` would be -like so: +For example, suppose the project requires A and B. When using the ``[project]`` +table to declare metadata, the ``pyproject.toml`` would be like so: .. code-block:: toml @@ -25,7 +36,7 @@ like so: Additionally, it's best practice to indicate any known lower or upper bounds. For example, it may be known, that your project requires at least v1 of 'A', and -v2 of 'B', so it would be like so: +v2 of 'B'. .. code-block:: toml @@ -47,16 +58,16 @@ so it makes sense to not allow v2: "B >= 2" ] -It is not considered best practice to use ``dependencies`` to pin +It is not considered best practice to use metadata dependencies to pin dependencies to specific versions, or to specify transitive dependencies -(i.e. dependencies of your dependencies). This is overly-restrictive, and +(i.e., dependencies of your dependencies). This is overly restrictive, and prevents the user from gaining the benefit of dependency upgrades. -Lastly, it's important to understand that ``dependencies`` is a listing of -"abstract" requirements, i.e., just names and version restrictions, but doesn't -determine where the dependencies will be fulfilled from (from what package -index or source). The where (i.e. how they are to be made "concrete") is to -be determined at install time, e.g., using :ref:`pip` options. [1]_ +Lastly, it's important to understand that metadata dependencies are "abstract" +requirements, i.e., just names and version restrictions, but don't determine +where the dependencies will be fulfilled from (from what package index or +source). The where (i.e. how they are to be made "concrete") is to be determined +at install time, e.g., using :ref:`pip` options. [1]_ Requirements files @@ -65,24 +76,23 @@ Requirements files :ref:`Requirements Files `, described most simply, are just a list of :ref:`pip:pip install` arguments placed into a file. -Whereas ``dependencies`` defines the dependencies for a single project, -:ref:`Requirements Files ` are often used to define -the requirements for a complete Python environment. +Whereas metadata dependencies define the dependencies for a single +project, requirements files are often used to define the requirements +for a complete Python environment. -Whereas ``dependencies`` requirements are minimal, requirements files +Whereas metadata dependencies requirements are minimal, requirements files often contain an exhaustive listing of pinned versions for the purpose of achieving :ref:`repeatable installations ` of a complete environment. -Whereas ``dependencies`` requirements are "abstract", i.e., not associated -with any particular index, requirements files often contain pip -options like ``--index-url`` or ``--find-links`` to make requirements -"concrete", i.e., associated with a particular index or directory of -packages. [1]_ +Whereas metadata dependencies are "abstract", i.e., not associated with any +particular index, requirements files often contain pip options like +``--index-url`` or ``--find-links`` to make requirements "concrete", i.e., +associated with a particular index or directory of packages. [1]_ -Whereas ``dependencies`` metadata is automatically analyzed by pip during an +Whereas metadata dependencies are automatically analyzed by pip during an install, requirements files are not, and only are used when a user specifically -installs them using ``python -m pip install -r``. +installs them using :samp:`python -m pip install -r {requirement_file.txt}`. ---- From eb2c2ca1925ed6b93bd4d2de044aaa8e70553db7 Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Wed, 6 Dec 2023 19:44:38 +0100 Subject: [PATCH 6/9] Add back label --- source/discussions/install-requires-vs-requirements.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 9ad048f9a..70316e299 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -1,3 +1,5 @@ +.. _`install_requires vs requirements files`: + ============================================ Metadata dependencies vs. requirements files ============================================ From 981773e44c1d8130b027806d9d6e44bedb08ae5f Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Sat, 3 Feb 2024 23:05:14 +0100 Subject: [PATCH 7/9] =?UTF-8?q?=E2=80=9Ci.e.,=E2=80=9D=20=E2=86=92=20?= =?UTF-8?q?=E2=80=9Ci.e.=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- .../discussions/install-requires-vs-requirements.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 70316e299..d03d859cd 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -13,7 +13,7 @@ install", perhaps with version constraints, like ``requests`` or Metadata dependencies ===================== -Packages can declare dependencies, i.e., other packages that they need to +Packages can declare dependencies, i.e. other packages that they need to function. The standard method to do so is to set the :ref:`dependencies key ` in the ``[project]`` section of a ``pyproject.toml`` file -- although other :term:`build backends ` @@ -62,11 +62,11 @@ so it makes sense to not allow v2: It is not considered best practice to use metadata dependencies to pin dependencies to specific versions, or to specify transitive dependencies -(i.e., dependencies of your dependencies). This is overly restrictive, and +(i.e. dependencies of your dependencies). This is overly restrictive, and prevents the user from gaining the benefit of dependency upgrades. Lastly, it's important to understand that metadata dependencies are "abstract" -requirements, i.e., just names and version restrictions, but don't determine +requirements, i.e. just names and version restrictions, but don't determine where the dependencies will be fulfilled from (from what package index or source). The where (i.e. how they are to be made "concrete") is to be determined at install time, e.g., using :ref:`pip` options. [1]_ @@ -87,9 +87,9 @@ often contain an exhaustive listing of pinned versions for the purpose of achieving :ref:`repeatable installations ` of a complete environment. -Whereas metadata dependencies are "abstract", i.e., not associated with any +Whereas metadata dependencies are "abstract", i.e. not associated with any particular index, requirements files often contain pip options like -``--index-url`` or ``--find-links`` to make requirements "concrete", i.e., +``--index-url`` or ``--find-links`` to make requirements "concrete", i.e. associated with a particular index or directory of packages. [1]_ Whereas metadata dependencies are automatically analyzed by pip during an From cdb9b29b5b38f3dbc8832bd11e0f059a43248637 Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Sat, 3 Feb 2024 23:10:08 +0100 Subject: [PATCH 8/9] :file: role and e.g. commas --- .../discussions/install-requires-vs-requirements.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index d03d859cd..8d4ebf5e6 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -7,8 +7,8 @@ Metadata dependencies vs. requirements files There are two main places where you will find list of "needed packages to install", perhaps with version constraints, like ``requests`` or ``requests==2.31.0``. These are metadata dependencies, typically in a -``pyproject.toml`` (or ``setup.py``) file, and requirements files, often called -``requirements.txt``. This page breaks down the differences. +:file:`pyproject.toml` (or :file:`setup.py`) file, and requirements files, often called +:file:`requirements.txt`. This page breaks down the differences. Metadata dependencies ===================== @@ -16,7 +16,7 @@ Metadata dependencies Packages can declare dependencies, i.e. other packages that they need to function. The standard method to do so is to set the :ref:`dependencies key ` in the ``[project]`` section of a -``pyproject.toml`` file -- although other :term:`build backends ` +:file:`pyproject.toml` file -- although other :term:`build backends ` may use different methods. There can also be groups of optional dependencies, also called "extras", which are typically specified in the ``optional-dependencies`` key of the ``[project]`` table. Both dependencies and @@ -28,7 +28,7 @@ the metadata dependencies. They should be used for packages that the project **minimally** needs to run correctly. For example, suppose the project requires A and B. When using the ``[project]`` -table to declare metadata, the ``pyproject.toml`` would be like so: +table to declare metadata, the :file:`pyproject.toml` would be like so: .. code-block:: toml @@ -69,7 +69,7 @@ Lastly, it's important to understand that metadata dependencies are "abstract" requirements, i.e. just names and version restrictions, but don't determine where the dependencies will be fulfilled from (from what package index or source). The where (i.e. how they are to be made "concrete") is to be determined -at install time, e.g., using :ref:`pip` options. [1]_ +at install time, e.g. using :ref:`pip` options. [1]_ Requirements files From 6c5dd62c6f532eb5ae2c18651abbd017f1925ab6 Mon Sep 17 00:00:00 2001 From: Jean Abou Samra Date: Sat, 3 Feb 2024 23:13:03 +0100 Subject: [PATCH 9/9] Avoid possible misunderstanding of abstract vs concrete dependencies --- source/discussions/install-requires-vs-requirements.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/install-requires-vs-requirements.rst b/source/discussions/install-requires-vs-requirements.rst index 8d4ebf5e6..475e4610e 100644 --- a/source/discussions/install-requires-vs-requirements.rst +++ b/source/discussions/install-requires-vs-requirements.rst @@ -23,8 +23,8 @@ also called "extras", which are typically specified in the extras are ultimately written by the build backend to the package's distribution metadata. On this page, we'll refer to these as "metadata dependencies". -When installing a package, installers like :ref:`pip` will automatically install -the metadata dependencies. They should be used for packages that the project +When installing a package, installers like :ref:`pip` will automatically resolve +the metadata dependencies and install them. They should be used for packages that the project **minimally** needs to run correctly. For example, suppose the project requires A and B. When using the ``[project]``