Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: remove pre-commit-uv and tox-uv from environment #452

Merged
merged 4 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
"mypy",
"oneline",
"pixi",
"PyPA",
"pytest",
"PYTHONHASHSEED",
"rtoml",
Expand Down Expand Up @@ -100,6 +101,7 @@
"prereleased",
"prettierignore",
"pyenv",
"pyproject",
"pyright",
"pyrightconfig",
"pyupgrade",
Expand Down
15 changes: 0 additions & 15 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,17 +1,3 @@
# Output files
*.gv
*.json
*.npy
*.pdf
*.pickle
*.png
*.svg
*.v2
*.xml
*.yaml
*.yml
commitlint.config.js

# Build files
*.egg-info/
*build/
Expand All @@ -24,7 +10,6 @@ version.py
*.pyc
.coverage
.coverage.*
.ipynb_checkpoints/
.mypy*/
.pytest_cache/
__pycache__/
Expand Down
4 changes: 4 additions & 0 deletions .prettierrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
overrides:
- files: "**.md.jinja"
options:
parser: markdown
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
"editor.rulers": [72],
"rewrap.wrappingColumn": 72
},
"[jinja-md]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
},
"[json]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.wordWrap": "on"
Expand Down
20 changes: 19 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,26 @@ uv sync
source .venv/bin/activate
```

Formatting and linting checks are automatically performed when committing changes. This is done with [pre-commit](https://pre-commit.com). To install the hooks in your local repository, run [`pre-commit install`](https://pre-commit.com/#3-install-the-git-hook-scripts) **once**:
Formatting and linting checks are automatically performed when committing changes. This is done with [pre-commit](https://pre-commit.com). To install the hooks in your local repository, run install `pre-commit` with `uv`:

```shell
uv tool install pre-commit --with pre-commit-uv --force-reinstall
```

and [`pre-commit install`](https://pre-commit.com/#3-install-the-git-hook-scripts) **once**:

```shell
pre-commit install --install-hooks
```

In addition, it may be handy to install `tox`:

```shell
uv tool install tox --with tox-uv
```

If the repository provides a Tox configuration under [`pyproject.toml`](./pyproject.toml), you can see which jobs it defines with:

```shell
tox list
```
2 changes: 0 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ dev = [
"compwa-policy[test]",
"labels",
"sphinx-autobuild",
"tox >=1.9", # for skip_install, use_develop
]
doc = [
"Sphinx",
Expand All @@ -51,7 +50,6 @@ doc = [
sty = [
"compwa-policy[types]",
"mypy",
"pre-commit-uv",
"ruff",
]
test = [
Expand Down
1 change: 1 addition & 0 deletions src/compwa_policy/.template/.cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@
"precommit",
"prereleased",
"prettierignore",
"pyproject",
"pyright",
"pyupgrade",
"redeboer",
Expand Down
20 changes: 19 additions & 1 deletion src/compwa_policy/.template/CONTRIBUTING.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,26 @@ uv sync
source .venv/bin/activate
```

Formatting and linting checks are automatically performed when committing changes. This is done with [pre-commit](https://pre-commit.com). To install the hooks in your local repository, run [`pre-commit install`](https://pre-commit.com/#3-install-the-git-hook-scripts) **once**:
Formatting and linting checks are automatically performed when committing changes. This is done with [pre-commit](https://pre-commit.com). To install the hooks in your local repository, run install `pre-commit` with `uv`:

```shell
uv tool install pre-commit --with pre-commit-uv --force-reinstall
```

and [`pre-commit install`](https://pre-commit.com/#3-install-the-git-hook-scripts) **once**:

```shell
pre-commit install --install-hooks
```

In addition, it may be handy to install `tox`:

```shell
uv tool install tox --with tox-uv
```

If the repository provides a Tox configuration under [`pyproject.toml`](./pyproject.toml), you can see which jobs it defines with:

```shell
tox list
```
2 changes: 1 addition & 1 deletion src/compwa_policy/check_dev_files/pixi/_update.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
Pyproject,
complies_with_subset,
)
from compwa_policy.utilities.python import split_dependency_definition
from compwa_policy.utilities.pyproject.setters import split_dependency_definition
from compwa_policy.utilities.readme import add_badge
from compwa_policy.utilities.toml import to_toml_array

Expand Down
85 changes: 4 additions & 81 deletions src/compwa_policy/check_dev_files/precommit.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,19 @@

import re
from pathlib import Path
from typing import TYPE_CHECKING, Any, cast
from typing import TYPE_CHECKING, cast

from ruamel.yaml.comments import CommentedMap, CommentedSeq
from ruamel.yaml.comments import CommentedMap

from compwa_policy.errors import PrecommitError
from compwa_policy.utilities import CONFIG_PATH
from compwa_policy.utilities.executor import Executor
from compwa_policy.utilities.precommit.getters import find_repo
from compwa_policy.utilities.precommit.struct import Hook
from compwa_policy.utilities.pyproject import ModifiablePyproject
from compwa_policy.utilities.python import split_dependency_definition
from compwa_policy.utilities.yaml import create_prettier_round_trip_yaml

if TYPE_CHECKING:
from collections.abc import MutableMapping

from compwa_policy.utilities.precommit import (
ModifiablePrecommit,
Precommit,
Expand All @@ -38,7 +35,8 @@ def main(precommit: ModifiablePrecommit, has_notebooks: bool) -> None:
do(_update_repo_urls, precommit)
if CONFIG_PATH.pyproject.exists():
with ModifiablePyproject.load() as pyproject:
do(_switch_to_precommit_uv, pyproject)
pyproject.remove_dependency("pre-commit")
pyproject.remove_dependency("pre-commit-uv")


def _sort_hooks(precommit: ModifiablePrecommit) -> None:
Expand Down Expand Up @@ -204,78 +202,3 @@ def _update_repo_urls(precommit: ModifiablePrecommit) -> None:
for url, new_url in updated_repos:
msg += f"\n {url} -> {new_url}"
precommit.changelog.append(msg)


def _switch_to_precommit_uv(pyproject: ModifiablePyproject) -> None:
__replace_precommit_in_conda()
__replace_precommit_in_pyproject(pyproject)


def __replace_precommit_in_conda() -> None:
if not CONFIG_PATH.conda.exists():
return
yaml = create_prettier_round_trip_yaml()
conda_env: CommentedMap = yaml.load(CONFIG_PATH.conda)
dependencies: CommentedSeq = conda_env.get("dependencies")
if dependencies is None:
return
precommit_idx = ___get_precommit_idx(dependencies)
if precommit_idx is None:
return
dependencies.pop(precommit_idx)
if "pip" not in dependencies:
dependencies.append("pip")
pip_dependencies = ___find_conda_pip_dict(dependencies)
if pip_dependencies is None:
pip_dependencies = CommentedMap({"pip": ["pre-commit-uv"]})
dependencies.append(pip_dependencies)
else:
pip_dependencies["pip"].append("pre-commit-uv")
pip_dependencies["pip"] = sorted(pip_dependencies["pip"])
yaml.dump(conda_env, CONFIG_PATH.conda)
msg = f"Switched to pre-commit-uv in {CONFIG_PATH.conda}"
raise PrecommitError(msg)


def ___get_precommit_idx(dependencies: list[str]) -> int | None:
for idx, dep in enumerate(dependencies):
if not isinstance(dep, str):
continue
name, *_ = split_dependency_definition(dep)
if name.lower() == "pre-commit":
return idx
return None


def ___find_conda_pip_dict(
dependencies: CommentedSeq,
) -> CommentedMap | None:
for dep in dependencies:
if isinstance(dep, CommentedMap) and "pip" in dep:
return dep
return None


def __replace_precommit_in_pyproject(pyproject: ModifiablePyproject) -> None:
table_key = "project.optional-dependencies"
if not pyproject.has_table(table_key):
return
optional_dependencies = pyproject.get_table(table_key)
updated = ___update_dependency_group(optional_dependencies, key="dev")
updated |= ___update_dependency_group(optional_dependencies, key="sty")
if updated:
msg = "Switched to pre-commit-uv in pyproject.toml"
pyproject.changelog.append(msg)


def ___update_dependency_group(
optional_dependencies: MutableMapping[str, Any], key: str
) -> bool:
dependencies: list[str] | None = optional_dependencies.get(key)
if dependencies is None:
return False
precommit_idx = ___get_precommit_idx(dependencies)
if precommit_idx is None:
return False
dependencies[precommit_idx] = "pre-commit-uv"
return True
20 changes: 12 additions & 8 deletions src/compwa_policy/check_dev_files/tox.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,24 +17,27 @@


def main(has_notebooks: bool) -> None:
_merge_tox_ini_into_pyproject()
tox = read_tox_config()
if tox is None:
return
if CONFIG_PATH.pyproject.is_file():
with ModifiablePyproject.load() as pyproject:
_merge_tox_ini_into_pyproject(pyproject)
pyproject.remove_dependency("tox")
pyproject.remove_dependency("tox-uv")
_check_expected_sections(tox, has_notebooks)


def _merge_tox_ini_into_pyproject() -> None:
def _merge_tox_ini_into_pyproject(pyproject: ModifiablePyproject) -> None:
if not CONFIG_PATH.tox.is_file():
return
with open(CONFIG_PATH.tox) as file:
tox_ini = file.read()
with ModifiablePyproject.load() as pyproject:
tox_table = pyproject.get_table("tool.tox", create=True)
tox_table["legacy_tox_ini"] = __ini_to_toml(tox_ini)
CONFIG_PATH.tox.unlink()
msg = f"Merged {CONFIG_PATH.tox} into {CONFIG_PATH.pyproject}"
pyproject.changelog.append(msg)
tox_table = pyproject.get_table("tool.tox", create=True)
tox_table["legacy_tox_ini"] = __ini_to_toml(tox_ini)
CONFIG_PATH.tox.unlink()
msg = f"Merged {CONFIG_PATH.tox} into {CONFIG_PATH.pyproject}"
pyproject.changelog.append(msg)


def __ini_to_toml(ini: str) -> String:
Expand Down Expand Up @@ -81,6 +84,7 @@ def read_tox_config() -> ConfigParser | None:
if tox_config_str is not None:
config = ConfigParser()
config.read_string(tox_config_str)
return config
return None


Expand Down
46 changes: 39 additions & 7 deletions src/compwa_policy/utilities/pyproject/setters.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from __future__ import annotations

import re
from collections import abc
from collections.abc import Iterable, Mapping, MutableMapping, Sequence
from typing import TYPE_CHECKING, Any, cast
Expand Down Expand Up @@ -85,7 +86,7 @@ def get_sub_table(
return cast(MutableMapping[str, Any], table)


def remove_dependency(
def remove_dependency( # noqa: C901
pyproject: PyprojectTOML,
package: str,
ignored_sections: Iterable[str] | None = None,
Expand All @@ -95,23 +96,54 @@ def remove_dependency(
return False
updated = False
dependencies = project.get("dependencies")
if dependencies is not None and package in dependencies:
dependencies.remove(package)
updated = True
if dependencies is not None:
package_names = [split_dependency_definition(p)[0] for p in dependencies]
if package in set(package_names):
idx = package_names.index(package)
dependencies.pop(idx)
updated = True
optional_dependencies = project.get("optional-dependencies")
if optional_dependencies is not None:
if ignored_sections is None:
ignored_sections = set()
else:
ignored_sections = set(ignored_sections)
for section, values in optional_dependencies.items():
for section, dependencies in optional_dependencies.items():
if section in ignored_sections:
continue
if package in values:
values.remove(package)
package_names = [split_dependency_definition(p)[0] for p in dependencies]
if package in set(package_names):
idx = package_names.index(package)
dependencies.pop(idx)
updated = True
if updated:
empty_sections = [k for k, v in optional_dependencies.items() if not v]
for section in empty_sections:
del optional_dependencies[section]
return updated


def split_dependency_definition(definition: str) -> tuple[str, str, str]:
"""Get the package name, operator, and version from a PyPI dependency definition.

>>> split_dependency_definition("julia")
('julia', '', '')
>>> split_dependency_definition("python==3.9.*")
('python', '==', '3.9.*')
>>> split_dependency_definition("graphviz # for binder")
('graphviz', '', '')
>>> split_dependency_definition("pip > 19 # needed")
('pip', '>', '19')
>>> split_dependency_definition("compwa-policy!= 3.14")
('compwa-policy', '!=', '3.14')
>>> split_dependency_definition("my_package~=1.2")
('my_package', '~=', '1.2')
>>> split_dependency_definition("any_version_package==*")
('any_version_package', '==', '*')
"""
matches = re.match(r"^([a-zA-Z0-9_-]+)([\!<=>~\s]*)([^ ^#]*)", definition)
if not matches:
msg = f"Could not extract package name and version from {definition}"
raise ValueError(msg)
package, operator, version = matches.groups()
return package.strip(), operator.strip(), version.strip()
Loading
Loading