release for Corporate Memory v23.3.x (#20)

* Revert "Revert "add example custom icon"" This reverts commit c5368e7. * use 4.2.0 and allow-prereleases * adjust dependencies * forward plugin base to 4.3.0 (cmem-cmempy >= 23.3) * avoid safety issue 62044 for pip less than 23.3 * fix race condition in task deploy (#19) * Execute clean and prepare sequencially in the deploy task * restructure deps --------- Co-authored-by: Sebastian Tramp <[email protected]> * change log * use poetry-dynamic-versioning option bump=true * pylint test folder as well * add target-version = "py311" to black * integrate ruff (removing bandit, flake8, black and pylint) * update README * ruff: ignore EM* * remove pylint gitlab job * remove PD rules * integrate pytest-dotenv * update README * fix missing check for poetry versioning plugin on build task --------- Co-authored-by: Natanael Arndt <[email protected]>
eccenca · Nov 16, 2023 · 6412162 · 6412162
1 parent df55b60
commit 6412162
Show file tree

Hide file tree

Showing 15 changed files with 268 additions and 189 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](https://semver.org/)
 
+## [Unreleased]
+
+### Changed
+
+- integrate ruff (removing bandit, flake8, black and pylint)
+- use plugin base 4.3.0 (cmem-cmempy >= 23.3)
+- use poetry-dynamic-versioning option bump=true
+  - 0.0.1.devX.. instead of 0.0.0.postX
+
+### Fixed
+
+- race condition in deploy task (#19)
+- missing check for poetry versioning plugin on build task
+
+
 ## [5.3.4] 2023-11-06
 
 ### Fixed

diff --git a/README.md b/README.md
@@ -13,24 +13,33 @@ You can use it to bootstrap an [eccenca Corporate Memory](https://documentation.
 * [Features](#features)
 * [Usage](#usage)
     * [Project Initialization](#project-initialization)
-    * [Project Update](#project-update)
+    * [Template Updates](#template-updates)
     * [Other Tasks](#other-tasks)
 * [Setup](#setup)
     * [Local Requirements](#local-requirements)
     * [Integration Tests](#integration-tests)
     * [CI Build Plan](#ci-build-plan)
+    * [Editor / IDE Support](#editor--ide-support)
+        * [PyCharm](#pycharm)
 
 <!-- vim-markdown-toc -->
 
 
 ## Features
 
-- [Python / poetry](https://python-poetry.org/) project with [pylint](https://pylint.pycqa.org/), [pytest](https://www.pytest.org/), [flake8](https://flake8.pycqa.org/), [mypy](http://mypy-lang.org/), [bandit](https://bandit.readthedocs.io/), [memray](https://bloomberg.github.io/memray/) and [safety](https://pyup.io/safety/) integration
-- Local build plan with [task](https://taskfile.dev/) (tested for Linux, MacOS and Windows/MinGW)
-- [Github build plan](https://github.com/eccenca/cmem-plugin-template/tree/main/src/.github/workflows)
-- [Gitlab build plan](https://github.com/eccenca/cmem-plugin-template/blob/main/src/.gitlab-ci.yml)
-- Badges, junit XML files and coverage stat generation
-
+- [Python / poetry](https://python-poetry.org/) project with
+  - [pytest](https://www.pytest.org/) (incl. [memray](https://bloomberg.github.io/memray/) + [pytest-dotenv](https://github.com/quiqua/pytest-dotenv)) as testing framework,
+  - [ruff](https://docs.astral.sh/ruff/) as all-hands linter and formatter,
+  - [mypy](http://mypy-lang.org/) as type checker, and
+  - [safety](https://pyup.io/safety/) as dependency vulnerability scanner.
+- Build plans for
+  - [gitlab](https://github.com/eccenca/cmem-plugin-template/blob/main/src/.gitlab-ci.yml),
+  - [github](https://github.com/eccenca/cmem-plugin-template/tree/main/src/.github/workflows), and
+  - locally with [task](https://taskfile.dev/) (tested for Linux, MacOS and Windows/MinGW).
+  - Including
+    - badge generation,
+    - JUnit XML file and
+    - coverage stat generation.
 
 ## Usage
 
@@ -60,48 +69,52 @@ $ git commit -m "init"
 $ pre-commit install
 ```
 
-Then you can run the local test suite an build a first deployment artifact:
+Then you can run the local test suite an build a first deployment artefact:
 
 ```shell-session
 $ task check build
 ```
 
-### Project Update
+### Template Updates
+
+We [continuously update](https://github.com/eccenca/cmem-plugin-template/graphs/code-frequency) this repository.
+This includes maintenance of dependencies, build plan updates and the adoption of new features from the [plugin base library](https://github.com/eccenca/cmem-plugin-base).
 
-From time to time, this template will be upgraded, so you can update your repository as well:
+In order to upgrade your plugin project to the latest template release, use the following command:
 
 ```shell-session
 $ copier update
 ```
 
+In order to prepare your plugin project for the upcoming next release, use this command:
+
 ```shell-session
 $ copier update -r develop
 ```
 
-Please have a look at the [copier documentation](https://copier.readthedocs.io/en/stable/updating/).
+Please also have a look at the [copier documentation](https://copier.readthedocs.io/en/stable/updating/).
 
 ### Other Tasks
 
-Available tasks for your project can be listed like this:
+All available tasks for your project can be listed like this:
 
 ```shell-session
 ∴ task
 task: Available tasks for this project:
-* build:               Build a tarball and a wheel package
-* check:               Run whole test suite incl. unit and integration tests
-* clean:               Removes dist, *.pyc and some caches
-* deploy:              Install plugin package in Corporate Memory
-* check:bandit:        Find common security issues
-* check:flake8:        Enforce standard source code style guide
-* check:linters:       Run all linter and static code analysis tests
-* check:mypy:          Find type errors
-* check:pylint:        Find code smells, errors and style issues
-* check:pytest:        Run unit and integration tests
-* check:safety:        Scan dependencies for vulnerabilities
-* python:format:       Format Python files
+* build:                   Build a tarball and a wheel package
+* check:                   Run whole test suite incl. unit and integration tests
+* clean:                   Removes dist, *.pyc and some caches
+* deploy:                  Install plugin package in Corporate Memory
+* check:linters:           Run all linter and static code analysis tests
+* check:mypy:              Complain about typing errors
+* check:pytest:            Run unit and integration tests
+* check:ruff:              Complain about everything else
+* check:safety:            Complain about vulnerabilities in dependencies
+* format:fix:              Format Python files and fix obvious issues
+* format:fix-unsafe:       Format Python files and fix 'unsafe' issues
 ```
 
-You can extend this task lisk by creating a file `TaskfileCustom.yaml` in your repository root:
+You can extend this task list by creating a file `TaskfileCustom.yaml` in your repository root:
 
 ```shell-session
 $ cat TaskfileCustom.yaml
@@ -124,8 +137,8 @@ tasks:
 The following tools are needed for local task execution:
 
 - Python 3.11
-- [copier](https://copier.readthedocs.io/) (>= v8.3) for project template rendering
-- [poetry](https://python-poetry.org/) (>= v1.6) for packaging and dependency managing (+ [dynamic versioning plugin](https://github.com/mtkennerly/poetry-dynamic-versioning))
+- [copier](https://copier.readthedocs.io/) (>= v9) for project template rendering
+- [poetry](https://python-poetry.org/) (>= v1.7) for packaging and dependency managing (+ [dynamic versioning plugin](https://github.com/mtkennerly/poetry-dynamic-versioning))
 - [pre-commit](https://pre-commit.com/) (>= v2.20) - managing and maintaining pre-commit hooks
 - [task](https://taskfile.dev/) (>= v3.29) for build task running (make sure to follow the installation instructions to avoid confusion with taskwarrior)
 - [cmemc](https://eccenca.com/go/cmemc) (>= v23.1) for interacting with eccenca Corporate Memory
@@ -145,8 +158,8 @@ $ poetry self add "poetry-dynamic-versioning[plugin]"
 
 ### Integration Tests
 
-This template uses pytest for testing.
-Testing your plugin is crucial and should be done locally and integrated with eccenca Corporate Memory.
+This template uses the [pytest](https://pytest.org) testing framework.
+Testing your plugin is crucial and should be done locally as well as  integrated with eccenca Corporate Memory.
 
 In order to setup access to a Corporate Memory deployment, you need to provide correct environment variables.
 Without these variables, only standalone tests can be executed (see `1 skipped`):
@@ -157,22 +170,18 @@ $ task check:pytest
 ... ===== 3 passed, 1 skipped in 0.09s =====
 ```
 
-By giving the correct [cmemc](https://eccenca.com/go/cmemc) [environment variables](https://documentation.eccenca.com/22.1/automate/cmemc-command-line-interface/installation-and-configuration/file-based-configuration/#configuration-variables), your plugin can be tested in an integrated way:
+By providing the correct [cmemc](https://eccenca.com/go/cmemc) [environment variables](https://documentation.eccenca.com/latest/automate/cmemc-command-line-interface/installation-and-configuration/file-based-configuration/#configuration-variables) in a `.env` file or directly in your environment, your plugin can be tested in an integrated way:
 
-```shell-session
+``` shell-session
+# Environment as direct variables:
 $ export CMEM_BASE_URI="https://cmem.example.org"
 $ export OAUTH_CLIENT_ID="cmem-service-account"
 $ export OAUTH_CLIENT_SECRET="..."
 $ export OAUTH_GRANT_TYPE="client_credentials"
-$ task check:pytest
-...
-... ===== 4 passed in 1.71s =====
-
 ```
 
-You can also add these variables to the `.env` file in your repository root (just make sure to never commit this file).
-
-```shell-session
+``` shell-session
+# Environment as .env files
 $ cat .env
 CMEM_BASE_URI="https://cmem.example.org"
 OAUTH_CLIENT_ID="cmem-service-account"
@@ -182,15 +191,23 @@ OAUTH_GRANT_TYPE="client_credentials"
 
 ### CI Build Plan
 
-The gitlab workflow / github action pipelines need the same environment variables as secrets:
+The gitlab workflow as well as the github action pipelines need the same environment variables as secrets:
 
 - For github, go to Settings > Secret > Actions > [New Repository Secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets)
 - For gitlab, go to Settings > CI/CD > Variables (Expand) > [Add Variable (protected, masked, all environments)](https://docs.gitlab.com/ee/ci/variables/)
 
-Example github pipelines can be seen [here](https://github.com/eccenca/cmem-plugin-kafka/actions) and [here](https://github.com/eccenca/cmem-plugin-graphql/actions).
+An example github pipeline can be seen [here](https://github.com/eccenca/cmem-plugin-kafka/actions).
 
 In addition to the eccenca Corporate Memory credential secrets, a `PYPI_TOKEN` secret can be set in order to use the `publish` task/workflow.
 
+### Editor / IDE Support
+
+#### PyCharm
+
+In order to have the best PyCharm experience when starting a project with this template, we suggest the following PyCharm plugins:
+
+- [Ruff](https://plugins.jetbrains.com/plugin/20574-ruff) will provide the linting hints which will be raised by the pipeline anyway.
+- [Taskfile](https://plugins.jetbrains.com/plugin/17058-taskfile) will allow for starting tasks.
 
 [version-shield]: https://img.shields.io/github/v/tag/eccenca/cmem-plugin-template?label=version&sort=semver
 [changelog]: https://github.com/eccenca/cmem-plugin-template/blob/main/CHANGELOG.md

diff --git a/src/.flake8 b/src/.flake8
diff --git a/src/.github/workflows/check.yml b/src/.github/workflows/check.yml
@@ -42,21 +42,13 @@ jobs:
         run: |
           poetry self add "poetry-dynamic-versioning[plugin]"
 
-      - name: bandit
-        run: |
-          task check:bandit
-
-      - name: flake8
-        run: |
-          task check:flake8
-
       - name: mypy
         run: |
           task check:mypy
 
-      - name: pylint
+      - name: ruff
         run: |
-          task check:pylint
+          task check:ruff
 
       - name: pytest
         env:

diff --git a/src/.gitlab-ci.yml b/src/.gitlab-ci.yml
@@ -19,25 +19,15 @@ stages:
   - build
   - publish
 
-bandit:
+ruff:
   stage: test
   script:
-    - task check:bandit
+    - task check:ruff
   artifacts:
     when: always
     reports:
       junit:
-        - dist/junit-bandit.xml
-
-flake8:
-  stage: test
-  script:
-    - task check:flake8
-  artifacts:
-    when: always
-    reports:
-      junit:
-        - dist/junit-flake8.xml
+        - dist/junit-ruff.xml
 
 mypy:
   stage: test
@@ -49,16 +39,6 @@ mypy:
       junit:
         - dist/junit-mypy.xml
 
-pylint:
-  stage: test
-  script:
-    - task check:pylint
-  artifacts:
-    when: always
-    reports:
-      junit:
-        - dist/junit-pylint.xml
-
 pytest:
   stage: test
   coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
@@ -86,7 +66,6 @@ safety:
 build:
   stage: build
   needs:
-    - bandit
     - mypy
     - pytest
     - safety
@@ -102,8 +81,7 @@ pypi:
   # publishing only available on a tag
   stage: publish
   needs:
-    - flake8
-    - pylint
+    - ruff
     - build
   allow_failure: true
   when: manual