Skip to content

Commit

Permalink
Merge pull request #25 from eccenca/develop
Browse files Browse the repository at this point in the history
prepare release
  • Loading branch information
seebi authored Sep 9, 2024
2 parents d7c8a46 + c76bb78 commit a09f5ef
Show file tree
Hide file tree
Showing 25 changed files with 331 additions and 100 deletions.
10 changes: 5 additions & 5 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
run: |
python3 -m pip install --user copier>=8.3.0
- name: create plugin with default answers and run check
- name: create projects with default answers and run check
env:
CMEM_BASE_URI: ${{ secrets.CMEM_BASE_URI }}
OAUTH_CLIENT_SECRET: ${{ secrets.OAUTH_CLIENT_SECRET }}
Expand All @@ -59,12 +59,12 @@ jobs:
uses: mikepenz/action-junit-report@v4
if: always() # always run even if the previous step fails
with:
report_paths: cmem-plugin-awesome/dist/junit-*.xml
report_paths: '**/dist/junit-*.xml'

- name: Publish Test and Coverage Report as PR comment
- name: Publish Test and Coverage Report of ONE test case as PR comment
uses: xportation/junit-coverage-report@main
if: github.event_name == 'pull_request'
with:
junit-path: cmem-plugin-awesome/dist/junit-pytest.xml
coverage-path: cmem-plugin-awesome/dist/coverage.xml
junit-path: plugin_dir/dist/junit-pytest.xml
coverage-path: plugin_dir/dist/coverage.xml

3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
cmem-plugin-awesome

*_dir
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](https://semver.org/)

## [7.0.0] 2024-09-09

### Changed

- Generalization of the template
- You can create now projects of the following types:
- eccenca Corporate Memory plugins (same as before)
- Generic Python Projects (this is new)
- The first template question will ask you for the project type.
- Most features depend on this project type and will adapt to the decision.

### Added

- more shields

### Fixed

- limitations of the 6.x template version regarding project name


## [6.4.0] 2024-08-18

### Changed
Expand Down
65 changes: 38 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
# cmem-plugin-template

[![workflow][build-shield-main]][github-actions] [![workflow][build-shield-develop]][github-actions] [![version][version-shield]][changelog] ![python-shield]
[![eccenca Corporate Memory][cmem-shield]][cmem] [![copier][copier-shield]][copier]
[![poetry][poetry-shield]][poetry-link] [![ruff][ruff-shield]][ruff-link] [![mypy][mypy-shield]][mypy-link] [![copier][copier-shield]][copier]
[![eccenca Corporate Memory][cmem-shield]][cmem]

This repository contains a [copier](https://copier.readthedocs.io/) template.

You can use it to bootstrap an [eccenca Corporate Memory](https://documentation.eccenca.com) [python plugin](https://documentation.eccenca.com/latest/develop/python-plugins/).
You can use it to bootstrap the following types of project:

- **[eccenca Corporate Memory](https://documentation.eccenca.com) [Python Plugin](https://documentation.eccenca.com/latest/develop/python-plugins/)**: Bootstrapping and keeping plugin repositories up-to-date was the initial idea of this template.
- **Generic Python Project**: After tons of commits, we wanted to use this template not only for plugins, but for all kinds of python projects.

<details>
<summary>Table of contents</summary>
<!-- vim-markdown-toc GFM -->

* [Features](#features)
Expand All @@ -18,17 +23,18 @@ You can use it to bootstrap an [eccenca Corporate Memory](https://documentation.
* [Setup](#setup)
* [Local Requirements](#local-requirements)
* [Integration Tests](#integration-tests)
* [Plugins only: Corporate Memory Environment](#plugins-only-corporate-memory-environment)
* [CI Build Plan](#ci-build-plan)
* [Editor / IDE Support](#editor--ide-support)
* [PyCharm](#pycharm)

<!-- vim-markdown-toc -->

</details>

## Features

- [Python / poetry](https://python-poetry.org/) project with
- [pytest](https://www.pytest.org/) (incl. [memray](https://bloomberg.github.io/memray/) + [pytest-dotenv](https://github.com/quiqua/pytest-dotenv)) as testing framework,
- [pytest](https://www.pytest.org/) (incl. [memray](https://bloomberg.github.io/memray/) + [pytest-dotenv](https://github.com/quiqua/pytest-dotenv) + [code coverage](https://github.com/pytest-dev/pytest-cov)) as the testing framework,
- [ruff](https://docs.astral.sh/ruff/) as all-hands linter and formatter,
- [mypy](http://mypy-lang.org/) as type checker, and
- [safety](https://pyup.io/safety/) as dependency vulnerability scanner.
Expand All @@ -48,6 +54,8 @@ You can use it to bootstrap an [eccenca Corporate Memory](https://documentation.
The following command will create a new project directory with the latest released template.
This produces a plugin which is compatible the [latest release of eccenca Corporate Memory](https://documentation.eccenca.com/latest/).

Note: Select 'Generic Python Project' as an answer to the initial question to skip creation of plugin specific code.

```shell-session
$ copier copy gh:eccenca/cmem-plugin-template cmem-plugin-my
```
Expand All @@ -59,17 +67,15 @@ This produces a plugin which is compatible the latest development snapshot of ec
$ copier copy -r develop gh:eccenca/cmem-plugin-template cmem-plugin-my
```

After that, you can initialize the repository and install git hooks:
After that, you need to initialize the repository and optionally install the git pre-commit hooks:

```shell-session
$ cd cmem-plugin-my
$ git init
$ git add .
$ git commit -m "init"
$ git init; git add .; git commit -m "init"
$ pre-commit install
```

Then you can run the local test suite an build a first deployment artefact:
Then you can run the local test suite and build a first deployment artefact:

```shell-session
$ task check build
Expand All @@ -80,13 +86,13 @@ $ task check build
We [continuously update](https://github.com/eccenca/cmem-plugin-template/graphs/code-frequency) this repository.
This includes maintenance of dependencies, build plan updates and the adoption of new features from the [plugin base library](https://github.com/eccenca/cmem-plugin-base).

In order to upgrade your plugin project to the latest template release, use the following command:
In order to upgrade your project to the latest template release, use the following command:

```shell-session
$ copier update
```

In order to prepare your plugin project for the upcoming next release, use this command:
In order to prepare your project for the upcoming next release, use this command:

```shell-session
$ copier update -r develop
Expand All @@ -96,15 +102,14 @@ Please also have a look at the [copier documentation](https://copier.readthedocs

### Other Tasks

All available tasks for your project can be listed like this:
The available tasks for your project can be listed like this (note that there are more tasks, prefixed with `plugin:` in case you started a plugin project):

```shell-session
∴ task
task: Available tasks for this project:
* build: Build a tarball and a wheel package
* check: Run whole test suite incl. unit and integration tests
* clean: Removes dist, *.pyc and some caches
* deploy: Install plugin package in Corporate Memory
* check:linters: Run all linter and static code analysis tests
* check:mypy: Complain about typing errors
* check:pytest: Run unit and integration tests
Expand Down Expand Up @@ -136,22 +141,20 @@ tasks:

The following tools are needed for local task execution:

- Python 3.11
- [copier](https://copier.readthedocs.io/) (>= v9) for project template rendering
- Python 3.11+
- [copier](https://copier.readthedocs.io/) (>= v9) for project template rendering and updating
- [task](https://taskfile.dev/) (>= v3.29) for running build tasks (make sure to follow the installation instructions to avoid confusion with taskwarrior)
- [poetry](https://python-poetry.org/) (>= v1.7) for packaging and dependency managing (+ [dynamic versioning plugin](https://github.com/mtkennerly/poetry-dynamic-versioning))
- [pre-commit](https://pre-commit.com/) (>= v2.20) - managing and maintaining pre-commit hooks
- [task](https://taskfile.dev/) (>= v3.29) for build task running (make sure to follow the installation instructions to avoid confusion with taskwarrior)
- [cmemc](https://eccenca.com/go/cmemc) (>= v23.1) for interacting with eccenca Corporate Memory
- [pre-commit](https://pre-commit.com/) (>= v2.20) for managing pre-commit hooks (optional)

Example installation of the requirements with [pipx](https://pypa.github.io/pipx/) on Ubuntu:
Example installation of the requirements with [pipx](https://pipx.pypa.io/) on Ubuntu:

```
``` shell-session
$ sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin
$ python3 -m pip install --user pipx
$ python3 -m pipx ensurepath
$ pipx install copier
$ pipx install pre-commit
$ pipx install cmem-cmemc
$ pipx install poetry
$ poetry self add "poetry-dynamic-versioning[plugin]"
```
Expand All @@ -164,13 +167,15 @@ Testing your plugin is crucial and should be done locally as well as integrated
In order to setup access to a Corporate Memory deployment, you need to provide correct environment variables.
Without these variables, only standalone tests can be executed (see `1 skipped`):

```shell-session
``` shell-session
$ task check:pytest
...
... ===== 3 passed, 1 skipped in 0.09s =====
```

By providing the correct [cmemc](https://eccenca.com/go/cmemc) [environment variables](https://documentation.eccenca.com/latest/automate/cmemc-command-line-interface/installation-and-configuration/file-based-configuration/#configuration-variables) in a `.env` file or directly in your environment, your plugin can be tested in an integrated way:
### Plugins only: Corporate Memory Environment

By providing the correct [cmemc](https://eccenca.com/go/cmemc) [environment variables](https://documentation.eccenca.com/latest/automate/cmemc-command-line-interface/configuration/environment-based-configuration/) in an `.env` file or directly in your environment, your plugin can be tested in an integrated way:

``` shell-session
# Environment as direct variables:
Expand All @@ -196,15 +201,15 @@ The gitlab workflow as well as the github action pipelines need the same environ
- For github, go to Settings > Secret > Actions > [New Repository Secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets)
- For gitlab, go to Settings > CI/CD > Variables (Expand) > [Add Variable (protected, masked, all environments)](https://docs.gitlab.com/ee/ci/variables/)

An example github pipeline can be seen [here](https://github.com/eccenca/cmem-plugin-kafka/actions).
An example github pipeline can be seen [here](https://github.com/eccenca/cmem-plugin-yaml/actions).

In addition to the eccenca Corporate Memory credential secrets, a `PYPI_TOKEN` secret can be set in order to use the `publish` task/workflow.

### Editor / IDE Support

#### PyCharm

In order to have the best PyCharm experience when starting a project with this template, we suggest the following PyCharm plugins:
In order to have the best PyCharm experience, when starting a project with this template, we suggest the following PyCharm plugins:

- [Ruff](https://plugins.jetbrains.com/plugin/20574-ruff) will provide the linting hints which will be raised by the pipeline anyway.
- [Taskfile](https://plugins.jetbrains.com/plugin/17058-taskfile) will allow for starting tasks.
Expand All @@ -215,8 +220,14 @@ In order to have the best PyCharm experience when starting a project with this t
[build-shield-main]: https://img.shields.io/github/actions/workflow/status/eccenca/cmem-plugin-template/check.yml?logo=github&branch=main&label=main
[build-shield-develop]: https://img.shields.io/github/actions/workflow/status/eccenca/cmem-plugin-template/check.yml?logo=github&branch=develop&label=develop
[copier]: https://copier.readthedocs.io/
[copier-shield]: https://img.shields.io/badge/made%20with-copier-orange
[copier-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-purple.json
[cmem]: https://documentation.eccenca.com
[cmem-shield]: https://img.shields.io/badge/made%20for-eccenca%20Corporate%20Memory-blue?logo=data:image/svg%2bxml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIj8+CjxzdmcKICAgaWQ9IkxheWVyXzEiCiAgIGRhdGEtbmFtZT0iTGF5ZXIgMSIKICAgdmlld0JveD0iMCAwIDgxLjI5MDAwMSA4Mi4yODk4NiIKICAgdmVyc2lvbj0iMS4xIgogICB3aWR0aD0iODEuMjkwMDAxIgogICBoZWlnaHQ9IjgyLjI4OTg2NCIKICAgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIgogICB4bWxuczpzdmc9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICA8ZGVmcwogICAgIGlkPSJkZWZzODI2Ij4KICAgIDxzdHlsZQogICAgICAgaWQ9InN0eWxlODI0Ij4KICAgICAgLmNscy0xIHsKICAgICAgICBmaWxsOiAjZjM5MjAwOwogICAgICB9CgogICAgICAuY2xzLTIgewogICAgICAgIGZpbGw6IG5vbmU7CiAgICAgICAgc3Ryb2tlOiAjZjM5MjAwOwogICAgICAgIHN0cm9rZS13aWR0aDogMS41cHg7CiAgICAgIH0KICAgIDwvc3R5bGU+CiAgPC9kZWZzPgogIDxnCiAgICAgaWQ9Imc4NDAiCiAgICAgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTMwLjc2LC0zMS4xNDAxMzkpIj4KICAgIDxwYXRoCiAgICAgICBjbGFzcz0iY2xzLTEiCiAgICAgICBkPSJNIDU1LjksODUuMTkgQSAyMC4xNCwyMC4xNCAwIDEgMCAzNS43Niw2NS4wNSAyMC4xNCwyMC4xNCAwIDAgMCA1NS45LDg1LjE5IFoiCiAgICAgICBpZD0icGF0aDgyOCIgLz4KICAgIDxwYXRoCiAgICAgICBjbGFzcz0iY2xzLTEiCiAgICAgICBkPSJtIDk4LDU0LjE0IGEgOSw5IDAgMSAwIC04Ljk1LC05IDguOTUsOC45NSAwIDAgMCA4Ljk1LDkgeiIKICAgICAgIGlkPSJwYXRoODMwIiAvPgogICAgPHBhdGgKICAgICAgIGNsYXNzPSJjbHMtMSIKICAgICAgIGQ9Ik0gODguMzUsMTA4LjQzIEEgMTIuMzEsMTIuMzEgMCAxIDAgNzYsOTYuMTIgMTIuMzEsMTIuMzEgMCAwIDAgODguMzEsMTA4LjQzIFoiCiAgICAgICBpZD0icGF0aDgzMiIgLz4KICAgIDxsaW5lCiAgICAgICBjbGFzcz0iY2xzLTIiCiAgICAgICB4MT0iODYuOTcwMDAxIgogICAgICAgeTE9IjkyLjA1OTk5OCIKICAgICAgIHgyPSI1OC43Nzk5OTkiCiAgICAgICB5Mj0iNjcuMzYwMDAxIgogICAgICAgaWQ9ImxpbmU4MzQiIC8+CiAgICA8bGluZQogICAgICAgY2xhc3M9ImNscy0yIgogICAgICAgeDE9Ijk5LjE4IgogICAgICAgeTE9IjQ1Ljg0IgogICAgICAgeDI9IjU1LjQ4IgogICAgICAgeTI9IjY2LjEyMDAwMyIKICAgICAgIGlkPSJsaW5lODM2IiAvPgogICAgPGxpbmUKICAgICAgIGNsYXNzPSJjbHMtMiIKICAgICAgIHgxPSI5Ny45ODk5OTgiCiAgICAgICB5MT0iNDQuNjUwMDAyIgogICAgICAgeDI9Ijg4LjM0OTk5OCIKICAgICAgIHkyPSI5Mi44Mzk5OTYiCiAgICAgICBpZD0ibGluZTgzOCIgLz4KICA8L2c+Cjwvc3ZnPgo=
[cmem-shield]: https://img.shields.io/endpoint?url=https://dev.documentation.eccenca.com/badge.json
[python-shield]: https://img.shields.io/badge/python-v3.11-blue
[mypy-link]: https://mypy-lang.org/
[mypy-shield]: https://www.mypy-lang.org/static/mypy_badge.svg
[ruff-link]: https://docs.astral.sh/ruff/
[ruff-shield]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json&label=Code%20Style
[poetry-link]: https://python-poetry.org/
[poetry-shield]: https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json

84 changes: 63 additions & 21 deletions Taskfile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
version: '3'

vars:
PLUGIN_DIR: cmem-plugin-awesome
TEMPLATE_REVISION:
sh: git rev-parse HEAD

tasks:

Expand All @@ -17,34 +18,75 @@ tasks:
clean:
desc: clean up working directory
cmds:
- rm -rf {{.PLUGIN_DIR}}
- rm -rf *_dir

create:
desc: create plugin directory with current version and ask for answers
desc: create a project with the current version and ask for answers
deps:
- clean
cmds:
- copier copy --vcs-ref {{.TEMPLATE_REVISION}} . {{.PLUGIN_DIR}}
vars:
TEMPLATE_REVISION:
sh: git rev-parse HEAD
- copier copy --vcs-ref {{.TEMPLATE_REVISION}} . new_dir

check:
desc: create plugin with default answers and run check
check:generate:case:
desc: generate a project with the current version from a TEST_CASE
preconditions:
- sh: "[[ -v TEST_CASE ]]"
msg: >
Environment needs 'TEST_CASE' variable.
See tests directory - all yaml files without suffix are test cases.
- sh: "[ -f tests/${TEST_CASE}.yml ]"
msg: Test case file ${TEST_CASE}.yml does not exist.
cmds:
- rm -rf {{.TEST_CASE}}_dir
- mkdir {{.TEST_CASE}}_dir
- cp tests/{{.TEST_CASE}}.yml {{.TEST_CASE}}_dir/.copier-answers.yml
- copier copy --vcs-ref {{.TEMPLATE_REVISION}} --defaults . {{.TEST_CASE}}_dir
- git config --global init.defaultBranch develop
- cd {{.TEST_CASE}}_dir && git init .
- cd {{.TEST_CASE}}_dir && git add .
- cd {{.TEST_CASE}}_dir && git config init.defaultBranch main
- cd {{.TEST_CASE}}_dir && git config user.name "Anonymous Person"
- cd {{.TEST_CASE}}_dir && git config user.email "[email protected]"
- cd {{.TEST_CASE}}_dir && git commit -m "init from template"

check:generate:cases:
desc: generate all test cases with the current template version
deps:
- clean
vars:
TEST_CASES:
sh: find tests -type f -name '*.yml' | sed 's/^tests\///g' | sed 's/\.yml$//g'
cmds:
- copier copy --vcs-ref {{.TEMPLATE_REVISION}} --defaults . {{.PLUGIN_DIR}}
- git config --global init.defaultBranch develop
- git init {{.PLUGIN_DIR}}
- cd {{.PLUGIN_DIR}} && git add .
- cd {{.PLUGIN_DIR}} && git config init.defaultBranch main
- cd {{.PLUGIN_DIR}} && git config user.name "Anonymous Person"
- cd {{.PLUGIN_DIR}} && git config user.email "[email protected]"
- cd {{.PLUGIN_DIR}} && git commit -m "init from template"
- cd {{.PLUGIN_DIR}} && poetry update
- cd {{.PLUGIN_DIR}} && task check
- for: { var: TEST_CASES }
cmd: TEST_CASE={{.ITEM}} task check:generate:case

check:validate:case:
desc: validate a project test case
preconditions:
- sh: "[[ -v TEST_CASE ]]"
msg: >
Environment needs 'TEST_CASE' variable.
See tests directory - all yaml files without suffix are test cases.
- sh: "[ -d ${TEST_CASE}_dir ]"
msg: >
Test case directory ${TEST_CASE}.yml does not exist.
Please use check:generate* tasks to create directories.
cmds:
- cd {{.TEST_CASE}}_dir && poetry update
- cd {{.TEST_CASE}}_dir && task check

check:validate:cases:
desc: validate all test cases
vars:
TEMPLATE_REVISION:
sh: git rev-parse HEAD
TEST_CASES:
sh: find tests -type f -name '*.yml' | sed 's/^tests\///g' | sed 's/\.yml$//g'
cmds:
- for: { var: TEST_CASES }
cmd: TEST_CASE={{.ITEM}} task check:validate:case

check:
desc: First generate, the validate all test cases
cmds:
- task: check:generate:cases
- task: check:validate:cases

Loading

0 comments on commit a09f5ef

Please sign in to comment.