Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitLab support with API-compatible utility functions. #383

Merged
merged 48 commits into from
Oct 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c0ebc38
Implemented GitLab utility functions using python-gitlab for API comp…
github-actions[bot] Oct 19, 2024
425f563
Added GitLab-compatible utility functions analogous to GitHub functio…
github-actions[bot] Oct 19, 2024
73100bb
Added functions: `check_if_file_exists`, `get_file_in_repository`, `s…
github-actions[bot] Oct 19, 2024
65596ec
Added functions for operations such as getting diffs, adding reaction…
github-actions[bot] Oct 19, 2024
da19a07
WIP: sync API between gitlab and github
haesleinhuepf Oct 19, 2024
85f0d76
Added label and assignee output to `get_issue_details` to match the G…
github-actions[bot] Oct 19, 2024
c7dbfc9
Corrected the parameter type in test_get_repository_file_contents and…
github-actions[bot] Oct 19, 2024
b4a6fc8
WIP: sync gitlab/github API
haesleinhuepf Oct 19, 2024
16e11a8
WIP: sync github and gitlab API
haesleinhuepf Oct 19, 2024
dec0848
bugfix default parameters
haesleinhuepf Oct 19, 2024
42265b8
WIP: sync github and gitlab API
haesleinhuepf Oct 19, 2024
e777d75
fix typo
haesleinhuepf Oct 19, 2024
d09bc71
ensure output is as similar as possible
haesleinhuepf Oct 19, 2024
4708d1b
fix docs
haesleinhuepf Oct 19, 2024
03ba180
ensure api-compatibility gitlab, github
haesleinhuepf Oct 19, 2024
3bf294d
sync github and gitlab api
haesleinhuepf Oct 19, 2024
414a8f1
fix tests
haesleinhuepf Oct 19, 2024
ddaf7e3
added tests for gitlab
haesleinhuepf Oct 19, 2024
50a5c91
bump version
haesleinhuepf Oct 19, 2024
50b2aa3
added missing requirements
haesleinhuepf Oct 19, 2024
6eadaa3
add api key to run tests
haesleinhuepf Oct 19, 2024
7e6bf4c
make server url configurable
haesleinhuepf Oct 20, 2024
028c962
fix test
haesleinhuepf Oct 20, 2024
f4747ef
exptend github/lab api
haesleinhuepf Oct 20, 2024
b6cab3c
use variable github/lab api
haesleinhuepf Oct 20, 2024
5873f77
add test-notebook for github/lab API
haesleinhuepf Oct 20, 2024
7c0aebd
clean up imports
haesleinhuepf Oct 20, 2024
943e175
bugfix: wrong import
haesleinhuepf Oct 20, 2024
62fbaf6
bugfix: NullPointerException
haesleinhuepf Oct 20, 2024
7da3ac8
use variable github/lab API
haesleinhuepf Oct 20, 2024
178fb6e
add installation instructions for gitlab
haesleinhuepf Oct 20, 2024
04034a7
get default branch when running on gitlab
haesleinhuepf Oct 20, 2024
02f6331
fix tests
haesleinhuepf Oct 20, 2024
12c110c
fix test
haesleinhuepf Oct 20, 2024
0483a5b
fix tests
haesleinhuepf Oct 20, 2024
2a1883b
fix tests
haesleinhuepf Oct 20, 2024
48e1e69
fix tests
haesleinhuepf Oct 20, 2024
84bff7b
fix test
haesleinhuepf Oct 20, 2024
360577e
fix tests
haesleinhuepf Oct 20, 2024
93477c6
store texts and bytes
haesleinhuepf Oct 20, 2024
575a936
fix bytes encoding
haesleinhuepf Oct 20, 2024
592a53f
fix encoding
haesleinhuepf Oct 20, 2024
5e9c3bd
Debug tracing
haesleinhuepf Oct 20, 2024
1721cee
call flexible API
haesleinhuepf Oct 20, 2024
b157b81
build link to image differently
haesleinhuepf Oct 20, 2024
6468eb9
corrected link to files / images
haesleinhuepf Oct 20, 2024
fa2e61f
added link to gitlab example
haesleinhuepf Oct 20, 2024
1f3015f
added link to gitlab example
haesleinhuepf Oct 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/git-bob.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ jobs:
KISSKI_API_KEY: "${{ secrets.KISSKI_API_KEY }}"
BLABLADOR_API_KEY: "${{ secrets.BLABLADOR_API_KEY }}"
GITHUB_API_KEY: "${{ secrets.GITHUB_TOKEN }}"
GITLAB_API_KEY: "${{ secrets.GITLAB_API_KEY }}"
GITHUB_RUN_ID: "${{ github.run_id }}"
TWINE_USERNAME: "${{ secrets.TWINE_USERNAME }}"
TWINE_PASSWORD: "${{ secrets.TWINE_PASSWORD }}"
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ jobs:
pip install -e .

- name: Test with pytest and generate coverage
env:
GITLAB_API_KEY: "${{ secrets.GITLAB_API_KEY }}"
run: |
pytest --cov=./ --cov-report=xml

Expand Down
59 changes: 59 additions & 0 deletions .gitlab/.gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
stages:
- respond

respond:
stage: respond
image: ubuntu:latest

rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_PIPELINE_SOURCE == "push"
- if: $CI_PIPELINE_SOURCE == "web"
- if: $CI_PIPELINE_SOURCE == "trigger" && $CI_PIPELINE_TRIGGER_SOURCE == "issue"

before_script:
- apt-get update && apt-get install -y python3 python3-pip git python3-venv
- python3 -m venv venv # Create a virtual environment
- source venv/bin/activate # Activate the virtual environment
- pip install --upgrade pip # Upgrade pip within the virtual environment


script:
- echo "Merge Request IID - $CI_MERGE_REQUEST_IID"
- echo "Project Namespace - $CI_PROJECT_NAMESPACE"
- echo "Project Name - $CI_PROJECT_NAME"
- echo "Pipeline ID - $CI_PIPELINE_ID"
- echo "Job ID - $CI_JOB_ID"

- pip install git-bob==0.10.0
- pip install -r requirements.txt

- |
git-bob github-action $CI_PROJECT_PATH $CI_MERGE_REQUEST_IID $CI_OPEN_MERGE_REQUESTS_IID

variables:
GIT_BOB_AGENT_NAME: "git-bob"
GIT_BOB_LLM_NAME: "$GIT_BOB_LLM_NAME"
ANTHROPIC_API_KEY: "$ANTHROPIC_API_KEY"
GOOGLE_API_KEY: "$GOOGLE_API_KEY"
OPENAI_API_KEY: "$OPENAI_API_KEY"
GH_MODELS_API_KEY: "$GH_MODELS_API_KEY"
KISSKI_API_KEY: "$KISSKI_API_KEY"
BLABLADOR_API_KEY: "$BLABLADOR_API_KEY"
GITLAB_API_KEY: "$GITLAB_API_TOKEN"
GITLAB_PIPELINE_ID: "$CI_PIPELINE_ID"
GIT_SERVER_URL: "https://gitlab.com/"
TWINE_USERNAME: "$TWINE_USERNAME"
TWINE_PASSWORD: "$TWINE_PASSWORD"
SYSTEM_MESSAGE: |
You are an extremely skilled python developer. Your name is git-bob. You are sometimes called gitlab-ci bot.
You can solve programming tasks and review code.
When asked to solve a specific problem, you keep your code changes minimal and only solve the problem at hand.
You cannot retrieve information from other sources but from gitlab.com.
Do not claim anything that you don't know.
In case you are asked to review code, you focus on the quality of the code.
VISION_SYSTEM_MESSAGE: |
You are an AI-based vision model with excellent skills when it comes to describing image. When describing an image, you typically explain:
* What is shown in the image.
* If the image shows clearly distinct objects in its channels, these structures are listed for each channel individually.
* You speculate how the image was acquired.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ Furthermore, to guide discussions, you may want to setup issue templates, e.g.
* [Bio-Image Analysis](https://github.com/haesleinhuepf/git-bob-playground/blob/main/.github/ISSUE_TEMPLATE/bioimage_analysis.md)
* [Statistics and Plotting](https://github.com/haesleinhuepf/git-bob-playground/blob/main/.github/ISSUE_TEMPLATE/statistics_plotting.md)

## Installation as gitlab pipeline

Since version 0.10.0 git-bob has experimental support for [gitlab](https://gitlab.com). You find detailed instructions how to install it [here](docs/installation-tutorial-gitlab.md).

## Usage

To trigger git-bob, you need to comment on an issue or pull request with the following command:
Expand Down Expand Up @@ -114,6 +118,7 @@ Here's the recommended workflow for using git-bob:
* [Basic data analysis and plotting](https://github.com/NFDI4BIOIMAGE/training/issues/250)
* [Documenting source code](https://github.com/haesleinhuepf/git-bob/pull/29)
* [Assisting scientific manuscript writing](https://github.com/haesleinhuepf/git-bob-manuscript/pull/9)
* [Plotting a circle of triangles (in gitlab)](https://gitlab.com/haesleinhuepf/git-bob-gitlab-playground/-/issues/8)

## Installation for development

Expand Down
Binary file added docs/images/install-gitlab1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab2a.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab7.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab8.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/install-gitlab9.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
84 changes: 84 additions & 0 deletions docs/installation-tutorial-gitlab.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Installation in git-lab

Since version 0.10.0 git-bob has experimental support for [gitlab](https://gitlab.com).
Feedback about the instructions provided here are very [welcome](https://github.com/haesleinhuepf/git-bob/issues/new)!

For demonstration purposes, a [playground repository on gitlab.com](https://gitlab.com/haesleinhuepf/git-bob-gitlab-playground) has been set up as demonstrated in the following.

## Create a new repository

* Create a new repository in your gitlab instance or on [https://gitlab.com](https://gitlab.com).
* Add the [.gitlab-ci.yml](../.gitlab/.gitlab-ci.yml) to the root folder of your repository.
In this file, make the following modifications:
* If you are working with an own gitlab instance, modify the variable `GIT_SERVER_URL`. Enter the url of your server and finish with "/".
* In the line `- pip install git-bob==0.10.0` consider updating the version number.
* Add a requirements.txt (or remove the line `- pip install -r requirements.txt`) from the yml file.

## Setting access tokens

In the gitlab web interface, go to project settings > CI / CD > Variables.
Add new variables named `GIT_BOB_LLM_NAME`, `OPENAI_API_KEY`, `GH_MODELS_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY` (depending on which LLM service provide you wish to use).

![img.png](images/install-gitlab1.png)

IMPORTANT: There seems a bug currently in the interface. After adding one variable, reload the page and click on "Add variable again".
Ensure that all created variables have the settings `Protected`, `Masked`, `Hidden` as shown here:

![img_6.png](images/install-gitlab2a.png)

Also configure a variable named `GITLAB_API_KEY`. You can retrieve this key from User > Preferences > Access Tokens.

![img.png](images/install-gitlab2.png)

When creating this key, make sure to select "api", "read repository" and "write repository":

![img_1.png](images/install-gitlab3.png)

This API key needs to be added as variable `GITLAB_API_KEY` as explained above.
DO NOT share this key with anyone and DO NOT save it in any file that is visible on the internet.

## Pipeline triggering

Next, we need to configure that new issues trigger git-bob.
You can also read more about how to do this in the [git lab documentation](https://docs.gitlab.com/ee/ci/triggers/).

Go to Project settings > CI / CD > Pipeline Trigger Tokens and create a token.
Copy the token, you will need it in the next step.

![img.png](images/install-gitlab4.png)

## Web hooks

Go to Project settings > Webhooks and create a new webhook.

![img_1.png](images/install-gitlab5.png)

As URL specify the following. Replace PROJECT_ID by your project's ID (a long number) and the TOKEN by the pipeline token you received above.
```
https://gitlab.com/api/v4/projects/PROJECT_ID/ref/main/trigger/pipeline?token=TOKEN&variables[CI_PIPELINE_TRIGGER_SOURCE]=issue
```

Then, configure masks to hide the token again, e.g. like this:

![img_2.png](images/install-gitlab6.png)

Also make sure to configure triggers: Comments, Issue events, Merge request events

![img_3.png](images/install-gitlab7.png)

In the list of web hooks it should then look like this:

![img_4.png](images/install-gitlab8.png)

## Testing the installation

Go to your repository and create a new issue ([example](https://gitlab.com/haesleinhuepf/git-bob-gitlab-playground/-/issues/2)). E.g. write a request like this and use a trigger word for git-bob:
```
What is the capital of Germany?

git-bob comment
```

If everything is configured correctly, git-bob will respond in your name, because it uses your Gitlab API key:

![img_5.png](images/install-gitlab9.png)
Loading
Loading