Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed Github Workflows #251

Merged
merged 7 commits into from
Sep 28, 2023
Merged

Fixed Github Workflows #251

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a0a5ead
Made run-tests workflow dependent on update-testing-data workflow
pauladkisson Sep 28, 2023
ce624f8
update awscli version
pauladkisson Sep 28, 2023
cfab180
pin wheel version to avoid future dependency conflicts
pauladkisson Sep 28, 2023
2158025
updated awscli version in run-tests
pauladkisson Sep 28, 2023
56cd15e
removed dependent run of run-tests
pauladkisson Sep 28, 2023
587865d
commented out awscli commands only
pauladkisson Sep 28, 2023
94ddfa0
refactored to trigger run-tests after update-testing-data
pauladkisson Sep 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 5 additions & 16 deletions .github/workflows/run-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
name: Full Tests
on:
schedule:
- cron: "0 0 * * *" # daily
pull_request:
workflow_dispatch:
workflow_run:
workflows: [update-testing-data]
types: [completed]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If going this route, you'll probably want to add the 'on success' logic to allow early exit (otherwise will proceed to run tests and fail when it doesn't find a certain file/folder); I wonder if success is an allowed type for the trigger itself?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seems reasonable. Apparently on: workflow_run only works on the default branch (see Issue), so I can't test it before merging? Kind of annoying.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that seems reasonable. Apparently on: workflow_run only works on the default branch (see Issue), so I can't test it before merging? Kind of annoying.

This kind of thing is typical of GitHub Action devops stuff - it's the one situation where committing to main to debug stuff is OK IMO

I'll approve and merge this and let you continue messing with it


jobs:
run:
Expand All @@ -29,18 +30,17 @@ jobs:
pip install pytest-xdist
git config --global user.email "[email protected]"
git config --global user.name "CI Almighty"
pip install wheel # needed for scanimage
pip install wheel==0.41.2 # needed for scanimage

- name: Install roiextractors with minimal requirements
run: pip install .[test]

- name: Run minimal tests
run: pytest tests/test_internals -n auto --dist loadscope

- name: Test full installation (-e needed for codecov report)
run: pip install -e .[full]



- name: Get ophys_testing_data current head hash
id: ophys
run: echo "::set-output name=HASH_OPHYS_DATASET::$(git ls-remote https://gin.g-node.org/CatalystNeuro/ophys_testing_data.git HEAD | cut -f1)"
Expand All @@ -50,17 +50,6 @@ jobs:
with:
path: ./ophys_testing_data
key: ophys-datasets-042023-${{ matrix.os }}-${{ steps.ophys.outputs.HASH_OPHYS_DATASET }}
- if: steps.cache-ophys-datasets.outputs.cache-hit == false
name: Install and configure AWS CLI
run: |
pip install awscli==1.25.27
aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}
aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- if: steps.cache-ophys-datasets.outputs.cache-hit == false
name: Download data from S3
run: aws s3 cp --recursive s3://${{ secrets.S3_GIN_BUCKET }}//ophys_testing_data ./ophys_testing_data



- name: Run full pytest with coverage
run: pytest -n auto --dist loadscope --cov=./ --cov-report xml:./codecov.xml
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/update-testing-data.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
name: Update testing data
name: update-testing-data
on:
schedule:
- cron: "0 0 * * *" # daily
workflow_dispatch:

jobs:
Expand All @@ -24,7 +26,7 @@ jobs:
pip install -U pip
git config --global user.email "[email protected]"
git config --global user.name "CI Almighty"
pip install wheel # needed for scanimage
pip install wheel==0.41.2 # needed for scanimage

- name: Get ophys_testing_data current head hash
id: ophys
Expand All @@ -38,7 +40,7 @@ jobs:
- if: steps.cache-ophys-datasets.outputs.cache-hit == false
name: Install and configure AWS CLI
run: |
pip install awscli==1.25.27
pip install awscli==1.29.56
aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}
aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- if: steps.cache-ophys-datasets.outputs.cache-hit == false
Expand Down
Loading