Merge pull request #16 from con/fail-on-slow

Make benchmark workflow fail if a benchmark slowed down too much
con · Nov 18, 2020 · 3bb9d0c · 3bb9d0c
2 parents 5e8d0f7 + d41cc82
commit 3bb9d0c
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml
@@ -81,3 +81,7 @@ jobs:
 
             - name: Compare benchmarks on previous & newest code
               run: asv compare refs/bm/merge-target refs/bm/pr
+
+            - name: Fail if any benchmarks have slowed down too much
+              run: |
+                ! asv compare --factor 1.2 --split refs/bm/merge-target refs/bm/pr | grep -q "got worse"