Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add durations to pytest run #913

Open
pedro-psb opened this issue Sep 12, 2024 · 1 comment
Open

Add durations to pytest run #913

pedro-psb opened this issue Sep 12, 2024 · 1 comment

Comments

@pedro-psb
Copy link
Member

Sometimes it's useful to know how long an individual test took to run on CI.

A recent case involved a sync to an amazon repository on pulp_rpm, which we are considering skipping depending on it's cost (because it's a pretty edgy case). On my machine this particular test took more than the whole CI run, so checking this timing locally was misleading.

Pytest has a builtin option to list those duration after a test session, which I think we could add to our pytest call.

It looks like this:

# --durations=0 list all test durations after each session
> pytest --durations=0 -v
==================================================================== test session starts =====================================================================
platform linux -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0 -- /tmp/foo/venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/foo
plugins: xdist-3.6.1
16 workers [4 items]
scheduling tests via LoadScheduling

test_foo.py::test_long
test_foo.py::test_bar
test_foo.py::test_bar3
test_foo.py::test_bar2
[gw1] [ 25%] SKIPPED test_foo.py::test_long
[gw0] [ 50%] PASSED test_foo.py::test_bar
[gw3] [ 75%] PASSED test_foo.py::test_bar3
[gw2] [100%] PASSED test_foo.py::test_bar2

===================================================================== slowest durations ======================================================================
2.00s call     test_foo.py::test_bar2
1.50s call     test_foo.py::test_bar3
1.00s call     test_foo.py::test_bar

(8 durations < 0.005s hidden.  Use -vv to show these durations.)
================================================================ 3 passed, 1 skipped in 3.91s ================================================================

I'm open to any objections.

@pedro-psb
Copy link
Member Author

Hmm, we do use that --durations in the performance test, so this was probably considered before.
Still, wanna hear opinions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant