Style Guide: Tests

General Info

• We use pytest along with numpy (specifically the numpy.testing module) to write test cases for MDAnalysis.
• Do not use anything from numpy.testing that depends on nose such as assert_raises.
• The testsuite follows the same directory structure as the package.

This style guide is meant to be concise and allow developers to quickly learn which constructs to use and which ones to avoid. The rationale for these choices is discussed in issue #1444.

Conventions followed

Assertions

• Use plain assert statements for comparing single values. E.g., to check the length of an iterable foo, do assert len(foo) == expected_length

• To check equality for a single value up to a certain precision, use assert_almost_equals from numpy.testing. Do not manually round off the value and use plain assert statements. Do not use pytest.approx

• To compare an iterable, use assert_equal from numpy.testing. Do not iterate over and compare every single value.

• To compare an iterable up to a certain precision, use assert_almost_equals from numpy.testing.

• Do not use assert_array_equal or assert_array_almost_equal from numpy.testing to compare array/array-like data structures. Instead, use assert_equal or assert_almost_equal.

Grouping

• Use classes to group tests if it makes sense (e.g., if the test class will be inherited by another test class and the code can be reused). We prefer subclassing over parametrizing classes (for examples, have a look at the topology module).

• For everything else, use plain functions.

Testing exceptions

• To check for a particular exception, use the pytest.raises(ExceptionName) context manager.
• Do not use assert_raises from numpy.testing.
• Do not use the pytest.mark.raises decorator.

Testing warnings

• To check for a particular warning, use the pytest.warns(WarningName) context manager.

Failing tests

• To mark an expected failure, use pytest.mark.xfail decorator
• To manually fail a test, make a call to pytest.fail

Temporary files and directories

Use pytest's inbuilt tmpdir fixture

• To create a temp file temporary_file = str(tmpdir.join('test.pdb'))
• To use a temp directory as the working directory, use tmpdir.as_cwd context manager

Skipping tests

• To skip tests based on a condition, use pytest.mark.skipif(condition) decorator.
• To skip a test if a module is not available for importing, use pytest.importorskip('module_name')

Testing the same function with different inputs

• Use the pytest.mark.parametrize decorator to test the same function for different inputs

Fixtures

• Use fixtures as much as possible to reuse "resources" between test methods/functions (example: universe fixture). The rule of thumb is to use the largest possible scope for the fixture.