pytest_xfail plugin

mark python test functions, classes or modules for conditional

skipping (skipif) or as expected-to-fail (xfail). Both declarations lead to special reporting and both can be systematically associated with functions, whole classes or modules. The difference between the two is that 'xfail' will still execute test functions but it will revert the outcome. A passing test is now a failure and failing test is expected. All skip conditions are reported at the end of test run through the terminal reporter.

skip a test function conditionally

Here is an example for skipping a test function on Python3:

@py.test.mark.skipif("sys.version_info >= (3,0)")
def test_function():
    ...

Conditions are specified as python expressions and can access the sys module. They can also access the config object and thus depend on command line or conftest options:

@py.test.mark.skipif("config.getvalue('db') is None")
def test_function(...):
    ...

conditionally mark a function as "expected to fail"

You can use the xfail keyword to mark your test functions as 'expected to fail':

@py.test.mark.xfail
def test_hello():
    ...

This test will be executed but no traceback will be reported when it fails. Instead terminal reporting will list it in the "expected to fail" or "unexpectedly passing" sections. As with skipif you may selectively expect a failure depending on platform:

@py.test.mark.xfail("sys.version_info >= (3,0)")
def test_function():
    ...

skip/xfail a whole test class or module

Instead of marking single functions you can skip a whole class of tests when runnign on a specific platform:

class TestSomething:
    skipif = "sys.platform == 'win32'"

Or you can mark all test functions as expected to fail for a specific test configuration:

xfail = "config.getvalue('db') == 'mysql'"

skip if a dependency cannot be imported

You can use a helper to skip on a failing import:

docutils = py.test.importorskip("docutils")

You can use this helper at module level or within a test or setup function.

You can aslo skip if a library does not have the right version:

docutils = py.test.importorskip("docutils", minversion="0.3")

The version will be read from the specified module's __version__ attribute.

dynamically skip from within a test or setup

If you want to skip the execution of a test you can call py.test.skip() within a test, a setup or from a funcarg factory function. Example:

def test_function():
    if not valid_config():
        py.test.skip("unsuppored configuration")

Start improving this plugin in 30 seconds

  1. Download pytest_xfail.py plugin source code
  2. put it somewhere as pytest_xfail.py into your import path
  3. a subsequent py.test run will use your local version

Checkout customize, other plugins or get in contact.