You can see command line options by running:
py.test -h
This will display all available command line options in your specific environment.
A unique feature of py.test are its conftest.py files which allow to:
or set particular variables to influence the testing process:
You may put a conftest.py files in your project root directory or into your package directory if you want to add project-specific test options.
py.test loads all conftest.py files upwards from the command line file arguments. It usually looks up configuration values right-to-left, i.e. the closer conftest files will be checked first. This means you can have a conftest.py in your very home directory to have some global configuration values.
py.test will lookup option values in this order:
To find out about the particular switches and type:
py.test --help-config
This will print information about all options in your environment, including your local plugins.
You can create directories by calling one of two methods on the config object:
temporary directories are created as sub directories of a per-session testdir and will keep around the directories of the last three test runs. You can set the base temporary directory through the command line --basetemp` option. When distributing tests on the same machine, py.test takes care to configure a basetemp directory for the sub processes such that all temporary data lands below below a single per-test run basetemp directory.
py.test implements much of its functionality by calling well specified hooks. Python modules which contain such hook functions are called plugins. Hook functions are discovered in conftest.py files or in named plugins. conftest.py files are sometimes called "anonymous" or conftest plugins. They are useful for keeping test extensions close to your application. Named plugins are normal python modules or packages that can be distributed separately. Named plugins need to follow a naming pattern; they have an all lowercase pytest_ prefixed name. While conftest plugins are discovered automatically, named plugins must be explicitely specified.
py.test loads plugin modules at tool startup in the following way:
You can specify plugins in a test module or a plugin like this:
pytest_plugins = "name1", "name2",
When the test module or plugin is loaded the specified plugins will be loaded. If you specify plugins without the pytest_ prefix it will be automatically added. All plugin names must be lowercase.
The purpose of conftest.py files is to allow project-specific test configuration. They thus make for a good place to implement project-specific test related features through hooks. For example you may set the collect_ignore variable depending on a command line option by defining the following hook in a conftest.py file:
# ./conftest.py in your root or package dir
collect_ignore = ['hello', 'test_world.py']
def pytest_addoption(parser):
parser.addoption("--runall", action="store_true", default=False)
def pytest_configure(config):
if config.getvalue("runall"):
collect_ignore[:] = []
If you want to make your plugin publically available, you can use setuptools or Distribute which both allow to register an entry point. py.test will register all objects with the pytest11 entry point. To make your plugin available you may insert the following lines in your setuptools/distribute-based setup-invocation:
# sample ./setup.py file
from setuptools import setup
setup(
name="myproject",
packages = ['myproject']
# the following makes a plugin available to py.test
entry_points = {
'pytest11': [
'name_of_plugin = myproject.pluginmodule',
]
},
)
If a package is installed with this setup, py.test will load myproject.pluginmodule under the name_of_plugin name and use it as a plugin.
If a plugin wants to collaborate with code from another plugin it can obtain a reference through the plugin manager like this:
plugin = config.pluginmanager.getplugin("name_of_plugin")
If you want to look at the names of existing plugins, use the --traceconfig option.
py.test calls hooks functions to implement its test collection, running and reporting process. When py.test loads a plugin it validates that all hook functions conform to the hook definition specification.
The hook function name and its argument names need to match exactly but it is allowed for an implementation to accept less parameters. You'll get useful errors on mistyped hook or argument names. Read on for some introductory information on particular hooks. It's sensible to look at existing plugins so see example usages and start off with your own plugin.
When the test tool starts up it will invoke all hooks that add command line options in the python standard optparse style.
def pytest_addoption(parser):
""" add command line options. """"
parser.addoption("--myopt", dest="myopt", action="store_true")
After all these hooks have been called, the command line is parser and a config object is created and another hook is invoked, for example:
def pytest_configure(config):
config.getvalue("myopt")
When the test run finishes this corresponding finalizer hook is called:
def pytest_unconfigure(config):
...
If you want to make global helper functions or objects available to your test code you can implement:
def pytest_namespace():
""" return dictionary with items to be made available on py.test. namespace """
All such returned items will be made available directly on the py.test namespace.
If you want to provide helpers that are specific to a test function run or need to be setup per test function run, please refer to the funcargs mechanism.
Each test item is usually executed by calling the following three hooks:
pytest_runtest_setup(item)
pytest_runtest_call(item)
pytest_runtest_teardown(item)
For each of the three invocations a call object encapsulates information about the outcome of the call and is subsequently used to make a report object:
report = hook.pytest_runtest_makereport(item, call)
For example, the pytest_pdb plugin uses this hook to activate interactive debugging on failures when --pdb is specified on the command line.
Usually three reports will be generated for a single test item for each of the three runtest hooks respectively. If pytest_runtest_setup fails then pytest_runtest_teardown will be called but not pytest_runtest_call.
Each of the up to three reports is eventually fed to the logreport hook:
pytest_runtest_logreport(report)
A report object contains status and reporting information:
report.longrepr = string/lines/object to print
report.when = "setup", "call" or "teardown"
report.shortrepr = letter for progress-report
report.passed = True or False
report.failed = True or False
report.skipped = True or False
The terminal plugin uses this hook to print information about a test run.
The whole protocol described here is implemented via this hook:
pytest_runtest_protocol(item) -> True
The call object contains information about a performed call:
call.excinfo = ExceptionInfo object or None
call.when = "setup", "call" or "teardown"
call.outerr = None or tuple of strings representing captured stdout/stderr
py.test calls the following two fundamental hooks for collecting files and directories:
def pytest_collect_directory(path, parent):
""" return Collection node or None for the given path. """
def pytest_collect_file(path, parent):
""" return Collection node or None for the given path. """
Both return a collection node for a given path. All returned nodes from all hook implementations will participate in the collection and running protocol. The parent object is the parent node and may be used to access command line options via the parent.config object.
For influencing the collection of objects in Python modules you can use the following hook:
def pytest_pycollect_makeitem(collector, name, obj):
""" return custom item/collector for a python object in a module, or None. """
This hook will be called for each Python object in a collected Python module. The return value is a custom collection node or None.
(alpha) For distributed testing it can be useful to prepare the remote environment. For this you can implement the newgateway hook:
def pytest_gwmanage_newgateway(gateway, platinfo):
""" called after a gateway is instantiated. """
The gateway object here has a spec attribute which is an execnet.XSpec object, which has attributes that map key/values as specified from a --txspec option. The platinfo object is a dictionary with information about the remote process:
The collecting process is iterative so that distribution and execution of tests can start as soon as the first test item is collected. Collection nodes with children are called "Collectors" and terminal nodes are called "Items". Here is an example of such a tree, generated with the command py.test --collectonly py/xmlobj:
<Directory 'xmlobj'> <Directory 'testing'> <Module 'test_html.py' (py.__.xmlobj.testing.test_html)> <Function 'test_html_name_stickyness'> <Function 'test_stylenames'> <Function 'test_class_None'> <Function 'test_alternating_style'> <Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)> <Function 'test_tag_with_text'> <Function 'test_class_identity'> <Function 'test_tag_with_text_and_attributes'> <Function 'test_tag_with_subclassed_attr_simple'> <Function 'test_tag_nested'> <Function 'test_tag_xmlname'>
By default all directories not starting with a dot are traversed, looking for test_*.py and *_test.py files. Those Python files are imported under their package name.
The Module collector looks for test functions and test classes and methods. Test functions and methods are prefixed test by default. Test classes must start with a capitalized Test prefix.
Test modules are imported under their fully qualified name. Given a filesystem fspath it is constructed as follows: