| Current File : //usr/share/doc/pytest-2.7.0/html/en/usage.html |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Usage and Invocations</title>
<link rel="stylesheet" href="_static/flasky.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '2.7.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="shortcut icon" href="_static/pytest1favi.ico"/>
<link rel="top" title="None" href="index.html" />
<link rel="up" title="Getting started basics" href="overview.html" />
<link rel="next" title="Good Integration Practises" href="goodpractises.html" />
<link rel="prev" title="Installation and Getting Started" href="getting-started.html" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9">
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="goodpractises.html" title="Good Integration Practises"
accesskey="N">next</a></li>
<li class="right" >
<a href="getting-started.html" title="Installation and Getting Started"
accesskey="P">previous</a> |</li>
<li><a href="contents.html">pytest-2.7.0</a> »</li>
<li><a href="overview.html" accesskey="U">Getting started basics</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="usage-and-invocations">
<span id="usage"></span><h1>Usage and Invocations<a class="headerlink" href="#usage-and-invocations" title="Permalink to this headline">¶</a></h1>
<div class="section" id="calling-pytest-through-python-m-pytest">
<span id="cmdline"></span><h2>Calling pytest through <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">pytest</span></tt><a class="headerlink" href="#calling-pytest-through-python-m-pytest" title="Permalink to this headline">¶</a></h2>
<p class="versionadded">
<span class="versionmodified">New in version 2.0.</span></p>
<p>If you use Python-2.5 or later you can invoke testing through the
Python interpreter from the command line:</p>
<div class="highlight-python"><pre>python -m pytest [...]</pre>
</div>
<p>This is equivalent to invoking the command line script <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">[...]</span></tt>
directly.</p>
</div>
<div class="section" id="getting-help-on-version-option-names-environment-variables">
<h2>Getting help on version, option names, environment variables<a class="headerlink" href="#getting-help-on-version-option-names-environment-variables" title="Permalink to this headline">¶</a></h2>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">version</span> <span class="c"># shows where pytest was imported from</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">fixtures</span> <span class="c"># show available builtin function arguments</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">h</span> <span class="o">|</span> <span class="o">--</span><span class="n">help</span> <span class="c"># show help on command line and config file options</span>
</pre></div>
</div>
</div>
<div class="section" id="stopping-after-the-first-or-n-failures">
<h2>Stopping after the first (or N) failures<a class="headerlink" href="#stopping-after-the-first-or-n-failures" title="Permalink to this headline">¶</a></h2>
<p>To stop the testing process after the first (N) failures:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">x</span> <span class="c"># stop after first failure</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">maxfail</span><span class="o">=</span><span class="mi">2</span> <span class="c"># stop after two failures</span>
</pre></div>
</div>
</div>
<div class="section" id="specifying-tests-selecting-tests">
<h2>Specifying tests / selecting tests<a class="headerlink" href="#specifying-tests-selecting-tests" title="Permalink to this headline">¶</a></h2>
<p>Several test run options:</p>
<div class="highlight-python"><pre>py.test test_mod.py # run tests in module
py.test somepath # run all tests below somepath
py.test -k stringexpr # only run tests with names that match the
# the "string expression", e.g. "MyClass and not method"
# will select TestMyClass.test_something
# but not TestMyClass.test_method_simple
py.test test_mod.py::test_func # only run tests that match the "node ID",
# e.g "test_mod.py::test_func" will select
# only test_func in test_mod.py</pre>
</div>
<p>Import ‘pkg’ and use its filesystem location to find and run tests:</p>
<div class="highlight-python"><pre>py.test --pyargs pkg # run all tests found below directory of pypkg</pre>
</div>
</div>
<div class="section" id="modifying-python-traceback-printing">
<h2>Modifying Python traceback printing<a class="headerlink" href="#modifying-python-traceback-printing" title="Permalink to this headline">¶</a></h2>
<p>Examples for modifying traceback printing:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">showlocals</span> <span class="c"># show local variables in tracebacks</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">l</span> <span class="c"># show local variables (shortcut)</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="nb">long</span> <span class="c"># the default informative traceback formatting</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">native</span> <span class="c"># the Python standard library formatting</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">short</span> <span class="c"># a shorter traceback format</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">line</span> <span class="c"># only one line per failure</span>
</pre></div>
</div>
</div>
<div class="section" id="dropping-to-pdb-python-debugger-on-failures">
<h2>Dropping to <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> (Python Debugger) on failures<a class="headerlink" href="#dropping-to-pdb-python-debugger-on-failures" title="Permalink to this headline">¶</a></h2>
<p>Python comes with a builtin Python debugger called <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a>. <tt class="docutils literal"><span class="pre">pytest</span></tt>
allows one to drop into the <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> prompt via a command line option:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pdb</span>
</pre></div>
</div>
<p>This will invoke the Python debugger on every failure. Often you might
only want to do this for the first failing test to understand a certain
failure situation:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">x</span> <span class="o">--</span><span class="n">pdb</span> <span class="c"># drop to PDB on first failure, then end test session</span>
<span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pdb</span> <span class="o">--</span><span class="n">maxfail</span><span class="o">=</span><span class="mi">3</span> <span class="c"># drop to PDB for first three failures</span>
</pre></div>
</div>
<p>Note that on any failure the exception information is stored on
<tt class="docutils literal"><span class="pre">sys.last_value</span></tt>, <tt class="docutils literal"><span class="pre">sys.last_type</span></tt> and <tt class="docutils literal"><span class="pre">sys.last_traceback</span></tt>. In
interactive use, this allows one to drop into postmortem debugging with
any debug tool. One can also manually access the exception information,
for example:</p>
<div class="highlight-python"><pre>>> import sys
>> sys.last_traceback.tb_lineno
42
>> sys.last_value
AssertionError('assert result == "ok"',)</pre>
</div>
</div>
<div class="section" id="setting-a-breakpoint-aka-set-trace">
<h2>Setting a breakpoint / aka <tt class="docutils literal"><span class="pre">set_trace()</span></tt><a class="headerlink" href="#setting-a-breakpoint-aka-set-trace" title="Permalink to this headline">¶</a></h2>
<p>If you want to set a breakpoint and enter the <tt class="docutils literal"><span class="pre">pdb.set_trace()</span></tt> you
can use a helper:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">test_function</span><span class="p">():</span>
<span class="o">...</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">set_trace</span><span class="p">()</span> <span class="c"># invoke PDB debugger and tracing</span>
</pre></div>
</div>
<p>Prior to pytest version 2.0.0 you could only enter <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> tracing if you disabled
capturing on the command line via <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">-s</span></tt>. In later versions, pytest
automatically disables its output capture when you enter <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> tracing:</p>
<ul class="simple">
<li>Output capture in other tests is not affected.</li>
<li>Any prior test output that has already been captured and will be processed as
such.</li>
<li>Any later output produced within the same test will not be captured and will
instead get sent directly to <tt class="docutils literal"><span class="pre">sys.stdout</span></tt>. Note that this holds true even
for test output occuring after you exit the interactive <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> tracing session
and continue with the regular test run.</li>
</ul>
<p>Since pytest version 2.4.0 you can also use the native Python
<tt class="docutils literal"><span class="pre">import</span> <span class="pre">pdb;pdb.set_trace()</span></tt> call to enter <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a> tracing without having to use
the <tt class="docutils literal"><span class="pre">pytest.set_trace()</span></tt> wrapper or explicitly disable pytest’s output
capturing via <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">-s</span></tt>.</p>
</div>
<div class="section" id="profiling-test-execution-duration">
<span id="durations"></span><h2>Profiling test execution duration<a class="headerlink" href="#profiling-test-execution-duration" title="Permalink to this headline">¶</a></h2>
<p>To get a list of the slowest 10 test durations:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">durations</span><span class="o">=</span><span class="mi">10</span>
</pre></div>
</div>
</div>
<div class="section" id="creating-junitxml-format-files">
<h2>Creating JUnitXML format files<a class="headerlink" href="#creating-junitxml-format-files" title="Permalink to this headline">¶</a></h2>
<p>To create result files which can be read by <a class="reference external" href="http://hudson-ci.org/">Hudson</a> or other Continuous
integration servers, use this invocation:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">junitxml</span><span class="o">=</span><span class="n">path</span>
</pre></div>
</div>
<p>to create an XML file at <tt class="docutils literal"><span class="pre">path</span></tt>.</p>
</div>
<div class="section" id="creating-resultlog-format-files">
<h2>Creating resultlog format files<a class="headerlink" href="#creating-resultlog-format-files" title="Permalink to this headline">¶</a></h2>
<p>To create plain-text machine-readable result files you can issue:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">resultlog</span><span class="o">=</span><span class="n">path</span>
</pre></div>
</div>
<p>and look at the content at the <tt class="docutils literal"><span class="pre">path</span></tt> location. Such files are used e.g.
by the <a class="reference external" href="http://buildbot.pypy.org/summary">PyPy-test</a> web page to show test results over several revisions.</p>
</div>
<div class="section" id="sending-test-report-to-online-pastebin-service">
<h2>Sending test report to online pastebin service<a class="headerlink" href="#sending-test-report-to-online-pastebin-service" title="Permalink to this headline">¶</a></h2>
<p><strong>Creating a URL for each test failure</strong>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pastebin</span><span class="o">=</span><span class="n">failed</span>
</pre></div>
</div>
<p>This will submit test run information to a remote Paste service and
provide a URL for each failure. You may select tests as usual or add
for example <tt class="docutils literal"><span class="pre">-x</span></tt> if you only want to send one particular failure.</p>
<p><strong>Creating a URL for a whole test session log</strong>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pastebin</span><span class="o">=</span><span class="nb">all</span>
</pre></div>
</div>
<p>Currently only pasting to the <a class="reference external" href="http://bpaste.net">http://bpaste.net</a> service is implemented.</p>
</div>
<div class="section" id="disabling-plugins">
<h2>Disabling plugins<a class="headerlink" href="#disabling-plugins" title="Permalink to this headline">¶</a></h2>
<p>To disable loading specific plugins at invocation time, use the <tt class="docutils literal"><span class="pre">-p</span></tt> option
together with the prefix <tt class="docutils literal"><span class="pre">no:</span></tt>.</p>
<p>Example: to disable loading the plugin <tt class="docutils literal"><span class="pre">doctest</span></tt>, which is responsible for
executing doctest tests from text files, invoke py.test like this:</p>
<div class="highlight-python"><pre>py.test -p no:doctest</pre>
</div>
</div>
<div class="section" id="calling-pytest-from-python-code">
<span id="pytest-main-usage"></span><h2>Calling pytest from Python code<a class="headerlink" href="#calling-pytest-from-python-code" title="Permalink to this headline">¶</a></h2>
<p class="versionadded">
<span class="versionmodified">New in version 2.0.</span></p>
<p>You can invoke <tt class="docutils literal"><span class="pre">pytest</span></tt> from Python code directly:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">()</span>
</pre></div>
</div>
<p>this acts as if you would call “py.test” from the command line.
It will not raise <tt class="docutils literal"><span class="pre">SystemExit</span></tt> but return the exitcode instead.
You can pass in options and arguments:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">([</span><span class="s">'-x'</span><span class="p">,</span> <span class="s">'mytestdir'</span><span class="p">])</span>
</pre></div>
</div>
<p>or pass in a string:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s">"-x mytestdir"</span><span class="p">)</span>
</pre></div>
</div>
<p>You can specify additional plugins to <tt class="docutils literal"><span class="pre">pytest.main</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of myinvoke.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">class</span> <span class="nc">MyPlugin</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">pytest_sessionfinish</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">print</span><span class="p">(</span><span class="s">"*** test run reporting finishing"</span><span class="p">)</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s">"-qq"</span><span class="p">,</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">MyPlugin</span><span class="p">()])</span>
</pre></div>
</div>
<p>Running it will show that <tt class="docutils literal"><span class="pre">MyPlugin</span></tt> was added and its
hook was invoked:</p>
<div class="highlight-python"><pre>$ python myinvoke.py
*** test run reporting finishing</pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="contents.html">
<img class="logo" src="_static/pytest1.png" alt="Logo"/>
</a></p><h3><a href="contents.html">Table Of Contents</a></h3>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="contents.html">Contents</a></li>
<li><a href="getting-started.html">Install</a></li>
<li><a href="example/index.html">Examples</a></li>
<li><a href="customize.html">Customize</a></li>
<li><a href="contact.html">Contact</a></li>
<li><a href="talks.html">Talks/Posts</a></li>
<li><a href="changelog.html">Changelog</a></li>
</ul>
<hr>
<ul>
<li><a class="reference internal" href="#">Usage and Invocations</a><ul>
<li><a class="reference internal" href="#calling-pytest-through-python-m-pytest">Calling pytest through <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">pytest</span></tt></a></li>
<li><a class="reference internal" href="#getting-help-on-version-option-names-environment-variables">Getting help on version, option names, environment variables</a></li>
<li><a class="reference internal" href="#stopping-after-the-first-or-n-failures">Stopping after the first (or N) failures</a></li>
<li><a class="reference internal" href="#specifying-tests-selecting-tests">Specifying tests / selecting tests</a></li>
<li><a class="reference internal" href="#modifying-python-traceback-printing">Modifying Python traceback printing</a></li>
<li><a class="reference internal" href="#dropping-to-pdb-python-debugger-on-failures">Dropping to PDB (Python Debugger) on failures</a></li>
<li><a class="reference internal" href="#setting-a-breakpoint-aka-set-trace">Setting a breakpoint / aka <tt class="docutils literal"><span class="pre">set_trace()</span></tt></a></li>
<li><a class="reference internal" href="#profiling-test-execution-duration">Profiling test execution duration</a></li>
<li><a class="reference internal" href="#creating-junitxml-format-files">Creating JUnitXML format files</a></li>
<li><a class="reference internal" href="#creating-resultlog-format-files">Creating resultlog format files</a></li>
<li><a class="reference internal" href="#sending-test-report-to-online-pastebin-service">Sending test report to online pastebin service</a></li>
<li><a class="reference internal" href="#disabling-plugins">Disabling plugins</a></li>
<li><a class="reference internal" href="#calling-pytest-from-python-code">Calling pytest from Python code</a></li>
</ul>
</li>
</ul>
<h3>Related Topics</h3>
<ul>
<li><a href="contents.html">Documentation overview</a><ul>
<li><a href="overview.html">Getting started basics</a><ul>
<li>Previous: <a href="getting-started.html" title="previous chapter">Installation and Getting Started</a></li>
<li>Next: <a href="goodpractises.html" title="next chapter">Good Integration Practises</a></li>
</ul></li>
</ul></li>
</ul><h3>Useful Links</h3>
<ul>
<li><a href="index.html">The pytest Website</a></li>
<li><a href="contributing.html">Contribution Guide</a></li>
<li><a href="https://pypi.python.org/pypi/pytest">pytest @ PyPI</a></li>
<li><a href="https://bitbucket.org/pytest-dev/pytest/">pytest @ Bitbucket</a></li>
<li><a href="http://pytest.org/latest/plugins_index/index.html">3rd party plugins</a></li>
<li><a href="https://bitbucket.org/pytest-dev/pytest/issues?status=new&status=open">Issue Tracker</a></li>
<li><a href="http://pytest.org/latest/pytest.pdf">PDF Documentation</a>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
© Copyright 2014, holger krekel.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
</div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-7597274-13']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</body>
</html>