| Current File : //usr/share/doc/pytest-2.7.0/html/en/example/parametrize.html |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Parametrizing tests</title>
<link rel="stylesheet" href="../_static/flasky.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.7.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="shortcut icon" href="../_static/pytest1favi.ico"/>
<link rel="top" title="None" href="../index.html" />
<link rel="up" title="Usages and Examples" href="index.html" />
<link rel="next" title="Working with custom markers" href="markers.html" />
<link rel="prev" title="Basic patterns and examples" href="simple.html" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9">
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="markers.html" title="Working with custom markers"
accesskey="N">next</a></li>
<li class="right" >
<a href="simple.html" title="Basic patterns and examples"
accesskey="P">previous</a> |</li>
<li><a href="../contents.html">pytest-2.7.0</a> »</li>
<li><a href="index.html" accesskey="U">Usages and Examples</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="parametrizing-tests">
<span id="paramexamples"></span><h1>Parametrizing tests<a class="headerlink" href="#parametrizing-tests" title="Permalink to this headline">¶</a></h1>
<p><tt class="docutils literal"><span class="pre">pytest</span></tt> allows to easily parametrize test functions.
For basic docs, see <a class="reference internal" href="../parametrize.html#parametrize-basics"><em>Parametrizing fixtures and test functions</em></a>.</p>
<p>In the following we provide some examples using
the builtin mechanisms.</p>
<div class="section" id="generating-parameters-combinations-depending-on-command-line">
<h2>Generating parameters combinations, depending on command line<a class="headerlink" href="#generating-parameters-combinations-depending-on-command-line" title="Permalink to this headline">¶</a></h2>
<p>Let’s say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument. Let’s first write a simple (do-nothing) computation test:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_compute.py</span>
<span class="k">def</span> <span class="nf">test_compute</span><span class="p">(</span><span class="n">param1</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">param1</span> <span class="o"><</span> <span class="mi">4</span>
</pre></div>
</div>
<p>Now we add a test configuration like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span>
<span class="k">def</span> <span class="nf">pytest_addoption</span><span class="p">(</span><span class="n">parser</span><span class="p">):</span>
<span class="n">parser</span><span class="o">.</span><span class="n">addoption</span><span class="p">(</span><span class="s">"--all"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s">"store_true"</span><span class="p">,</span>
<span class="n">help</span><span class="o">=</span><span class="s">"run all combinations"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'param1'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span>
<span class="k">if</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">option</span><span class="o">.</span><span class="n">all</span><span class="p">:</span>
<span class="n">end</span> <span class="o">=</span> <span class="mi">5</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">end</span> <span class="o">=</span> <span class="mi">2</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s">"param1"</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="n">end</span><span class="p">))</span>
</pre></div>
</div>
<p>This means that we only run 2 tests if we do not pass <tt class="docutils literal"><span class="pre">--all</span></tt>:</p>
<div class="highlight-python"><pre>$ py.test -q test_compute.py
..
2 passed in 0.01 seconds</pre>
</div>
<p>We run only two computations, so we see two dots.
let’s run the full monty:</p>
<div class="highlight-python"><pre>$ py.test -q --all
....F
================================= FAILURES =================================
_____________________________ test_compute[4] ______________________________
param1 = 4
def test_compute(param1):
> assert param1 < 4
E assert 4 < 4
test_compute.py:3: AssertionError
1 failed, 4 passed in 0.01 seconds</pre>
</div>
<p>As expected when running the full range of <tt class="docutils literal"><span class="pre">param1</span></tt> values
we’ll get an error on the last one.</p>
</div>
<div class="section" id="different-options-for-test-ids">
<h2>Different options for test IDs<a class="headerlink" href="#different-options-for-test-ids" title="Permalink to this headline">¶</a></h2>
<p>pytest will build a string that is the test ID for each set of values in a
parametrized test. These IDs can be used with <tt class="docutils literal"><span class="pre">-k</span></tt> to select specific cases
to run, and they will also identify the specific case when one is failing.
Running pytest with <tt class="docutils literal"><span class="pre">--collect-only</span></tt> will show the generated IDs.</p>
<p>Numbers, strings, booleans and None will have their usual string representation
used in the test ID. For other objects, pytest will make a string based on
the argument name:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># contents of test_time.py</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
<span class="n">testdata</span> <span class="o">=</span> <span class="p">[(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2001</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2001</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">11</span><span class="p">),</span> <span class="n">timedelta</span><span class="p">(</span><span class="mi">1</span><span class="p">)),</span>
<span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2001</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">11</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2001</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">timedelta</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)),</span>
<span class="p">]</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s">"a,b,expected"</span><span class="p">,</span> <span class="n">testdata</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_timedistance_v0</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">expected</span><span class="p">):</span>
<span class="n">diff</span> <span class="o">=</span> <span class="n">a</span> <span class="o">-</span> <span class="n">b</span>
<span class="k">assert</span> <span class="n">diff</span> <span class="o">==</span> <span class="n">expected</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s">"a,b,expected"</span><span class="p">,</span> <span class="n">testdata</span><span class="p">,</span> <span class="n">ids</span><span class="o">=</span><span class="p">[</span><span class="s">"forward"</span><span class="p">,</span> <span class="s">"backward"</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">test_timedistance_v1</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">expected</span><span class="p">):</span>
<span class="n">diff</span> <span class="o">=</span> <span class="n">a</span> <span class="o">-</span> <span class="n">b</span>
<span class="k">assert</span> <span class="n">diff</span> <span class="o">==</span> <span class="n">expected</span>
<span class="k">def</span> <span class="nf">idfn</span><span class="p">(</span><span class="n">val</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="p">(</span><span class="n">datetime</span><span class="p">,)):</span>
<span class="c"># note this wouldn't show any hours/minutes/seconds</span>
<span class="k">return</span> <span class="n">val</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">'%Y%m</span><span class="si">%d</span><span class="s">'</span><span class="p">)</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s">"a,b,expected"</span><span class="p">,</span> <span class="n">testdata</span><span class="p">,</span> <span class="n">ids</span><span class="o">=</span><span class="n">idfn</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_timedistance_v2</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">expected</span><span class="p">):</span>
<span class="n">diff</span> <span class="o">=</span> <span class="n">a</span> <span class="o">-</span> <span class="n">b</span>
<span class="k">assert</span> <span class="n">diff</span> <span class="o">==</span> <span class="n">expected</span>
</pre></div>
</div>
<p>In <tt class="docutils literal"><span class="pre">test_timedistance_v0</span></tt>, we let pytest generate the test IDs.</p>
<p>In <tt class="docutils literal"><span class="pre">test_timedistance_v1</span></tt>, we specified <tt class="docutils literal"><span class="pre">ids</span></tt> as a list of strings which were
used as the test IDs. These are succinct, but can be a pain to maintain.</p>
<p>In <tt class="docutils literal"><span class="pre">test_timedistance_v2</span></tt>, we specified <tt class="docutils literal"><span class="pre">ids</span></tt> as a function that can generate a
string representation to make part of the test ID. So our <tt class="docutils literal"><span class="pre">datetime</span></tt> values use the
label generated by <tt class="docutils literal"><span class="pre">idfn</span></tt>, but because we didn’t generate a label for <tt class="docutils literal"><span class="pre">timedelta</span></tt>
objects, they are still using the default pytest representation:</p>
<div class="highlight-python"><pre>$ py.test test_time.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.7.0
rootdir: /tmp/doc-exec-169, inifile:
============================= in 0.00 seconds =============================
ERROR: file not found: test_time.py</pre>
</div>
</div>
<div class="section" id="a-quick-port-of-testscenarios">
<h2>A quick port of “testscenarios”<a class="headerlink" href="#a-quick-port-of-testscenarios" title="Permalink to this headline">¶</a></h2>
<p>Here is a quick port to run tests configured with <a class="reference external" href="http://pypi.python.org/pypi/testscenarios/">test scenarios</a>,
an add-on from Robert Collins for the standard unittest framework. We
only have to work a bit to construct the correct arguments for pytest’s
<a class="reference internal" href="../parametrize.html#_pytest.python.Metafunc.parametrize" title="_pytest.python.Metafunc.parametrize"><tt class="xref py py-func docutils literal"><span class="pre">Metafunc.parametrize()</span></tt></a>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_scenarios.py</span>
<span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="n">idlist</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">argvalues</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">scenario</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">cls</span><span class="o">.</span><span class="n">scenarios</span><span class="p">:</span>
<span class="n">idlist</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">scenario</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">items</span> <span class="o">=</span> <span class="n">scenario</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="n">argnames</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]</span>
<span class="n">argvalues</span><span class="o">.</span><span class="n">append</span><span class="p">(([</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]))</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="n">argnames</span><span class="p">,</span> <span class="n">argvalues</span><span class="p">,</span> <span class="n">ids</span><span class="o">=</span><span class="n">idlist</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="s">"class"</span><span class="p">)</span>
<span class="n">scenario1</span> <span class="o">=</span> <span class="p">(</span><span class="s">'basic'</span><span class="p">,</span> <span class="p">{</span><span class="s">'attribute'</span><span class="p">:</span> <span class="s">'value'</span><span class="p">})</span>
<span class="n">scenario2</span> <span class="o">=</span> <span class="p">(</span><span class="s">'advanced'</span><span class="p">,</span> <span class="p">{</span><span class="s">'attribute'</span><span class="p">:</span> <span class="s">'value2'</span><span class="p">})</span>
<span class="k">class</span> <span class="nc">TestSampleWithScenarios</span><span class="p">:</span>
<span class="n">scenarios</span> <span class="o">=</span> <span class="p">[</span><span class="n">scenario1</span><span class="p">,</span> <span class="n">scenario2</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">test_demo1</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attribute</span><span class="p">):</span>
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">attribute</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_demo2</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attribute</span><span class="p">):</span>
<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">attribute</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span>
</pre></div>
</div>
<p>this is a fully self-contained example which you can run with:</p>
<div class="highlight-python"><pre>$ py.test test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.7.0
rootdir: /tmp/doc-exec-169, inifile:
collected 4 items
test_scenarios.py ....
========================= 4 passed in 0.01 seconds =========================</pre>
</div>
<p>If you just collect tests you’ll also nicely see ‘advanced’ and ‘basic’ as variants for the test function:</p>
<div class="highlight-python"><pre>$ py.test --collect-only test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.7.0
rootdir: /tmp/doc-exec-169, inifile:
collected 4 items
<Module 'test_scenarios.py'>
<Class 'TestSampleWithScenarios'>
<Instance '()'>
<Function 'test_demo1[basic]'>
<Function 'test_demo2[basic]'>
<Function 'test_demo1[advanced]'>
<Function 'test_demo2[advanced]'>
============================= in 0.01 seconds =============================</pre>
</div>
<p>Note that we told <tt class="docutils literal"><span class="pre">metafunc.parametrize()</span></tt> that your scenario values
should be considered class-scoped. With pytest-2.3 this leads to a
resource-based ordering.</p>
</div>
<div class="section" id="deferring-the-setup-of-parametrized-resources">
<h2>Deferring the setup of parametrized resources<a class="headerlink" href="#deferring-the-setup-of-parametrized-resources" title="Permalink to this headline">¶</a></h2>
<p>The parametrization of test functions happens at collection
time. It is a good idea to setup expensive resources like DB
connections or subprocess only when the actual test is run.
Here is a simple example how you can achieve that, first
the actual test requiring a <tt class="docutils literal"><span class="pre">db</span></tt> object:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_backends.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">test_db_initialized</span><span class="p">(</span><span class="n">db</span><span class="p">):</span>
<span class="c"># a dummy test</span>
<span class="k">if</span> <span class="n">db</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span> <span class="o">==</span> <span class="s">"DB2"</span><span class="p">:</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">fail</span><span class="p">(</span><span class="s">"deliberately failing for demo purposes"</span><span class="p">)</span>
</pre></div>
</div>
<p>We can now add a test configuration that generates two invocations of
the <tt class="docutils literal"><span class="pre">test_db_initialized</span></tt> function and also implements a factory that
creates a database object for the actual test invocations:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'db'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s">"db"</span><span class="p">,</span> <span class="p">[</span><span class="s">'d1'</span><span class="p">,</span> <span class="s">'d2'</span><span class="p">],</span> <span class="n">indirect</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">DB1</span><span class="p">:</span>
<span class="s">"one database object"</span>
<span class="k">class</span> <span class="nc">DB2</span><span class="p">:</span>
<span class="s">"alternative database object"</span>
<span class="nd">@pytest.fixture</span>
<span class="k">def</span> <span class="nf">db</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">if</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span> <span class="o">==</span> <span class="s">"d1"</span><span class="p">:</span>
<span class="k">return</span> <span class="n">DB1</span><span class="p">()</span>
<span class="k">elif</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span> <span class="o">==</span> <span class="s">"d2"</span><span class="p">:</span>
<span class="k">return</span> <span class="n">DB2</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">"invalid internal test config"</span><span class="p">)</span>
</pre></div>
</div>
<p>Let’s first see how it looks like at collection time:</p>
<div class="highlight-python"><pre>$ py.test test_backends.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.7.0
rootdir: /tmp/doc-exec-169, inifile:
collected 2 items
<Module 'test_backends.py'>
<Function 'test_db_initialized[d1]'>
<Function 'test_db_initialized[d2]'>
============================= in 0.00 seconds =============================</pre>
</div>
<p>And then when we run the test:</p>
<div class="highlight-python"><pre>$ py.test -q test_backends.py
.F
================================= FAILURES =================================
_________________________ test_db_initialized[d2] __________________________
db = <conftest.DB2 object at 0x2b160a531f98>
def test_db_initialized(db):
# a dummy test
if db.__class__.__name__ == "DB2":
> pytest.fail("deliberately failing for demo purposes")
E Failed: deliberately failing for demo purposes
test_backends.py:6: Failed
1 failed, 1 passed in 0.01 seconds</pre>
</div>
<p>The first invocation with <tt class="docutils literal"><span class="pre">db</span> <span class="pre">==</span> <span class="pre">"DB1"</span></tt> passed while the second with <tt class="docutils literal"><span class="pre">db</span> <span class="pre">==</span> <span class="pre">"DB2"</span></tt> failed. Our <tt class="docutils literal"><span class="pre">db</span></tt> fixture function has instantiated each of the DB values during the setup phase while the <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> generated two according calls to the <tt class="docutils literal"><span class="pre">test_db_initialized</span></tt> during the collection phase.</p>
</div>
<div class="section" id="parametrizing-test-methods-through-per-class-configuration">
<h2>Parametrizing test methods through per-class configuration<a class="headerlink" href="#parametrizing-test-methods-through-per-class-configuration" title="Permalink to this headline">¶</a></h2>
<p>Here is an example <tt class="docutils literal"><span class="pre">pytest_generate_function</span></tt> function implementing a
parametrization scheme similar to Michael Foord’s <a class="reference external" href="http://code.google.com/p/unittest-ext/source/browse/trunk/params.py">unittest
parameterizer</a> but in a lot less code:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of ./test_parametrize.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="c"># called once per each test function</span>
<span class="n">funcarglist</span> <span class="o">=</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">cls</span><span class="o">.</span><span class="n">params</span><span class="p">[</span><span class="n">metafunc</span><span class="o">.</span><span class="n">function</span><span class="o">.</span><span class="n">__name__</span><span class="p">]</span>
<span class="n">argnames</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">funcarglist</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="n">argnames</span><span class="p">,</span> <span class="p">[[</span><span class="n">funcargs</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">argnames</span><span class="p">]</span>
<span class="k">for</span> <span class="n">funcargs</span> <span class="ow">in</span> <span class="n">funcarglist</span><span class="p">])</span>
<span class="k">class</span> <span class="nc">TestClass</span><span class="p">:</span>
<span class="c"># a map specifying multiple argument sets for a test method</span>
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">'test_equals'</span><span class="p">:</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">2</span><span class="p">),</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">3</span><span class="p">),</span> <span class="p">],</span>
<span class="s">'test_zerodivision'</span><span class="p">:</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span> <span class="p">],</span>
<span class="p">}</span>
<span class="k">def</span> <span class="nf">test_equals</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">a</span> <span class="o">==</span> <span class="n">b</span>
<span class="k">def</span> <span class="nf">test_zerodivision</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="ne">ZeroDivisionError</span><span class="p">,</span> <span class="s">"a/b"</span><span class="p">)</span>
</pre></div>
</div>
<p>Our test generator looks up a class-level definition which specifies which
argument sets to use for each test function. Let’s run it:</p>
<div class="highlight-python"><pre>$ py.test -q
F..
================================= FAILURES =================================
________________________ TestClass.test_equals[1-2] ________________________
self = <test_parametrize.TestClass object at 0x2ab66352a978>, a = 1, b = 2
def test_equals(self, a, b):
> assert a == b
E assert 1 == 2
test_parametrize.py:18: AssertionError
1 failed, 2 passed in 0.01 seconds</pre>
</div>
</div>
<div class="section" id="indirect-parametrization-with-multiple-fixtures">
<h2>Indirect parametrization with multiple fixtures<a class="headerlink" href="#indirect-parametrization-with-multiple-fixtures" title="Permalink to this headline">¶</a></h2>
<p>Here is a stripped down real-life example of using parametrized
testing for testing serialization of objects between different python
interpreters. We define a <tt class="docutils literal"><span class="pre">test_basic_objects</span></tt> function which
is to be run with different sets of arguments for its three arguments:</p>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">python1</span></tt>: first python interpreter, run to pickle-dump an object to a file</li>
<li><tt class="docutils literal"><span class="pre">python2</span></tt>: second interpreter, run to pickle-load an object from a file</li>
<li><tt class="docutils literal"><span class="pre">obj</span></tt>: object to be dumped/loaded</li>
</ul>
<div class="highlight-python"><div class="highlight"><pre><span class="sd">"""</span>
<span class="sd">module containing a parametrized tests testing cross-python</span>
<span class="sd">serialization via the pickle module.</span>
<span class="sd">"""</span>
<span class="kn">import</span> <span class="nn">py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="n">pythonlist</span> <span class="o">=</span> <span class="p">[</span><span class="s">'python2.6'</span><span class="p">,</span> <span class="s">'python2.7'</span><span class="p">,</span> <span class="s">'python3.3'</span><span class="p">]</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="n">pythonlist</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">python1</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">tmpdir</span><span class="p">):</span>
<span class="n">picklefile</span> <span class="o">=</span> <span class="n">tmpdir</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s">"data.pickle"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">Python</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">,</span> <span class="n">picklefile</span><span class="p">)</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="n">pythonlist</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">python2</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">python1</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Python</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">,</span> <span class="n">python1</span><span class="o">.</span><span class="n">picklefile</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Python</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">version</span><span class="p">,</span> <span class="n">picklefile</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span> <span class="o">=</span> <span class="n">py</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">local</span><span class="o">.</span><span class="n">sysfind</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">:</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">skip</span><span class="p">(</span><span class="s">"</span><span class="si">%r</span><span class="s"> not found"</span> <span class="o">%</span><span class="p">(</span><span class="n">version</span><span class="p">,))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span> <span class="o">=</span> <span class="n">picklefile</span>
<span class="k">def</span> <span class="nf">dumps</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="n">dumpfile</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="o">.</span><span class="n">dirpath</span><span class="p">(</span><span class="s">"dump.py"</span><span class="p">)</span>
<span class="n">dumpfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">py</span><span class="o">.</span><span class="n">code</span><span class="o">.</span><span class="n">Source</span><span class="p">(</span><span class="s">"""</span>
<span class="s"> import pickle</span>
<span class="s"> f = open(</span><span class="si">%r</span><span class="s">, 'wb')</span>
<span class="s"> s = pickle.dump(</span><span class="si">%r</span><span class="s">, f, protocol=2)</span>
<span class="s"> f.close()</span>
<span class="s"> """</span> <span class="o">%</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="p">),</span> <span class="n">obj</span><span class="p">)))</span>
<span class="n">py</span><span class="o">.</span><span class="n">process</span><span class="o">.</span><span class="n">cmdexec</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">,</span> <span class="n">dumpfile</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">load_and_is_true</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">):</span>
<span class="n">loadfile</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="o">.</span><span class="n">dirpath</span><span class="p">(</span><span class="s">"load.py"</span><span class="p">)</span>
<span class="n">loadfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">py</span><span class="o">.</span><span class="n">code</span><span class="o">.</span><span class="n">Source</span><span class="p">(</span><span class="s">"""</span>
<span class="s"> import pickle</span>
<span class="s"> f = open(</span><span class="si">%r</span><span class="s">, 'rb')</span>
<span class="s"> obj = pickle.load(f)</span>
<span class="s"> f.close()</span>
<span class="s"> res = eval(</span><span class="si">%r</span><span class="s">)</span>
<span class="s"> if not res:</span>
<span class="s"> raise SystemExit(1)</span>
<span class="s"> """</span> <span class="o">%</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="p">),</span> <span class="n">expression</span><span class="p">)))</span>
<span class="k">print</span> <span class="p">(</span><span class="n">loadfile</span><span class="p">)</span>
<span class="n">py</span><span class="o">.</span><span class="n">process</span><span class="o">.</span><span class="n">cmdexec</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">,</span> <span class="n">loadfile</span><span class="p">))</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s">"obj"</span><span class="p">,</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="p">{},</span> <span class="p">{</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">},])</span>
<span class="k">def</span> <span class="nf">test_basic_objects</span><span class="p">(</span><span class="n">python1</span><span class="p">,</span> <span class="n">python2</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="n">python1</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">python2</span><span class="o">.</span><span class="n">load_and_is_true</span><span class="p">(</span><span class="s">"obj == </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">obj</span><span class="p">)</span>
</pre></div>
</div>
<p>Running it results in some skips if we don’t have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):</p>
<div class="highlight-python"><pre>. $ py.test -rs -q multipython.py
...........................
27 passed in 1.70 seconds</pre>
</div>
</div>
<div class="section" id="indirect-parametrization-of-optional-implementations-imports">
<h2>Indirect parametrization of optional implementations/imports<a class="headerlink" href="#indirect-parametrization-of-optional-implementations-imports" title="Permalink to this headline">¶</a></h2>
<p>If you want to compare the outcomes of several implementations of a given
API, you can write test functions that receive the already imported implementations
and get skipped in case the implementation is not importable/available. Let’s
say we have a “base” implementation and the other (possibly optimized ones)
need to provide similar results:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s">"session"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">basemod</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">pytest</span><span class="o">.</span><span class="n">importorskip</span><span class="p">(</span><span class="s">"base"</span><span class="p">)</span>
<span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s">"session"</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="p">[</span><span class="s">"opt1"</span><span class="p">,</span> <span class="s">"opt2"</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">optmod</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">pytest</span><span class="o">.</span><span class="n">importorskip</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">)</span>
</pre></div>
</div>
<p>And then a base implementation of a simple function:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of base.py</span>
<span class="k">def</span> <span class="nf">func1</span><span class="p">():</span>
<span class="k">return</span> <span class="mi">1</span>
</pre></div>
</div>
<p>And an optimized version:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of opt1.py</span>
<span class="k">def</span> <span class="nf">func1</span><span class="p">():</span>
<span class="k">return</span> <span class="mf">1.0001</span>
</pre></div>
</div>
<p>And finally a little test module:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_module.py</span>
<span class="k">def</span> <span class="nf">test_func1</span><span class="p">(</span><span class="n">basemod</span><span class="p">,</span> <span class="n">optmod</span><span class="p">):</span>
<span class="k">assert</span> <span class="nb">round</span><span class="p">(</span><span class="n">basemod</span><span class="o">.</span><span class="n">func1</span><span class="p">(),</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="nb">round</span><span class="p">(</span><span class="n">optmod</span><span class="o">.</span><span class="n">func1</span><span class="p">(),</span> <span class="mi">3</span><span class="p">)</span>
</pre></div>
</div>
<p>If you run this with reporting for skips enabled:</p>
<div class="highlight-python"><pre>$ py.test -rs test_module.py
=========================== test session starts ============================
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.7.0
rootdir: /tmp/doc-exec-169, inifile:
collected 2 items
test_module.py .s
========================= short test summary info ==========================
SKIP [1] /tmp/doc-exec-169/conftest.py:10: could not import 'opt2'
=================== 1 passed, 1 skipped in 0.01 seconds ====================</pre>
</div>
<p>You’ll see that we don’t have a <tt class="docutils literal"><span class="pre">opt2</span></tt> module and thus the second test run
of our <tt class="docutils literal"><span class="pre">test_func1</span></tt> was skipped. A few notes:</p>
<ul class="simple">
<li>the fixture functions in the <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file are “session-scoped” because we
don’t need to import more than once</li>
<li>if you have multiple test functions and a skipped import, you will see
the <tt class="docutils literal"><span class="pre">[1]</span></tt> count increasing in the report</li>
<li>you can put <a class="reference internal" href="../parametrize.html#pytest-mark-parametrize"><em>@pytest.mark.parametrize</em></a> style
parametrization on the test functions to parametrize input/output
values as well.</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../contents.html">
<img class="logo" src="../_static/pytest1.png" alt="Logo"/>
</a></p><h3><a href="../contents.html">Table Of Contents</a></h3>
<ul>
<li><a href="../index.html">Home</a></li>
<li><a href="../contents.html">Contents</a></li>
<li><a href="../getting-started.html">Install</a></li>
<li><a href="index.html">Examples</a></li>
<li><a href="../customize.html">Customize</a></li>
<li><a href="../contact.html">Contact</a></li>
<li><a href="../talks.html">Talks/Posts</a></li>
<li><a href="../changelog.html">Changelog</a></li>
</ul>
<hr>
<ul>
<li><a class="reference internal" href="#">Parametrizing tests</a><ul>
<li><a class="reference internal" href="#generating-parameters-combinations-depending-on-command-line">Generating parameters combinations, depending on command line</a></li>
<li><a class="reference internal" href="#different-options-for-test-ids">Different options for test IDs</a></li>
<li><a class="reference internal" href="#a-quick-port-of-testscenarios">A quick port of “testscenarios”</a></li>
<li><a class="reference internal" href="#deferring-the-setup-of-parametrized-resources">Deferring the setup of parametrized resources</a></li>
<li><a class="reference internal" href="#parametrizing-test-methods-through-per-class-configuration">Parametrizing test methods through per-class configuration</a></li>
<li><a class="reference internal" href="#indirect-parametrization-with-multiple-fixtures">Indirect parametrization with multiple fixtures</a></li>
<li><a class="reference internal" href="#indirect-parametrization-of-optional-implementations-imports">Indirect parametrization of optional implementations/imports</a></li>
</ul>
</li>
</ul>
<h3>Related Topics</h3>
<ul>
<li><a href="../contents.html">Documentation overview</a><ul>
<li><a href="index.html">Usages and Examples</a><ul>
<li>Previous: <a href="simple.html" title="previous chapter">Basic patterns and examples</a></li>
<li>Next: <a href="markers.html" title="next chapter">Working with custom markers</a></li>
</ul></li>
</ul></li>
</ul><h3>Useful Links</h3>
<ul>
<li><a href="../index.html">The pytest Website</a></li>
<li><a href="../contributing.html">Contribution Guide</a></li>
<li><a href="https://pypi.python.org/pypi/pytest">pytest @ PyPI</a></li>
<li><a href="https://bitbucket.org/pytest-dev/pytest/">pytest @ Bitbucket</a></li>
<li><a href="http://pytest.org/latest/plugins_index/index.html">3rd party plugins</a></li>
<li><a href="https://bitbucket.org/pytest-dev/pytest/issues?status=new&status=open">Issue Tracker</a></li>
<li><a href="http://pytest.org/latest/pytest.pdf">PDF Documentation</a>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
© Copyright 2014, holger krekel.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
</div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-7597274-13']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</body>
</html>