Update Guide to 8e5a339

Signed-off-by: Michał Górny <mgorny@gentoo.org>

1 files changed, 43 insertions, 9 deletions

diff --git a/guide/test.html b/guide/test.html
index 08b95de..0090c43 100644
--- a/guide/test.html
+++ b/guide/test.html
@@ -358,6 +358,14 @@ in the staging area or build directory.  In fact, this is often
 necessary to prevent import collisions — e.g. when modules would
 be loaded first from the staging area due to explicit imports
 then again from the current directory due to test discovery.</p>
+<div class="admonition warning">
+<p class="admonition-title">Warning</p>
+<p>Not all packages fail explicitly like that.  In particular,
+if the C extensions are optional, the package may either skip tests
+requiring them or silently fall back to testing the pure Python
+variant.  Special caution is advised when packaging software using
+C extensions with top-level source layout.</p>
+</div>
 <p>Unfortunately, this does not work correctly if C extensions are built
 as part of these packages.  Since the package imported relatively
 does not include the necessary extensions, the imports fail, e.g.:</p>
@@ -387,23 +395,49 @@ does not include the necessary extensions, the imports fail, e.g.:</p>
 <span class="x">E   ImportError: cannot import name &#39;login&#39; from &#39;systemd&#39; (/tmp/portage/dev-python/python-systemd-234-r2/work/python-systemd-234/systemd/__init__.py)</span>
 </pre></div>
 </div>
-<p>The preferred solution is to change the working directory before running
-the tests.  If tests are installed as a part of the package, they can
-be discovered through package search e.g. using pytest:</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Historically, the primary recommendation was to change directory
+prior to running the test suite.  However, this was change since
+the <code class="docutils literal notranslate"><span class="pre">rm</span></code> approach is easier in PEP 517 mode, and less likely
+to trigger other issues (e.g. pytest missing its configuration,
+tests relying on relative paths).</p>
+</div>
+<p>The modern preference is to remove the package directory prior to
+running the test suite.  In PEP 517 mode, the build system is only run
+as part of <code class="docutils literal notranslate"><span class="pre">src_compile</span></code>, and therefore the original sources are not
+needed afterwards.  For example, when the tests are in a separate
+directory:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
+<span class="w">    </span>rm<span class="w"> </span>-rf<span class="w"> </span>frozendict<span class="w"> </span><span class="o">||</span><span class="w"> </span>die
+<span class="w">    </span>epytest
+<span class="o">}</span>
+</pre></div>
+</div>
+<p>When the tests are installed as a part of the installed package,
+<code class="docutils literal notranslate"><span class="pre">--pyargs</span></code> option can be used to find them via package lookup:</p>
 <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
-<span class="w">    </span><span class="nb">cd</span><span class="w"> </span><span class="s2">&quot;</span><span class="si">${</span><span class="nv">T</span><span class="si">}</span><span class="s2">&quot;</span><span class="w"> </span><span class="o">||</span><span class="w"> </span>die
-<span class="w">    </span>epytest<span class="w"> </span>--pyargs<span class="w"> </span>systemd
+<span class="w">    </span>rm<span class="w"> </span>-rf<span class="w"> </span>numpy<span class="w"> </span><span class="o">||</span><span class="w"> </span>die
+<span class="w">    </span>epytest<span class="w"> </span>--pyargs<span class="w"> </span>numpy
 <span class="o">}</span>
 </pre></div>
 </div>
-<p>If the tests are in a separate directory, an absolute path can be used
-instead:</p>
+<p>The other possible solution is to change the working directory prior
+to running the test suite, either into an arbitrary directory that
+does not feature the collision, or into the install directory.
+The latter could possible be necessary if the tests are installed
+as part of the package, and assume paths relative to the source
+directory:</p>
 <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span>
-<span class="w">    </span><span class="nb">cd</span><span class="w"> </span><span class="s2">&quot;</span><span class="si">${</span><span class="nv">T</span><span class="si">}</span><span class="s2">&quot;</span><span class="w"> </span><span class="o">||</span><span class="w"> </span>die
-<span class="w">    </span>epytest<span class="w"> </span><span class="s2">&quot;</span><span class="si">${</span><span class="nv">S</span><span class="si">}</span><span class="s2">&quot;</span>/test
+<span class="w">    </span><span class="nb">cd</span><span class="w"> </span><span class="s2">&quot;</span><span class="si">${</span><span class="nv">BUILD_DIR</span><span class="si">}</span><span class="s2">/install</span><span class="k">$(</span>python_get_sitedir<span class="k">)</span><span class="s2">&quot;</span><span class="w"> </span><span class="o">||</span><span class="w"> </span>die
+<span class="w">    </span>epytest
 <span class="o">}</span>
 </pre></div>
 </div>
+<p>However, please note that changing the working directory leads to pytest
+missing its configuration (<code class="docutils literal notranslate"><span class="pre">pytest.ini</span></code>, <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>,
+<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>) which in turn could lead to warnings about missing
+marks or misleading test suite problems.</p>
 </section>
 <section id="checklist-for-dealing-with-test-failures">
 <h2>Checklist for dealing with test failures<a class="headerlink" href="#checklist-for-dealing-with-test-failures" title="Permalink to this heading">¶</a></h2>