1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
<title>Advanced dependencies — Gentoo Python Guide documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css" />
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Resolving test suite problems" href="test.html" />
<link rel="prev" title="Common helper functions" href="helper.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="advanced-dependencies">
<h1>Advanced dependencies<a class="headerlink" href="#advanced-dependencies" title="Permalink to this heading">¶</a></h1>
<span class="target" id="index-0"></span><section id="requesting-use-flags-on-the-python-interpreter">
<span id="index-1"></span><h2>Requesting USE flags on the Python interpreter<a class="headerlink" href="#requesting-use-flags-on-the-python-interpreter" title="Permalink to this heading">¶</a></h2>
<p>While the majority of Python standard library modules are available
unconditionally, a few are controlled by USE flags. For example,
the <a class="reference external" href="https://docs.python.org/3.8/library/sqlite3.html">sqlite3</a> module requires <code class="docutils literal notranslate"><span class="pre">sqlite</span></code> flag to be enabled
on the interpreter. If a package requires this module, it needs
to enforce the matching flag via a USE dependency.</p>
<p>In order to create a USE dependency on the Python interpreter, set
<code class="docutils literal notranslate"><span class="pre">PYTHON_REQ_USE</span></code> before inheriting the eclass. This will cause
the eclass to generate appropriate dependency string in <code class="docutils literal notranslate"><span class="pre">PYTHON_DEPS</span></code>.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span> <span class="c1"># Copyright 1999-2020 Gentoo Authors</span>
<span class="c1"># Distributed under the terms of the GNU General Public License v2</span>
<span class="nv">EAPI</span><span class="o">=</span><span class="m">7</span>
<span class="nv">PYTHON_COMPAT</span><span class="o">=(</span> python3_6 <span class="o">)</span>
<span class="hll"> <span class="nv">PYTHON_REQ_USE</span><span class="o">=</span><span class="s2">"sqlite"</span>
</span> inherit python-r1 gnome2-utils meson xdg-utils
<span class="nv">DESCRIPTION</span><span class="o">=</span><span class="s2">"Modern music player for GNOME"</span>
<span class="nv">HOMEPAGE</span><span class="o">=</span><span class="s2">"https://wiki.gnome.org/Apps/Lollypop"</span>
<span class="nv">SRC_URI</span><span class="o">=</span><span class="s2">"https://adishatz.org/</span><span class="si">${</span><span class="nv">PN</span><span class="si">}</span><span class="s2">/</span><span class="si">${</span><span class="nv">P</span><span class="si">}</span><span class="s2">.tar.xz"</span>
<span class="nv">KEYWORDS</span><span class="o">=</span><span class="s2">"~amd64"</span>
<span class="nv">LICENSE</span><span class="o">=</span><span class="s2">"GPL-3"</span>
<span class="nv">SLOT</span><span class="o">=</span><span class="s2">"0"</span>
<span class="nv">REQUIRED_USE</span><span class="o">=</span><span class="si">${</span><span class="nv">PYTHON_REQUIRED_USE</span><span class="si">}</span>
<span class="nv">DEPEND</span><span class="o">=</span><span class="s2">"</span><span class="si">${</span><span class="nv">PYTHON_DEPS</span><span class="si">}</span><span class="s2"></span>
<span class="s2"> ..."</span>
</pre></div>
</div>
<p>Full USE dependency syntax is permitted. For example, you can make
the dependency conditional to a flag on the package:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span> <span class="c1"># Copyright 1999-2020 Gentoo Authors</span>
<span class="c1"># Distributed under the terms of the GNU General Public License v2</span>
<span class="nv">EAPI</span><span class="o">=</span><span class="m">6</span>
<span class="nv">PYTHON_COMPAT</span><span class="o">=(</span> python3_6 <span class="o">)</span>
<span class="hll"> <span class="nv">PYTHON_REQ_USE</span><span class="o">=</span><span class="s2">"sqlite?"</span>
</span> inherit distutils-r1
<span class="nv">DESCRIPTION</span><span class="o">=</span><span class="s2">"A lightweight password-manager with multiple database backends"</span>
<span class="nv">HOMEPAGE</span><span class="o">=</span><span class="s2">"https://pwman3.github.io/pwman3/"</span>
<span class="nv">SRC_URI</span><span class="o">=</span><span class="s2">"https://github.com/pwman3/pwman3/archive/v</span><span class="si">${</span><span class="nv">PV</span><span class="si">}</span><span class="s2">.tar.gz -> </span><span class="si">${</span><span class="nv">P</span><span class="si">}</span><span class="s2">.tar.gz"</span>
<span class="nv">LICENSE</span><span class="o">=</span><span class="s2">"GPL-3"</span>
<span class="nv">SLOT</span><span class="o">=</span><span class="s2">"0"</span>
<span class="nv">KEYWORDS</span><span class="o">=</span><span class="s2">"~amd64"</span>
<span class="hll"> <span class="nv">IUSE</span><span class="o">=</span><span class="s2">"mongodb mysql postgres +sqlite"</span>
</span></pre></div>
</div>
<p>Finally, there are cases when the problem cannot be fully solved using
a single USE dependency. Additional Python interpreter dependencies
with specific USE flags can be constructed using <code class="docutils literal notranslate"><span class="pre">python_gen_impl_dep</span></code>
helper then. For example, the following ebuild requires Python with
SQLite support when running tests:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span> <span class="c1"># Copyright 1999-2020 Gentoo Authors</span>
<span class="c1"># Distributed under the terms of the GNU General Public License v2</span>
<span class="nv">EAPI</span><span class="o">=</span><span class="m">7</span>
<span class="nv">PYTHON_COMPAT</span><span class="o">=(</span> python<span class="o">{</span>2_7,3_<span class="o">{</span><span class="m">6</span>,7,8<span class="o">}}</span> pypy3 <span class="o">)</span>
inherit distutils-r1
<span class="nv">DESCRIPTION</span><span class="o">=</span><span class="s2">"Let your Python tests travel through time"</span>
<span class="nv">HOMEPAGE</span><span class="o">=</span><span class="s2">"https://github.com/spulec/freezegun"</span>
<span class="nv">SRC_URI</span><span class="o">=</span><span class="s2">"mirror://pypi/</span><span class="si">${</span><span class="nv">PN</span><span class="p">:</span><span class="nv">0</span><span class="p">:</span><span class="nv">1</span><span class="si">}</span><span class="s2">/</span><span class="si">${</span><span class="nv">PN</span><span class="si">}</span><span class="s2">/</span><span class="si">${</span><span class="nv">P</span><span class="si">}</span><span class="s2">.tar.gz"</span>
<span class="nv">LICENSE</span><span class="o">=</span><span class="s2">"Apache-2.0"</span>
<span class="nv">SLOT</span><span class="o">=</span><span class="s2">"0"</span>
<span class="nv">KEYWORDS</span><span class="o">=</span><span class="s2">"~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos"</span>
<span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> >dev-python/python-dateutil-2.0[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> dev-python/six[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> "</span>
<span class="nv">DEPEND</span><span class="o">=</span><span class="s2">"</span><span class="si">${</span><span class="nv">RDEPEND</span><span class="si">}</span><span class="s2"></span>
<span class="s2"> dev-python/setuptools[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> test? (</span>
<span class="hll"><span class="s2"> </span><span class="k">$(</span>python_gen_impl_dep sqlite<span class="k">)</span><span class="s2"></span>
</span><span class="s2"> dev-python/mock[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> dev-python/nose[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> )</span>
<span class="s2"> "</span>
distutils_enable_tests pytest
</pre></div>
</div>
</section>
<section id="dependencies-conditional-to-python-version">
<span id="index-2"></span><h2>Dependencies conditional to Python version<a class="headerlink" href="#dependencies-conditional-to-python-version" title="Permalink to this heading">¶</a></h2>
<p>When packaging software for multiple Python versions, it is quite likely
that you’ll find yourself needing some packages only with some
of the versions, and not with others. This is the case with backports
and other compatibility packages. It also happens if some
of the optional dependencies do not support the full set
of implementations your package supports.</p>
<p>A dependency that applies only to a subset of <code class="docutils literal notranslate"><span class="pre">PYTHON_COMPAT</span></code> can
be created using <code class="docutils literal notranslate"><span class="pre">python_gen_cond_dep</span></code> function (the same as used
in <code class="docutils literal notranslate"><span class="pre">python-single-r1</span></code>). It takes a dependency string template,
followed by zero or more implementation arguments. The dependencies
are output for every matching implementation.</p>
<p>The dependency template should contain literal (usually escaped through
use of single quotes) <code class="docutils literal notranslate"><span class="pre">${PYTHON_USEDEP}</span></code> that will be substituted
with partial USE dependency by the eclass function (when using
<code class="docutils literal notranslate"><span class="pre">python-single-r1</span></code>, <code class="docutils literal notranslate"><span class="pre">${PYTHON_SINGLE_USEDEP}</span></code> is also permitted).</p>
<p>The implementation arguments can be:</p>
<ol class="arabic">
<li><p>Literal implementation names. For example, if a particular feature
is only available on a subset of Python implementations supported
by the package:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> cli? (</span>
<span class="s2"> </span><span class="k">$(</span>python_gen_cond_dep <span class="s1">'</span>
<span class="s1"> dev-python/black[${PYTHON_USEDEP}]</span>
<span class="s1"> dev-python/click[${PYTHON_USEDEP}]</span>
<span class="s1"> '</span> python3_<span class="o">{</span><span class="m">8</span>..10<span class="o">}</span><span class="k">)</span><span class="s2"></span>
<span class="s2"> )</span>
<span class="s2">"</span>
</pre></div>
</div>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">fnmatch(3)</span></code>-style wildcard against implementation names.
For example, CFFI is part of PyPy’s stdlib, so the explicit package
needs to be only installed for CPython:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> </span><span class="k">$(</span>python_gen_cond_dep <span class="s1">'</span>
<span class="s1"> dev-python/cffi[${PYTHON_USEDEP}]</span>
<span class="s1"> '</span> <span class="s1">'python*'</span><span class="k">)</span><span class="s2"></span>
<span class="s2">"</span>
</pre></div>
</div>
<p>Remember that the patterns need to be escaped to prevent filename
expansion from happening.</p>
</li>
<li><p>Python standard library versions that are expanded into appropriate
implementations by the eclass. For example, this makes it convenient
to depend on backports:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> </span><span class="k">$(</span>python_gen_cond_dep <span class="s1">'</span>
<span class="s1"> dev-python/backports-zoneinfo[${PYTHON_USEDEP}]</span>
<span class="s1"> '</span> <span class="m">3</span>.8<span class="k">)</span><span class="s2"></span>
<span class="s2">"</span>
</pre></div>
</div>
<p>The advantage of this form is that the dependencies automatically
adjust as we switch PyPy3 to a newer stdlib version.</p>
</li>
</ol>
<p>An important feature of <code class="docutils literal notranslate"><span class="pre">python_gen_cond_dep</span></code> is that it handles
removal of old implementations gracefully. When one of the listed
implementations is no longer supported, it silently ignores it. This
makes it possible to remove old implementations without having to update
all dependency strings immediately.</p>
<p>For example, in the following example the dependency became empty when
Python 3.7 was removed:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> </span><span class="k">$(</span>python_gen_cond_dep <span class="s1">'</span>
<span class="s1"> dev-python/importlib_metadata[${PYTHON_USEDEP}]</span>
<span class="s1"> '</span> python3_7<span class="k">)</span><span class="s2">"</span>
</pre></div>
</div>
</section>
<section id="dependencies-on-cffi-and-greenlet">
<span id="index-3"></span><h2>Dependencies on CFFI and greenlet<a class="headerlink" href="#dependencies-on-cffi-and-greenlet" title="Permalink to this heading">¶</a></h2>
<p>The PyPy distribution includes special versions of the <a class="reference external" href="https://pypi.org/project/cffi/">cffi</a>
and <a class="reference external" href="https://pypi.org/project/greenlet/">greenlet</a> packages. For this reason, packages using CFFI
and/or greenlet and supporting PyPy3 need to make the explicit
dependencies conditional to CPython:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> </span><span class="k">$(</span>python_gen_cond_dep <span class="s1">'</span>
<span class="s1"> >=dev-python/cffi-1.1.0:=[${PYTHON_USEDEP}]</span>
<span class="s1"> '</span> <span class="s1">'python*'</span><span class="k">)</span><span class="s2"></span>
<span class="s2">"</span>
</pre></div>
</div>
</section>
<section id="optional-test-suite-dependencies-on-rust-packages">
<span id="index-4"></span><h2>Optional test suite dependencies on Rust packages<a class="headerlink" href="#optional-test-suite-dependencies-on-rust-packages" title="Permalink to this heading">¶</a></h2>
<p>When the test suite of a high-profile Python package starts depending
on Python-Rust packages, it may not be feasible to mask the package
on all architectures that are not supported by Rust. In this case,
it is preferable to skip the tests that require the particular
dependencies.</p>
<p>If upstream does not handle missing dependencies gracefully and refuses
to merge a patch to do so, it is possible to conditionally deselect
tests from the ebuild based on whether the particular dependencies are
installed:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span> <span class="o">{</span>
<span class="nb">local</span> <span class="nv">EPYTEST_DESELECT</span><span class="o">=()</span>
<span class="k">if</span> ! has_version <span class="s2">"dev-python/trustme[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]"</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">EPYTEST_DESELECT</span><span class="o">+=(</span>
tests/test_requests.py::TestRequests::test_https_warnings
<span class="o">)</span>
<span class="k">fi</span>
epytest
<span class="o">}</span>
</pre></div>
</div>
<p>Note that if the modules are imported in outer scope, ignoring the whole
test file may be necessary. If a file contains both tests requiring
the dependency and other useful tests, sometimes it is possible
to convince upstream to move imports into specific test functions,
in order to make it possible to deselect specific tests.</p>
<p>If the tests requiring these packages are not very important, it is
acceptable to skip the dependency and assume they would be run
if the package was installed independently. However, if they are
significant (e.g. tests for TLS support), the <code class="docutils literal notranslate"><span class="pre">test-rust</span></code> flag
can be used to pull them in, e.g.:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">IUSE</span><span class="o">=</span><span class="s2">"test test-rust"</span>
<span class="nv">RESTRICT</span><span class="o">=</span><span class="s2">"!test? ( test )"</span>
<span class="nv">BDEPEND</span><span class="o">=</span><span class="s2">"</span>
<span class="s2"> test? (</span>
<span class="s2"> test-rust? (</span>
<span class="s2"> dev-python/trustme[</span><span class="si">${</span><span class="nv">PYTHON_USEDEP</span><span class="si">}</span><span class="s2">]</span>
<span class="s2"> )</span>
<span class="s2"> )</span>
<span class="s2">"</span>
</pre></div>
</div>
<p>This flag is masked on profiles for architectures that do not provide
a Rust toolchain, and forced on all the remaining profiles. This
ensures that the respective tests are run whenever it is possible
to run them.</p>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">Gentoo Python Guide</a></h1>
<h3>Navigation</h3>
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="preface.html">Preface</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter.html">Python interpreters</a></li>
<li class="toctree-l1"><a class="reference internal" href="eclass.html">Choosing between Python eclasses</a></li>
<li class="toctree-l1"><a class="reference internal" href="basic.html">Common basics</a></li>
<li class="toctree-l1"><a class="reference internal" href="any.html">python-any-r1 — build-time dependency</a></li>
<li class="toctree-l1"><a class="reference internal" href="single.html">python-single-r1 — single-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="multi.html">python-r1 — multi-impl packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils.html">distutils-r1 — standard Python build systems</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils-legacy.html">distutils-r1 legacy concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="helper.html">Common helper functions</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Advanced dependencies</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#requesting-use-flags-on-the-python-interpreter">Requesting USE flags on the Python interpreter</a></li>
<li class="toctree-l2"><a class="reference internal" href="#dependencies-conditional-to-python-version">Dependencies conditional to Python version</a></li>
<li class="toctree-l2"><a class="reference internal" href="#dependencies-on-cffi-and-greenlet">Dependencies on CFFI and greenlet</a></li>
<li class="toctree-l2"><a class="reference internal" href="#optional-test-suite-dependencies-on-rust-packages">Optional test suite dependencies on Rust packages</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="test.html">Resolving test suite problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="pytest.html">pytest recipes</a></li>
<li class="toctree-l1"><a class="reference internal" href="concept.html">Advanced concepts</a></li>
<li class="toctree-l1"><a class="reference internal" href="expert-multi.html">Expert python-r1 usage</a></li>
<li class="toctree-l1"><a class="reference internal" href="buildsys.html">Integration with build systems written in Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="porting.html">Porting tips</a></li>
<li class="toctree-l1"><a class="reference internal" href="migration.html">Migration guides</a></li>
<li class="toctree-l1"><a class="reference internal" href="qawarn.html">QA checks and warnings</a></li>
<li class="toctree-l1"><a class="reference internal" href="package-maintenance.html">Python package maintenance</a></li>
<li class="toctree-l1"><a class="reference internal" href="interpreter-maintenance.html">Maintenance of Python implementations</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="helper.html" title="previous chapter">Common helper functions</a></li>
<li>Next: <a href="test.html" title="next chapter">Resolving test suite problems</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2020, Michał Górny, license: CC BY 4.0.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 5.3.0</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="_sources/depend.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
|
GitWeb