python-r1 — multi-impl packages

The python-r1 eclass is used to install multi-impl packages. It is considered an expert eclass — when possible, you should prefer using python-single-r1 instead. For packages using distutils or a similar Python build system, distutils-r1 eclass should be used instead.

Eclass reference: python-r1.eclass(5)

Manual install

The simplest case of multi-impl package is a package without a specific build system. The modules need to be installed manually here, and python_foreach_impl function is used to repeat the install step for all enabled implementations.

For simple use cases, the install command can be inlined:

 # Copyright 1999-2024 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=8

 PYTHON_COMPAT=( python3_{10..13} )

 inherit python-r1

 DESCRIPTION="Enhanced df with colors"
 HOMEPAGE="http://kassiopeia.juls.savba.sk/~garabik/software/pydf/"
 SRC_URI="http://kassiopeia.juls.savba.sk/~garabik/software/pydf/${PN}_${PV}.tar.gz"

 LICENSE="public-domain"
 SLOT="0"
 KEYWORDS="amd64 arm ~arm64 ppc ppc64 ~riscv x86 ~amd64-linux ~x86-linux"

 REQUIRED_USE="${PYTHON_REQUIRED_USE}"
 RDEPEND="${PYTHON_DEPS}"
 BDEPEND="${RDEPEND}"

 src_prepare() {
      default
      sed -i -e "s#/etc/pydfrc#${EPREFIX}/etc/pydfrc#" "${PN}" || die
 }

 src_install() {
      python_foreach_impl python_doscript "${PN}"
      insinto /etc
      doins "${PN}rc"
      doman "${PN}.1"
      einstalldocs
 }

While python_foreach_impl can be repeated multiple times, it is generally better to declare a function when multiple install commands need to be executed:

 # Copyright 1999-2024 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=8

 PYTHON_COMPAT=( python3_{10..13} )

 inherit python-r1

 DESCRIPTION="Check for mapped libs and open files that are marked as deleted"
 HOMEPAGE="https://github.com/klausman/lib_users"
 SRC_URI="
     https://github.com/klausman/${PN}/archive/v${PV}.tar.gz
         -> ${P}.tar.gz
 "

 LICENSE="GPL-2"
 SLOT="0"
 KEYWORDS="~alpha amd64 ~arm arm64 ~hppa ppc ppc64 ~sparc x86"
 IUSE="test"
 RESTRICT="!test? ( test )"

 REQUIRED_USE="${PYTHON_REQUIRED_USE}"

 DEPEND="${PYTHON_DEPS}
     test? (
         dev-python/nose2[${PYTHON_USEDEP}]
     )"
 RDEPEND="${PYTHON_DEPS}"

 src_test() {
     python_foreach_impl nose2 --verbosity=2
 }

 python_install() {
     python_newscript lib_users.py lib_users
     python_newscript fd_users.py fd_users
     # lib_users_util/ contains a test script we don't want, so do things by hand
     python_moduleinto lib_users_util
     python_domodule lib_users_util/common.py
     python_domodule lib_users_util/__init__.py
 }

 src_install() {
     python_foreach_impl python_install
     dodoc README.md TODO
 }

Dependencies

When depending on other Python packages, USE dependencies need to be declared in order to ensure that the dependencies would be built against all the Python implementations enabled for the package. This is easily done via appending the USE dependency string from ${PYTHON_USEDEP} to the dependencies:

RDEPEND="${PYTHON_DEPS}
    sys-apps/portage[${PYTHON_USEDEP}]
"
DEPEND="${RDEPEND}"

Pure Python autotools package

Another typical case for this eclass is to handle a pure Python package with a non-standard build system. In this case, it is generally necessary to call phase functions via python_foreach_impl. Whenever possible, out-of-source builds are recommended (i.e. installing to separate directories from a single source directory).

 # Copyright 1999-2023 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=7

 PYTHON_COMPAT=( python3_{10..13} )
 inherit autotools python-r1

 DESCRIPTION="Python wrapper for libcangjie"
 HOMEPAGE="http://cangjians.github.io/"
 SRC_URI="https://github.com/Cangjians/py${PN}/releases/download/v${PV}/${P#py}.tar.xz"

 LICENSE="LGPL-3+"
 SLOT="0"
 KEYWORDS="amd64 x86"

 REQUIRED_USE="${PYTHON_REQUIRED_USE}"

 RDEPEND="${PYTHON_DEPS}
     app-i18n/libcangjie"
 DEPEND="${RDEPEND}"
 BDEPEND="dev-python/cython[${PYTHON_USEDEP}]
     virtual/pkgconfig"

 src_prepare() {
     default
     eautoreconf
 }

 src_configure() {
     python_configure() {
         ECONF_SOURCE="${S}" econf
     }
     python_foreach_impl run_in_build_dir python_configure
 }

 src_compile() {
     python_foreach_impl run_in_build_dir default
 }

 src_test() {
     python_foreach_impl run_in_build_dir default
 }

 src_install() {
     python_install() {
         default
         python_optimize
     }
     python_foreach_impl run_in_build_dir python_install
     einstalldocs

     find "${D}" -name '*.la' -delete || die
 }

Note the use of run_in_build_dir helper from multibuild eclass (direct inherit is unnecessary here, as it is considered implicit part of python-r1 API). It changes the directory to BUILD_DIR (which is set by python_foreach_impl to a unique directory for each implementation) and runs the specified command there. In this case, the ebuild performs autotools out-of-source build in a dedicated directory for every interpreter enabled.

Also note that the in-build-dir call to default does not install documentation from source directory, hence the additional einstalldocs call. Libtool-based packages install .la files that are unnecessary for Python extensions, hence they are removed afterwards.

If the package in question does not support out-of-source builds (e.g. due to a buggy build system), python_copy_sources function can be used to duplicate the package’s sources in build directories for each implementation. The same ebuild easily can be changed to do that:

 # Copyright 1999-2023 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=7

 PYTHON_COMPAT=( python3_{10..13} )
 inherit autotools python-r1

 DESCRIPTION="Python wrapper for libcangjie"
 HOMEPAGE="http://cangjians.github.io/"
 SRC_URI="https://github.com/Cangjians/py${PN}/releases/download/v${PV}/${P#py}.tar.xz"

 LICENSE="LGPL-3+"
 SLOT="0"
 KEYWORDS="amd64 x86"

 REQUIRED_USE="${PYTHON_REQUIRED_USE}"

 RDEPEND="${PYTHON_DEPS}
     app-i18n/libcangjie"
 DEPEND="${RDEPEND}"
 BDEPEND="dev-python/cython[${PYTHON_USEDEP}]
     virtual/pkgconfig"

 src_prepare() {
     default
     eautoreconf
     python_copy_sources
 }

 src_configure() {
     python_configure() {
         ECONF_SOURCE="${S}" econf
     }
     python_foreach_impl run_in_build_dir python_configure
 }

 src_compile() {
     python_foreach_impl run_in_build_dir default
 }

 src_test() {
     python_foreach_impl run_in_build_dir default
 }

 src_install() {
     python_install() {
         default
         python_optimize
     }
     python_foreach_impl run_in_build_dir python_install
     einstalldocs

     find "${D}" -name '*.la' -delete || die
 }

Note that besides adding python_copy_sources call, ECONF_SOURCE has been removed in order to disable out-of-source builds.

Conditional Python use

When the package installs Python components conditionally to a USE flag, the respective USE conditional needs to be consistently used in metadata variables and in python_foreach_impl calls.

 # Copyright 1999-2020 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=6
 PYTHON_COMPAT=( python2_7 )

 inherit gnome2 python-r1

 DESCRIPTION="Canvas widget for GTK+ using the cairo 2D library for drawing"
 HOMEPAGE="https://wiki.gnome.org/GooCanvas"

 LICENSE="LGPL-2"
 SLOT="2.0"
 KEYWORDS="~alpha amd64 ia64 ppc ppc64 sparc x86"
 IUSE="python"
 REQUIRED_USE="python? ( ${PYTHON_REQUIRED_USE} )"

 # python only enables python specific binding override
 RDEPEND="
     python? (
         ${PYTHON_DEPS}
         >=dev-python/pygobject-2.90.4:3[${PYTHON_USEDEP}] )
 "
 DEPEND="${RDEPEND}"

 src_prepare() {
     # Python bindings are built/installed manually.
     sed -e "/SUBDIRS = python/d" -i bindings/Makefile.am \
         bindings/Makefile.in || die

     gnome2_src_prepare
 }

 src_configure() {
     gnome2_src_configure \
         --disable-python
 }

 src_install() {
     gnome2_src_install

     if use python; then
         sub_install() {
             python_moduleinto $(python -c "import gi;print gi._overridesdir")
             python_domodule bindings/python/GooCanvas.py
         }
         python_foreach_impl sub_install
     fi
 }

Note that in many cases, you will end up having to disable upstream rules for installing Python files as they are suitable only for single-impl installs.

Additional build-time Python use

Some packages additionally require Python at build time, independently of Python components installed (i.e. outside python_foreach_impl). The eclass provides extensive API for this purpose but for now we’ll focus on the simplest case where the global code does not have any dependencies or they are a subset of dependencies declared already.

In this case, it is sufficient to call python_setup before the routine requiring Python. It will choose the most preferred of enabled implementations, and set the global environment for it. Note that it is entirely normal that the same environment will be set inside python_foreach_impl afterwards.

 # Copyright 1999-2024 Gentoo Authors
 # Distributed under the terms of the GNU General Public License v2

 EAPI=8

 PYTHON_COMPAT=( python3_{10..13} )
 PYTHON_REQ_USE="ncurses,readline"

 inherit python-r1

 DESCRIPTION="QEMU + Kernel-based Virtual Machine userland tools"
 HOMEPAGE="http://www.qemu.org http://www.linux-kvm.org"
 SRC_URI="http://wiki.qemu-project.org/download/${P}.tar.xz"

 LICENSE="GPL-2 LGPL-2 BSD-2"
 SLOT="0"
 KEYWORDS="amd64 ~arm64 ~ppc ~ppc64 x86"
 IUSE="python"
 REQUIRED_USE="${PYTHON_REQUIRED_USE}"

 BDEPEND="${PYTHON_DEPS}"
 RDEPEND="python? ( ${PYTHON_DEPS} )"

 src_configure() {
     python_setup
     ./configure || die
 }

 qemu_python_install() {
     python_domodule "${S}/python/qemu"

     python_doscript "${S}/scripts/kvm/vmxcap"
     python_doscript "${S}/scripts/qmp/qmp-shell"
     python_doscript "${S}/scripts/qmp/qemu-ga-client"
 }

 src_install() {
     default
     if use python; then
         python_foreach_impl qemu_python_install
     fi
 }

Note that the parts affecting installation of runtime components (RDEPEND, python_foreach_impl) are made conditional to the USE flag, while parts affecting build time (REQUIRED_USE, BDEPEND, python_setup) are unconditional.