diff options
| author |   André Erdmann <dywi@mailerd.de> | 2012-08-16 18:28:19 +0200 |
|---|---|---|
| committer | André Erdmann <dywi@mailerd.de> | 2012-08-16 18:28:19 +0200 |
| commit | 961ebb5758d419464398faaa33ebebfa9cabee27 (patch) | |
| tree | dcbc41b407692c7c40b377604f18d334d3561fed /doc | |
| parent | doc: Introduction, Dependency Resolution (diff) | |
| download | R_overlay-961ebb5758d419464398faaa33ebebfa9cabee27.tar.gz R_overlay-961ebb5758d419464398faaa33ebebfa9cabee27.tar.bz2 R_overlay-961ebb5758d419464398faaa33ebebfa9cabee27.zip | |
doc: html version
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/html/usage.html | 970 |
1 files changed, 585 insertions, 385 deletions
diff --git a/doc/html/usage.html b/doc/html/usage.html index bc9cb85..8326ec4 100644 --- a/doc/html/usage.html +++ b/doc/html/usage.html @@ -314,6 +314,10 @@ ul.auto-toc { </style> </head> <body> +<div class="header"> +Automatically Generated Overlay of R packages - Manual (Aug 16 2012) +<hr class="header"/> +</div> <div class="document"> @@ -329,70 +333,108 @@ ul.auto-toc { </ul> </li> <li><a class="reference internal" href="#running-roverlay" id="id9">3 Running Roverlay</a><ul class="auto-toc"> -<li><a class="reference internal" href="#required-configuration-steps" id="id10">3.1 Required configuration steps</a></li> -<li><a class="reference internal" href="#running-it" id="id11">3.2 Running it</a></li> -<li><a class="reference internal" href="#providing-a-package-mirror" id="id12">3.3 Providing a package mirror</a></li> +<li><a class="reference internal" href="#required-configuration-steps" id="id10">3.1 Required configuration steps</a><ul class="auto-toc"> +<li><a class="reference internal" href="#extended-configuration-where-to-go-from-here" id="id11">3.1.1 Extended Configuration / Where to go from here?</a></li> +</ul> +</li> +<li><a class="reference internal" href="#running-it" id="id12">3.2 Running it</a></li> +<li><a class="reference internal" href="#providing-a-package-mirror" id="id13">3.3 Providing a package mirror</a></li> </ul> </li> -<li><a class="reference internal" href="#implementation-overview" id="id13">4 Implementation Overview</a><ul class="auto-toc"> -<li><a class="reference internal" href="#repositories-getting-packages" id="id14">4.1 Repositories / Getting Packages</a><ul class="auto-toc"> -<li><a class="reference internal" href="#a-word-about-repo-config-files" id="id15">4.1.1 A word about repo config files</a></li> -<li><a class="reference internal" href="#rsync-repos" id="id16">4.1.2 Rsync repos</a></li> -<li><a class="reference internal" href="#getting-packages-from-a-repository-that-supports-http-only" id="id17">4.1.3 Getting packages from a repository that supports http only</a></li> -<li><a class="reference internal" href="#getting-packages-from-several-remotes-using-http-and-a-package-list" id="id18">4.1.4 Getting packages from several remotes using http and a package list</a></li> -<li><a class="reference internal" href="#using-local-directories" id="id19">4.1.5 Using local directories</a></li> +<li><a class="reference internal" href="#basic-implementation-overview" id="id14">4 Basic Implementation Overview</a><ul class="auto-toc"> +<li><a class="reference internal" href="#how-roverlay-works" id="id15">4.1 How <em>roverlay</em> works</a></li> +<li><a class="reference internal" href="#expected-overlay-result-structure-of-the-generated-overlay" id="id16">4.2 Expected Overlay Result / Structure of the generated overlay</a><ul class="auto-toc"> +<li><a class="reference internal" href="#expected-ebuild-result" id="id17">4.2.1 Expected Ebuild Result</a></li> +<li><a class="reference internal" href="#expected-metadata-xml-result" id="id18">4.2.2 Expected <em>metadata.xml</em> Result</a></li> </ul> </li> -<li><a class="reference internal" href="#dependency-rules-resolving-dependencies" id="id20">4.2 Dependency Rules / Resolving Dependencies</a><ul class="auto-toc"> -<li><a class="reference internal" href="#dependency-types" id="id21">4.2.1 Dependency types</a></li> -<li><a class="reference internal" href="#simple-dependency-rules" id="id22">4.2.2 Simple Dependency Rules</a><ul class="auto-toc"> -<li><a class="reference internal" href="#rule-variants" id="id23">4.2.2.1 Rule Variants</a></li> -<li><a class="reference internal" href="#rule-types" id="id24">4.2.2.2 Rule types</a></li> -<li><a class="reference internal" href="#rule-file-examples" id="id25">4.2.2.3 Rule File Examples</a></li> -<li><a class="reference internal" href="#rule-file-syntax" id="id26">4.2.2.4 Rule File Syntax</a></li> </ul> </li> +<li><a class="reference internal" href="#repositories-getting-packages" id="id19">5 Repositories / Getting Packages</a><ul class="auto-toc"> +<li><a class="reference internal" href="#a-word-about-repo-config-files" id="id20">5.1 A word about repo config files</a></li> +<li><a class="reference internal" href="#rsync-repos" id="id21">5.2 Rsync repos</a></li> +<li><a class="reference internal" href="#getting-packages-from-a-repository-that-supports-http-only" id="id22">5.3 Getting packages from a repository that supports http only</a></li> +<li><a class="reference internal" href="#getting-packages-from-several-remotes-using-http-and-a-package-list" id="id23">5.4 Getting packages from several remotes using http and a package list</a></li> +<li><a class="reference internal" href="#using-local-directories" id="id24">5.5 Using local directories</a></li> </ul> </li> -<li><a class="reference internal" href="#expected-overlay-result-structure-of-the-generated-overlay" id="id27">4.3 Expected Overlay Result / Structure of the generated overlay</a></li> +<li><a class="reference internal" href="#dependency-rules" id="id25">6 Dependency Rules</a><ul class="auto-toc"> +<li><a class="reference internal" href="#simple-dependency-rules" id="id26">6.1 Simple Dependency Rules</a><ul class="auto-toc"> +<li><a class="reference internal" href="#rule-variants" id="id27">6.1.1 Rule Variants</a></li> +<li><a class="reference internal" href="#rule-types" id="id28">6.1.2 Rule types</a></li> +<li><a class="reference internal" href="#rule-file-examples" id="id29">6.1.3 Rule File Examples</a></li> +<li><a class="reference internal" href="#rule-file-syntax" id="id30">6.1.4 Rule File Syntax</a></li> </ul> </li> -<li><a class="reference internal" href="#depres-console" id="id28">5 DepRes Console</a></li> -<li><a class="reference internal" href="#configuration-reference" id="id29">6 Configuration Reference</a><ul class="auto-toc"> -<li><a class="reference internal" href="#dependency-rules" id="id30">6.1 Dependency Rules</a></li> -<li><a class="reference internal" href="#repositories" id="id31">6.2 Repositories</a></li> -<li><a class="reference internal" href="#main-config" id="id32">6.3 Main Config</a><ul class="auto-toc"> -<li><a class="reference internal" href="#misc-options" id="id33">6.3.1 misc options</a></li> -<li><a class="reference internal" href="#overlay-options" id="id34">6.3.2 overlay options</a></li> -<li><a class="reference internal" href="#other-config-files" id="id35">6.3.3 other config files</a></li> -<li><a class="reference internal" href="#logging" id="id36">6.3.4 logging</a><ul class="auto-toc"> -<li><a class="reference internal" href="#console-logging" id="id37">6.3.4.1 console logging</a></li> -<li><a class="reference internal" href="#file-logging" id="id38">6.3.4.2 file logging</a></li> </ul> </li> -<li><a class="reference internal" href="#options-for-debugging-manual-dependency-rule-creation-and-testing" id="id39">6.3.5 options for debugging, manual dependency rule creation and testing</a></li> +<li><a class="reference internal" href="#configuration-reference" id="id31">7 Configuration Reference</a><ul class="auto-toc"> +<li><a class="reference internal" href="#misc-options" id="id32">7.1 misc options</a></li> +<li><a class="reference internal" href="#overlay-options" id="id33">7.2 overlay options</a></li> +<li><a class="reference internal" href="#other-config-files" id="id34">7.3 other config files</a></li> +<li><a class="reference internal" href="#logging" id="id35">7.4 logging</a><ul class="auto-toc"> +<li><a class="reference internal" href="#console-logging" id="id36">7.4.1 console logging</a></li> +<li><a class="reference internal" href="#file-logging" id="id37">7.4.2 file logging</a></li> </ul> </li> -<li><a class="reference internal" href="#id2" id="id40">6.4 Field Definition</a></li> +<li><a class="reference internal" href="#options-for-debugging-manual-dependency-rule-creation-and-testing" id="id38">7.5 options for debugging, manual dependency rule creation and testing</a></li> </ul> </li> -<li><a class="reference internal" href="#appendix" id="id41">7 Appendix</a><ul class="auto-toc"> -<li><a class="reference internal" href="#default-field-definition-file" id="id42">7.1 Default Field Definition File</a></li> +<li><a class="reference internal" href="#field-definition-config" id="id39">8 Field Definition Config</a><ul class="auto-toc"> +<li><a class="reference internal" href="#example-the-default-field-definition-file" id="id40">8.1 Example: The default field definition file</a></li> </ul> </li> +<li><a class="reference internal" href="#dependency-resolution-console" id="id41">9 Dependency Resolution Console</a></li> +<li><a class="reference internal" href="#implementation-overview" id="id42">10 Implementation Overview</a></li> </ul> </div> <div class="section" id="introduction"> -<h1><a class="toc-backref" href="#id3">1 Introduction</a></h1> -<p><em>roverlay</em> is -Automatically Generated Overlay of R packages;; -GSoC Project;; -<>;;</p> +<h1><a class="toc-backref" href="#contents">1 Introduction</a></h1> +<p><em>roverlay</em> is an application that aims to provide integration of R packages +in Gentoo by creating a portage overlay for them. +Naturally, this also requires proper dependency resolution, especially on the +system level which cannot be done by <em>install.packages()</em> in R.</p> +<p>The project associated with <em>roverlay</em> is called +<em>Automatically generated overlay of R packages</em>, the initial work has been +done during the <em>Google Summer of Code 2012</em>.</p> +<p>At its current state, <em>roverlay</em> is capable of creating a complete overlay +with metadata and Manifest files by reading R packages. +It's also able to work incrementally, i.e. update an existing <em>R Overlay</em>. +Most aspects of overlay creation are configurable with text files.</p> +<p><em>roverlay</em> is written in python. There's no homepage available, only a +<a class="reference external" href="http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=summary">git repository</a> that contains the source code.</p> +<p>This document is targeted at</p> +<blockquote> +<ul> +<li><p class="first">overlay maintainers who <strong>use roverlay</strong> to create the R Overlay</p> +<p>The most relevant chapters are <a class="reference internal" href="#installation">Installation</a> (2) and +<a class="reference internal" href="#running-roverlay">Running Roverlay</a> (3). Additionally, have a look at +<a class="reference internal" href="#basic-implementation-overview">Basic Implementation Overview</a> (4) if you want to know what <em>roverlay</em> +does and what to expect from the generated overlay.</p> +</li> +<li><p class="first"><em>roverlay</em> maintainers who <strong>control and test overlay creation</strong>, +e.g. configure which R packages will be part of the generated overlay</p> +<p>Depending on what you want to configure, chapters 5-8 are relevant, +namely <a class="reference internal" href="#repositories-getting-packages">Repositories / Getting Packages</a>, <a class="reference internal" href="#dependency-rules">Dependency Rules</a>, +<a class="reference internal" href="#configuration-reference">Configuration Reference</a> and <a class="reference internal" href="#field-definition-config">Field Definition Config</a>.</p> +<p>There's another chapter that is only interesting for testing, the +<a class="reference internal" href="#dependency-resolution-console">Dependency Resolution Console</a> (9), which can be used to interactively +test dependency rules.</p> +</li> +<li><p class="first"><em>roverlay</em> code maintainers who want to know <strong>how roverlay works</strong> for +code improvements etc.</p> +<p>The most important chapter is <a class="reference internal" href="#implementation-overview">Implementation Overview</a> (10) which has +references to other chapters (4-8) where required.</p> +</li> +</ul> +</blockquote> +<p>It's expected that you already know about <em>R packages</em> (basically a tarball) +and some portage basics, e.g. <em>Depend Atoms</em> and what an overlay is.</p> </div> <div class="section" id="installation"> -<h1><a class="toc-backref" href="#id4">2 Installation</a></h1> +<h1><a class="toc-backref" href="#contents">2 Installation</a></h1> <div class="section" id="prerequisites"> -<h2><a class="toc-backref" href="#id5">2.1 Prerequisites</a></h2> +<h2><a class="toc-backref" href="#contents">2.1 Prerequisites</a></h2> <ul> <li><p class="first">python >= 2.7 (tested with python 2.7 and 3.2)</p> </li> @@ -407,17 +449,18 @@ GSoC Project;; package downloads during Manifest creation (optional)</li> </ul> </li> -<li><p class="first">for generating documentation files: python docutils > 0.8.1</p> +<li><p class="first">for generating documentation files: python docutils >= 0.9</p> </li> -<li><p class="first">hardware requirements (when the default configuration):</p> +<li><p class="first">hardware requirements (when using the default configuration):</p> <blockquote> <dl class="docutils"> <dt>disk</dt> <dd><ul class="first last simple"> +<li>50-55GB disk space for the R packages</li> <li>a filesystem that supports symbolic links</li> -<li>50-55GB disk space for the R packages when using the default -R package hosts (CRAN with archive, BIOC, R-Forge and Omegahat)</li> -<li><a lot of inodes for PORTAGE_RO_DISTDIR/__tmp__ and the overlay></li> +<li>there will be many small-sized files (ebuilds), +so a filesystem with lots of inodes and a small block size +may be advantageous</li> </ul> </dd> <dt>memory</dt> @@ -425,11 +468,8 @@ R package hosts (CRAN with archive, BIOC, R-Forge and Omegahat)</li> write mechanism in use. The amount can be halved (approximately) when using a slower one.</p> </dd> -<dt>other</dt> -<dd><p class="first last">No other hardware requirements. <as a measurement for computing speed, -i/o requirements: -overlay creation takes 3-4h on modern desktop hardware with overlay -writing to a tmpfs, most time is spent for Manifest creation></p> +<dt>time</dt> +<dd><p class="first last">Expect 3-6h execution time, depending on computing and I/O speed.</p> </dd> </dl> </blockquote> @@ -437,13 +477,13 @@ writing to a tmpfs, most time is spent for Manifest creation></p> </ul> </div> <div class="section" id="via-emerge-gentoo"> -<h2><a class="toc-backref" href="#id6">2.2 via emerge (Gentoo)</a></h2> +<h2><a class="toc-backref" href="#contents">2.2 via emerge (Gentoo)</a></h2> <p>A live ebuild is available, <a class="reference external" href="http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=blob;f=roverlay-9999.ebuild;hb=refs/heads/master">roverlay-9999.ebuild</a>. -Add it to your local overlay and run <tt class="docutils literal">emerge roverlay</tt>, which will also -install all config files into <em>/etc/roverlay</em>.</p> +Add it to your local overlay and run <tt class="docutils literal">emerge roverlay</tt>, which also installs +all necessary config files into <em>/etc/roverlay</em>.</p> </div> <div class="section" id="manual-installation"> -<h2><a class="toc-backref" href="#id7">2.3 Manual Installation</a></h2> +<h2><a class="toc-backref" href="#contents">2.3 Manual Installation</a></h2> <p>After installing the dependencies as listed in <a class="reference internal" href="#prerequisites">Prerequisites</a>, clone the <a class="reference external" href="http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=summary">roverlay git repo</a> and then install <em>roverlay</em> and its python modules:</p> @@ -454,12 +494,12 @@ git clone git://git.overlays.gentoo.org/proj/R_overlay.git </pre> <p><tt class="docutils literal">make install</tt> also accepts some variables, namely:</p> <ul class="simple"> -<li><em>DESTDIR</em> defaults to ''</li> +<li><em>DESTDIR</em></li> <li><em>BINDIR</em>, defaults to <em>DESTDIR</em>/usr/local/bin</li> <li><em>PYMOD_FILE_LIST</em>, which lists the installed python module files and defaults to './roverlay_files.list'</li> -<li><em>PYTHON</em>, name of path of the python interpreter that will run 'setup.py', -defaults to 'python'</li> +<li><em>PYTHON</em>, name or path of the python interpreter that is used to run +'setup.py', defaults to 'python'</li> </ul> <p><em>roverlay</em> can later be uninstalled with <tt class="docutils literal">make uninstall</tt>.</p> <div class="note"> @@ -472,19 +512,19 @@ The <em>make</em> targets take care of this.</p> </div> <div class="warning"> <p class="first admonition-title">Warning</p> -<p class="last">Support for this installation type is limited - it won't install/create +<p class="last">Support for this installation type is limited - it doesn't create/install any config files!</p> </div> </div> <div class="section" id="using-roverlay-without-installation"> -<h2><a class="toc-backref" href="#id8">2.4 Using <em>roverlay</em> without installation</a></h2> -<p>This is possible, too. Make sure to meet the dependencies as listed -in <a class="reference internal" href="#prerequisites">Prerequisites</a>. -Then, simply clone the git repository to a local directory that -will later be referenced as the <em>R Overlay src directory</em>.</p> +<h2><a class="toc-backref" href="#contents">2.4 Using <em>roverlay</em> without installation</a></h2> +<p>This is possible, too. +Make sure to meet the dependencies as listed in <a class="reference internal" href="#prerequisites">Prerequisites</a>. +Then, simply clone the git repository to a local directory that is referenced +as the <em>R Overlay src directory</em> from now on.</p> <div class="note"> <p class="first admonition-title">Note</p> -<p>You'll have to cd into the <em>R Overlay src directory</em> before running +<p>You have to <em>cd</em> into the <em>R Overlay src directory</em> before running <em>roverlay</em> to ensure that the python modules can be imported correctly.</p> <p>You can work around this by setting up a wrapper script:</p> <pre class="code sh last literal-block"> @@ -497,11 +537,11 @@ will later be referenced as the <em>R Overlay src directory</em>.</p> </div> </div> <div class="section" id="running-roverlay"> -<h1><a class="toc-backref" href="#id9">3 Running Roverlay</a></h1> +<h1><a class="toc-backref" href="#contents">3 Running Roverlay</a></h1> <div class="section" id="required-configuration-steps"> -<h2><a class="toc-backref" href="#id10">3.1 Required configuration steps</a></h2> -<p><em>roverlay</em> needs a configuration file to run.</p> -<p>If you've installed <em>roverlay</em> with <em>emerge</em>, it will look for the config +<h2><a class="toc-backref" href="#contents">3.1 Required configuration steps</a></h2> +<p><em>roverlay</em> needs a configuration file to run. +If you've installed <em>roverlay</em> with <em>emerge</em>, it will look for the config file in that order:</p> <ol class="arabic simple"> <li><em><current directory>/R-overlay.conf</em></li> @@ -537,7 +577,7 @@ see <a class="reference internal" href="#providing-a-package-mirror">Providing a <p class="last">Example: DISTFILES = ~/roverlay/distfiles</p> </dd> <dt>LOG_FILE</dt> -<dd><p class="first">This sets the log file.</p> +<dd><p class="first">This sets the log file. An empty value disables file logging.</p> <p class="last">Example: LOG_FILE = ~/roverlay/log/roverlay.log</p> </dd> <dt>LOG_LEVEL</dt> @@ -545,8 +585,8 @@ see <a class="reference internal" href="#providing-a-package-mirror">Providing a that don't override this option. Valid log levels are <tt class="docutils literal">DEBUG</tt>, <tt class="docutils literal">INFO</tt>, <tt class="docutils literal">WARN</tt>/<tt class="docutils literal">WARNING</tt>, <tt class="docutils literal">ERROR</tt> and <tt class="docutils literal">CRITICAL</tt>.</p> <p>Example: LOG_LEVEL = WARNING</p> -<div class="caution last"> -<p class="first admonition-title">Caution!</p> +<div class="note last"> +<p class="first admonition-title">Note</p> <p class="last">Be careful with low log levels, especially <em>DEBUG</em>. They produce a lot of messages that you probably don't want to see and increase the size of log files dramatically.</p> @@ -568,15 +608,13 @@ have reasonable defaults if <em>roverlay</em> has been installed using <em>emerg <dl class="docutils"> <dt>SIMPLE_RULES_FILE</dt> <dd><p class="first">This option lists dependency rule files and/or directories with -such files that should be used for dependency resolution (see -<a class="reference internal" href="#dependency-rules-resolving-dependencies">Dependency Rules / Resolving Dependencies</a>). +such files that should be used for dependency resolution. Although not required, this option is <strong>recommended</strong> since ebuild creation without dependency rules fails for most R packages.</p> <p class="last">Example: SIMPLE_RULES_FILE = ~/roverlay/config/simple-deprules.d</p> </dd> <dt>REPO_CONFIG</dt> -<dd><p class="first">A list with one or more files that list repositories -(see <a class="reference internal" href="#repositories-getting-packages">Repositories / Getting Packages</a>). +<dd><p class="first">A list with one or more files that list repositories. This option is <strong>required</strong> and can be overridden on the command line via one or more <tt class="docutils literal"><span class="pre">repo-config</span> <file></tt> statements.</p> <p class="last">Example: REPO_CONFIG = ~/roverlay/config/repo.list</p> @@ -602,10 +640,31 @@ named <em>R-packages.eclass</em> should be part of your installation.</p> <p>There's another option that is useful if you want to create new dependency rules, <a class="reference internal" href="#log-file-unresolvable">LOG_FILE_UNRESOLVABLE</a>, which will write all unresolvable dependencies to the specified file (one dependency string per line).</p> -<p>For details and a full configuration reference, see <a class="reference internal" href="#configuration-reference">Configuration Reference</a>.</p> +<div class="section" id="extended-configuration-where-to-go-from-here"> +<h3><a class="toc-backref" href="#contents">3.1.1 Extended Configuration / Where to go from here?</a></h3> +<p>Proceed with <a class="reference internal" href="#running-it">Running it</a> if you're fine with the default configuration +and the changes you've already made, otherwise the following chapters are +relevant and should provide you with the knowledge to determine the ideal +configuration.</p> +<dl class="docutils"> +<dt>Repositories</dt> +<dd>See <a class="reference internal" href="#repositories-getting-packages">Repositories / Getting Packages</a>, which describes how repositories +can be configured.</dd> +<dt>Dependency Rules</dt> +<dd>See <a class="reference internal" href="#dependency-rules">Dependency Rules</a>, which explains how dependency rules work and +how they're written.</dd> +<dt>Main Config</dt> +<dd>See <a class="reference internal" href="#configuration-reference">Configuration Reference</a> for all main config options like log file +rotation and assistance for writing new <em>dependency rules</em>.</dd> +<dt>Field Definition</dt> +<dd>Refer to <a class="reference internal" href="#id2">Field Definition</a> in case you want to change <em>how</em> R packages +are being read, e.g. if you want the 'Depents' information field (obviously +a typo) to be understood as 'Depends'.</dd> +</dl> +</div> </div> <div class="section" id="running-it"> -<h2><a class="toc-backref" href="#id11">3.2 Running it</a></h2> +<h2><a class="toc-backref" href="#contents">3.2 Running it</a></h2> <p>If you've installed <em>roverlay</em>, you can run it with <tt class="docutils literal">roverlay</tt>, otherwise you'll have to cd into the <em>R overlay src directory</em> and run <tt class="docutils literal">./roverlay.py</tt>.</p> <p>In any case, the basic <em>roverlay</em> script usage is</p> @@ -621,7 +680,8 @@ as described in <a class="reference internal" href="#required-configuration-step The default command is <em>create</em>, which downloads the R packages (unless explicitly forbidden to do so) and generates the overlay. This is the desired behavior in most cases, so simply running <tt class="docutils literal">roverlay</tt> should be -fine.</p> +fine. See <a class="reference internal" href="#basic-implementation-overview">Basic Implementation Overview</a> if you'd rather like to know +in detail what <em>roverlay</em> does before running it.</p> <p><em>roverlay</em> also accepts some <strong>options</strong>, most notably:</p> <table class="docutils option-list" frame="void" rules="none"> <col class="option" /> @@ -645,7 +705,7 @@ The latter one is faster, but consumes more memory since ebuilds must be kept until all packages have been processed. Test results show that memory consumption increases by factor 2 when using the faster write mechanism (at ca. 95% ebuild creation success rate), -<while ebuild write time decreases by ???>.</p> +<<while ebuild write time decreases by ???>>.</p> <p class="last">Summary: Expect 300 (slow) or 600MB (fast) memory consumption when using the default package repositories.</p> </td></tr> @@ -717,18 +777,19 @@ and resolving dependencies.</p> </dl> </div> <div class="section" id="providing-a-package-mirror"> -<h2><a class="toc-backref" href="#id12">3.3 Providing a package mirror</a></h2> -<p><No recommendations at this time. The current ManifestCreation implementation -creates a directory <em><distfiles root>/__tmp__</em> with symlinks to all packages, -which could be used for providing packages, but this may change -in near future since external Manifest creation is too slow -(takes >60% of overlay creation time).></p> +<h2><a class="toc-backref" href="#contents">3.3 Providing a package mirror</a></h2> +<p>No recommendations available at this time. +The current ManifestCreation implementation creates a directory +<em><distfiles root>/__tmp__</em> with symlinks to all packages, which could be used +for providing packages, but this may change in near future since external +Manifest creation is too slow (takes >60% of overlay creation time).</p> </div> </div> -<div class="section" id="implementation-overview"> -<h1><a class="toc-backref" href="#id13">4 Implementation Overview</a></h1> -<p>This section gives a basic overview of how <em>roverlay</em> works.</p> -<p><how <em>roverlay</em> basically works:></p> +<div class="section" id="basic-implementation-overview"> +<h1><a class="toc-backref" href="#contents">4 Basic Implementation Overview</a></h1> +<div class="section" id="how-roverlay-works"> +<h2><a class="toc-backref" href="#contents">4.1 How <em>roverlay</em> works</a></h2> +<p>These are the steps that <em>roverlay</em> performs:</p> <ol class="arabic simple"> <li><strong>sync</strong> - get R packages using various methods (rsync, http, local directory)</li> @@ -744,21 +805,28 @@ an ebuild</li> (thread-able on a per package basis)</li> </ol> <blockquote> -<ul class="simple"> -<li>read <em>p</em>'s DESCRIPTION file that contains information fields -like 'Depends', 'Description' and 'Suggests'</li> -<li>resolve <em>p</em>'s dependencies<ul> -<li>differentiate between <em>required</em> and <em>optional</em> dependencies +<ul> +<li><p class="first">read <em>p</em>'s DESCRIPTION file that contains information fields +like 'Depends', 'Description' and 'Suggests'</p> +</li> +<li><p class="first">resolve <em>p</em>'s dependencies</p> +<ul> +<li><p class="first">differentiate between <em>required</em> and <em>optional</em> dependencies (for example, dependencies from the 'Depends' field are required, -while those from 'Suggests' are optional)</li> -<li><strong>immediately stop</strong> processing <em>p</em> if a <em>required</em> dependency -cannot be resolved in which case a valid ebuild cannot be created</li> +while those from 'Suggests' are optional)</p> +</li> +<li><p class="first"><strong>immediately stop</strong> processing <em>p</em> if a <em>required</em> dependency +cannot be resolved in which case a valid ebuild cannot be created</p> +<p>See also: <a class="reference internal" href="#dependency-resolution">Dependency Resolution</a></p> +</li> </ul> </li> -<li>create an ebuild for <em>p</em> by using the dependency resolution results -and a few information fields like 'Description'</li> -<li><strong>done</strong> with <em>p</em> - the overlay creator takes control over <em>p</em> -and may decide to write <em>p</em>'s ebuild now (or later)</li> +<li><p class="first">create an ebuild for <em>p</em> by using the dependency resolution results +and a few information fields like 'Description'</p> +</li> +<li><p class="first"><strong>done</strong> with <em>p</em> - the overlay creator takes control over <em>p</em> +and may decide to write <em>p</em>'s ebuild now (or later)</p> +</li> </ul> </blockquote> <ol class="arabic simple" start="5"> @@ -778,14 +846,166 @@ and may decide to write <em>p</em>'s ebuild now (or later)</li> </ul> </li> </ol> +</div> +<div class="section" id="expected-overlay-result-structure-of-the-generated-overlay"> +<h2><a class="toc-backref" href="#contents">4.2 Expected Overlay Result / Structure of the generated overlay</a></h2> +<p>Assuming that you're using the default configuration (where possible) and +the <em>R-packages</em> eclass file, the result should look like:</p> +<pre class="code text literal-block"> +<overlay root>/ +<overlay root>/eclass +<overlay root>/eclass/R-packages.eclass +<overlay root>/profiles +<overlay root>/profiles/categories +<overlay root>/profiles/repo_name +<overlay root>/profiles/use.desc +<overlay root>/sci-R/<many directories per R package> +<overlay root>/sci-R/seewave/ +<overlay root>/sci-R/seewave/Manifest +<overlay root>/sci-R/seewave/metadata.xml +<overlay root>/sci-R/seewave/seewave-1.5.9.ebuild +<overlay root>/sci-R/seewave/seewave-1.6.4.ebuild +</pre> +<div class="section" id="expected-ebuild-result"> +<h3><a class="toc-backref" href="#contents">4.2.1 Expected Ebuild Result</a></h3> +<dl class="docutils"> +<dt>Ebuild Template</dt> +<dd><pre class="code text first literal-block"> +<ebuild header> +inherit <eclass(es)> + +DESCRIPTION="<the R package's description>" +SRC_URI="<src uri for the R package>" + +IUSE="${IUSE:-} + R_suggests +" +R_SUGGESTS="<R package suggestions>" +DEPEND="<build time dependencies for the R package>" +RDEPEND="${DEPEND:-} + <runtime dependencies> + R_suggests? ( ${R_SUGGESTS} ) +" + +_UNRESOLVED_PACKAGES=(<unresolvable, but optional dependencies>) +</pre> +<p>Some of the variables may be missing if they're not needed.</p> +<p>A really minimal ebuild would look like:</p> +<pre class="code text last literal-block"> +<ebuild header> +inherit <eclass(es)> + +SRC_URI="<src uri for the R package>" +</pre> +</dd> +<dt>Example: seewave 1.6.4 from CRAN:</dt> +<dd><p class="first">The default ebuild header (which cannot be changed) automatically sets +the ebuild's copyright year to 1999-<em><current year></em>.</p> +<pre class="code sh last literal-block"> +<span class="c"># Copyright 1999-2012 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 +# $Header: $ +</span> +<span class="nv">EAPI</span><span class="o">=</span>4 +inherit R-packages + +<span class="nv">DESCRIPTION</span><span class="o">=</span><span class="s2">"Time wave analysis and graphical representation"</span> +<span class="nv">SRC_URI</span><span class="o">=</span><span class="s2">"http://cran.r-project.org/src/contrib/seewave_1.6.4.tar.gz"</span> + +<span class="nv">IUSE</span><span class="o">=</span><span class="s2">"${IUSE:-} + R_suggests +"</span> +<span class="nv">R_SUGGESTS</span><span class="o">=</span><span class="s2">"sci-R/sound + sci-R/audio +"</span> +<span class="nv">DEPEND</span><span class="o">=</span><span class="s2">"sci-R/fftw + sci-R/tuneR + >=dev-lang/R-2.15.0 + sci-R/rpanel + sci-R/rgl +"</span> +<span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"${DEPEND:-} + media-libs/flac + sci-libs/fftw + media-libs/libsndfile + R_suggests? ( ${R_SUGGESTS} ) +"</span> +</pre> +</dd> +<dt>Example: MetaPCA 0.1.3 from CRAN's archive:</dt> +<dd><p class="first">Note the shortened <em>DESCRIPTION</em> variable that points to the <em>metadata.xml</em> +file. This happens if the description is too long.</p> +<pre class="code sh last literal-block"> +<span class="c"># Copyright 1999-2012 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 +# $Header: $ +</span> +<span class="nv">EAPI</span><span class="o">=</span>4 +inherit R-packages + +<span class="nv">DESCRIPTION</span><span class="o">=</span><span class="s2">"MetaPCA: Meta-analysis in the Di... (see metadata)"</span> +<span class="nv">SRC_URI</span><span class="o">=</span><span class="s2">"http://cran.r-project.org/src/contrib/Archive/MetaPCA/MetaPCA_0.1.3.tar.gz"</span> + +<span class="nv">IUSE</span><span class="o">=</span><span class="s2">"${IUSE:-} + R_suggests +"</span> +<span class="nv">R_SUGGESTS</span><span class="o">=</span><span class="s2">"sci-R/doMC + sci-R/affy + sci-R/ellipse + sci-R/pcaPP + sci-R/MASS + sci-R/impute + sci-R/doSMP + sci-R/GEOquery +"</span> +<span class="nv">DEPEND</span><span class="o">=</span><span class="s2">"sci-R/foreach"</span> +<span class="nv">RDEPEND</span><span class="o">=</span><span class="s2">"${DEPEND:-} + R_suggests? ( ${R_SUGGESTS} ) +"</span> + +<span class="nv">_UNRESOLVED_PACKAGES</span><span class="o">=(</span><span class="s1">'hgu133plus2.db'</span><span class="o">)</span> +</pre> +</dd> +</dl> +</div> +<div class="section" id="expected-metadata-xml-result"> +<h3><a class="toc-backref" href="#contents">4.2.2 Expected <em>metadata.xml</em> Result</a></h3> +<p>The <em>metadata.xml</em> will contain the full description for the latest version +of a package.</p> +<dl class="docutils"> +<dt>Example: seewave from CRAN</dt> +<dd><p class="first">Note the ' // ' delimiter. It will be used to separate description strings +if a package has more than one, e.g. one in its <em>Title</em> and one in its +<em>Description</em> information field.</p> +<pre class="code xml last literal-block"> +<span class="cp"><?xml version="1.0" encoding="UTF-8"?></span> +<span class="cp"><!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd"></span> +<span class="nt"><pkgmetadata></span> + <span class="nt"><longdescription></span> + Time wave analysis and graphical representation // seewave + provides functions for analysing, manipulating, displaying, + editing and synthesizing time waves (particularly sound). This + package processes time analysis (oscillograms and envelopes), + spectral content, resonance quality factor, entropy, cross + correlation and autocorrelation, zero-crossing, dominant + frequency, analytic signal, frequency coherence, 2D and 3D + spectrograms and many other analyses. + <span class="nt"></longdescription></span> +<span class="nt"></pkgmetadata></span> +</pre> +</dd> +</dl> +</div> +</div> +</div> <div class="section" id="repositories-getting-packages"> -<h2><a class="toc-backref" href="#id14">4.1 Repositories / Getting Packages</a></h2> +<span id="repositories"></span><h1><a class="toc-backref" href="#contents">5 Repositories / Getting Packages</a></h1> <p><em>roverlay</em> is capable of downloading R packages via rsync and http, -and is able to use any packages locally available. The concrete method used -to get and use the packages is determined by the concrete -<strong>type of a repository</strong> and that's what this section is about.</p> +and is able to use any packages locally available. The method used to get and +use the packages is determined by the concrete <strong>type of a repository</strong> and +that's what this section is about. It also covers repository configuration.</p> <div class="section" id="a-word-about-repo-config-files"> -<span id="repo-config"></span><h3><a class="toc-backref" href="#id15">4.1.1 A word about repo config files</a></h3> +<span id="repo-config"></span><h2><a class="toc-backref" href="#contents">5.1 A word about repo config files</a></h2> <p>Repo config files use <a class="reference external" href="http://docs.python.org/library/configparser.html">ConfigParser</a> syntax (known from ini files).</p> <p>Each repo entry section is introduced with <tt class="docutils literal">[<repo name>]</tt> and defines</p> <ul class="simple"> @@ -798,21 +1018,22 @@ verification</li> <p>Such options are declared with <tt class="docutils literal"><option> = <value></tt> in the repo entry.</p> <p id="repo-config-options">The common options for repository entries are:</p> <ul> -<li><p class="first"><em>type</em>, which declares the repository type. Available types are:</p> -<blockquote> +<li><p class="first"><em>type</em> (optional), which declares the repository type. +Available types are:</p> <ul class="simple"> <li><a class="reference internal" href="#rsync">rsync</a></li> <li><a class="reference internal" href="#websync-repo">websync_repo</a></li> <li><a class="reference internal" href="#websync-pkglist">websync_pkglist</a></li> <li><a class="reference internal" href="#local">local</a></li> </ul> -</blockquote> -<p>Defaults to <em>rsync</em></p> +<p>Defaults to <em>rsync</em>.</p> </li> -<li><p class="first"><em>src_uri</em>, which declares how ebuilds can download the packages. Some repo -types use this for downloading, too.</p> +<li><p class="first"><em>src_uri</em> (<strong>required</strong>), +which declares how ebuilds can download the packages. +Some repo types use this for downloading, too.</p> </li> -<li><p class="first"><em>directory</em>, which explicitly sets the package directory to use. +<li><p class="first"><em>directory</em> (optional), +which explicitly sets the package directory to use. The default behavior is to use <a class="reference internal" href="#distfiles-root">DISTFILES_ROOT</a>/<repo name></p> </li> </ul> @@ -823,9 +1044,9 @@ creating the default directory.</p> </div> </div> <div class="section" id="rsync-repos"> -<span id="rsync"></span><h3><a class="toc-backref" href="#id16">4.1.2 Rsync repos</a></h3> +<span id="rsync"></span><h2><a class="toc-backref" href="#contents">5.2 Rsync repos</a></h2> <p>Runs <em>rsync</em> to download packages. Automatic sync retries are supported if -<em>rsync</em>'s exit codes indicates chances of success. +<em>rsync</em>'s exit code indicates chances of success. For example, up to 3 retries are granted if <em>rsync</em> returns <em>Partial transfer due to vanished source files</em> which likely happens when syncing big repositories like CRAN.</p> @@ -866,7 +1087,7 @@ Options with whitespace are not supported.</li> </ul> </div> <div class="section" id="getting-packages-from-a-repository-that-supports-http-only"> -<span id="websync-repo"></span><h3><a class="toc-backref" href="#id17">4.1.3 Getting packages from a repository that supports http only</a></h3> +<span id="websync-repo"></span><h2><a class="toc-backref" href="#contents">5.3 Getting packages from a repository that supports http only</a></h2> <p>This is your best bet if the remote is a repository but doesn't offer rsync access. Basic digest verification is supported (MD5). The remote has to have a package list file, typically named @@ -916,7 +1137,7 @@ Defaults to <em>src_uri</em>/<em>pkglist_file</em></li> </div> </div> <div class="section" id="getting-packages-from-several-remotes-using-http-and-a-package-list"> -<span id="websync-pkglist"></span><h3><a class="toc-backref" href="#id18">4.1.4 Getting packages from several remotes using http and a package list</a></h3> +<span id="websync-pkglist"></span><h2><a class="toc-backref" href="#contents">5.4 Getting packages from several remotes using http and a package list</a></h2> <p>This is not a real repository type, instead it creates a <em>local</em> repository by downloading single R packages from different remotes. Its only requirement is that a package is downloadable via http. @@ -929,7 +1150,7 @@ http://download.r-forge.r-project.org/src/contrib/zoo_1.7-7.tar.gz http://www.omegahat.org/R/src/contrib/Aspell_0.2-0.tar.gz ... </pre> -<p>Comments are not supported. Assuming that such a package list exists as <at?> +<p>Comments are not supported. Assuming that such a package list exists at <em>~/roverlay/config/http_packages.list</em>, an example entry in the repo config file would be:</p> <pre class="code ini literal-block"> @@ -943,7 +1164,7 @@ file would be:</p> </ul> </div> <div class="section" id="using-local-directories"> -<span id="local"></span><h3><a class="toc-backref" href="#id19">4.1.5 Using local directories</a></h3> +<span id="local"></span><h2><a class="toc-backref" href="#contents">5.5 Using local directories</a></h2> <p>Using local package directories is possible, too.</p> <p>Example:</p> <pre class="code ini literal-block"> @@ -965,47 +1186,19 @@ you should consider using one of the <strong>websync</strong> repo types, </div> </div> </div> -<div class="section" id="dependency-rules-resolving-dependencies"> -<h2><a class="toc-backref" href="#id20">4.2 Dependency Rules / Resolving Dependencies</a></h2> -<div class="section" id="dependency-types"> -<h3><a class="toc-backref" href="#id21">4.2.1 Dependency types</a></h3> -<p>Every <em>dependency string</em> has a <em>dependency type</em> that declares how a -dependency should be resolved. It has one or more of these properties:</p> -<dl class="docutils"> -<dt>Mandatory</dt> -<dd>Ebuild creation fails if the <em>dependency string</em> in question cannot -be resolved.</dd> -<dt>Optional</dt> -<dd>The opposite of <em>Mandatory</em>, ebuild creation keeps going even if the -<em>dependency string</em> is unresolvable.</dd> -<dt>Package Dependency</dt> -<dd>This declares that the <em>dependency string</em> could be another R package.</dd> -<dt>System Dependency</dt> -<dd>This declares that the <em>dependency string</em> could be a system dependency, -e.g. a scientific library or a video encoder.</dd> -<dt>Try other dependency types</dt> -<dd>This declares that the <em>dependency string</em> can be resolved by ignoring its -dependency type partially. This property allows to resolve -package dependencies as system dependencies and vice versa.</dd> -</dl> -<p><em>Mandatory</em> and <em>Option</em> are mutually exclusive.</p> -<p>The <em>dependency type</em> of a <em>dependency string</em> is determined by the its origin, -i.e. info field in the package's DESCRIPTION file. -The <em>Suggests</em> field, for example, -gets the <em>"package dependency and optional"</em> type, -whereas the <em>SystemRequirements</em> gets <em>"system dependency and mandatory"</em>.</p> -</div> +<div class="section" id="dependency-rules"> +<h1><a class="toc-backref" href="#contents">6 Dependency Rules</a></h1> <div class="section" id="simple-dependency-rules"> -<h3><a class="toc-backref" href="#id22">4.2.2 Simple Dependency Rules</a></h3> +<h2><a class="toc-backref" href="#contents">6.1 Simple Dependency Rules</a></h2> <p><em>Simple dependency rules</em> use a dictionary and string transformations to resolve dependencies. <em>Fuzzy simple dependency rules</em> extend these by -a set of regexes, which allows to resolve many dependency strings that -minimally differ (e.g. only in the required version and/or comments: +a set of regular expressions, which allows to resolve many dependency strings +that minimally differ (e.g. only in the required version and/or comments: <cite>R (>= 2.10)</cite> and <cite>R [2.14] from http://www.r-project.org/</cite>) with a single dictionary entry.</p> <p>This is the only rule implementation currently available.</p> <div class="section" id="rule-variants"> -<h4><a class="toc-backref" href="#id23">4.2.2.1 Rule Variants</a></h4> +<h3><a class="toc-backref" href="#contents">6.1.1 Rule Variants</a></h3> <dl class="docutils"> <dt>default</dt> <dd>The expected behavior of a dictionary-based rule: It matches one or more @@ -1016,7 +1209,7 @@ resolve them as <strong>nothing</strong>.</dd> </dl> </div> <div class="section" id="rule-types"> -<h4><a class="toc-backref" href="#id24">4.2.2.2 Rule types</a></h4> +<h3><a class="toc-backref" href="#contents">6.1.2 Rule types</a></h3> <dl class="docutils"> <dt>Simple Rules</dt> <dd><p class="first">A simple rule resolves <strong>exact string matches</strong> (case-insensitive).</p> @@ -1057,9 +1250,9 @@ as "<dev-lang/R-2.14"</li> </dl> </div> <div class="section" id="rule-file-examples"> -<h4><a class="toc-backref" href="#id25">4.2.2.3 Rule File Examples</a></h4> +<h3><a class="toc-backref" href="#contents">6.1.3 Rule File Examples</a></h3> <p>This sections lists some rule file examples. -See <a class="reference internal" href="#rule-file-syntax">Rule File Syntax</a> for a formal<precise,..?> description.</p> +See <a class="reference internal" href="#rule-file-syntax">Rule File Syntax</a> for a formal description.</p> <dl class="docutils"> <dt>Example 1 - <em>default</em> fuzzy rule</dt> <dd><p class="first">A rule that matches many dependencies on dev-lang/R, for example @@ -1073,9 +1266,15 @@ resolves them as '>=dev-lang/R-2.12', '>=dev-lang/R-2.14', <dt>Example 2 - <em>default</em> simple rule stub</dt> <dd><p class="first">A rule that case-insensitively matches 'zoo' and resolves it as 'sci-R/zoo', assuming the OVERLAY_CATEGORY is set to 'sci-R':</p> -<pre class="code text last literal-block"> +<pre class="code text literal-block"> zoo </pre> +<div class="note last"> +<p class="first admonition-title">Note</p> +<p class="last">R Package rules are dynamically created at runtime and therefore not +needed. Write them only if certain R package dependencies cannot +be resolved. See <em>Selfdep</em> in <a class="reference internal" href="#rule-file-syntax">Rule File Syntax</a> for details.</p> +</div> </dd> <dt>Example 3 - <em>default</em> simple rule</dt> <dd><p class="first">A rule that matches several <em>dependency strings</em> and resolves them @@ -1109,7 +1308,7 @@ if you've installed <em>roverlay</em> with <em>emerge</em>, else in <em><R Overlay src directory>/simple-deprules.d</em>.</p> </div> <div class="section" id="rule-file-syntax"> -<span id="dependency-rule-file-syntax"></span><h4><a class="toc-backref" href="#id26">4.2.2.4 Rule File Syntax</a></h4> +<span id="dependency-rule-file-syntax"></span><h3><a class="toc-backref" href="#contents">6.1.4 Rule File Syntax</a></h3> <p>Simple dependency rule files have a special syntax. Each rule is introduced with the resolving <em>dependency</em> prefixed by a <em>keychar</em> that sets the rule type if required. The <em>dependency strings</em> resolved by this rule are listed @@ -1176,8 +1375,8 @@ Use braces <em>( ~... )</em> to work around that.</p> <dt>Multi line rules</dt> <dd><p class="first">resolve several <em>dependency strings</em>. Their rule block begins with '{' + newline, followed by one -<em>dependency string</em> per line, and ends with '}'.</p> -<dl class="last docutils"> +<em>dependency string</em> per line, and ends with '}' in a separate line.</p> +<dl class="docutils"> <dt>Syntax:</dt> <dd><pre class="code text first last literal-block"> [<keychar>]<dependency> { @@ -1188,13 +1387,49 @@ Their rule block begins with '{' + newline, followed by one </pre> </dd> </dl> +<div class="note last"> +<p class="first admonition-title">Note</p> +<p class="last">Technically, this rule syntax can be used to specify rules with +zero or more <em>dependency strings</em>. An empty rule doesn't make much sense, +though.</p> +</div> +</dd> +<dt>Comments</dt> +<dd>start with <strong>#</strong>. There are a few exceptions to that, the <em>#deptype</em> and +<em>#! NOPARSE</em> keywords. Comments inside rule blocks are not allowed and +will be read as normal <em>dependency strings</em>.</dd> +<dt>Selfdep</dt> +<dd><p class="first">This is another name for <em>dependency strings</em> that are resolved by an +R package with the same name, which is also part of the overlay being +created.</p> +<p>Example: <em>zoo</em> is resolved as <em>sci-R/zoo</em>, assuming that <a class="reference internal" href="#overlay-category">OVERLAY_CATEGORY</a> +is set to <em>sci-R</em></p> +<p>Writing selfdep rules is not necessary since <em>roverlay</em> automatically +creates rules for all known R packages (see <a class="reference internal" href="#dynamic-selfdep-rule-pool">Dynamic Selfdep Rule Pool</a> +for details).</p> +<p>There are a few exceptions to that in which case selfdep rules have to +be written:</p> +<ul class="simple"> +<li>The <em>dependency string</em> is assumed to be a system dependency (not an +R package). This is likely a "bug" in the DESCRIPTION file of the +R package being processed.</li> +<li>The R package name is not ebuild-name compliant (e.g. contains the '.' +char, which is remapped to '_'.). +Most <em>char remap</em> cases are handled properly, but there may be a few +exceptions.</li> +</ul> +<div class="caution last"> +<p class="first admonition-title">Caution!</p> +<p class="last">Writing unnecessary selfdep rules slows dependency resolution down! +Each rule will exist twice, once as <em>dynamic</em> rule and once as +the written one.</p> +</div> </dd> <dt>Rule Stubs</dt> -<dd><p class="first">There's a shorter syntax for dependencies that are resolved within the -created overlay. For example, if your OVERLAY_CATEGORY is -<em>sci-R</em>, <em>zoo</em> should be resolved as <em>sci-R/zoo</em>. -This rule can be written as a single word, <em>zoo</em>. Such stubs are called -<strong>selfdeps</strong>.</p> +<dd><p class="first">There's a shorter syntax for selfdeps. +For example, if your OVERLAY_CATEGORY is <em>sci-R</em>, +<em>zoo</em> should be resolved as <em>sci-R/zoo</em>. +This rule can be written as a single word, <em>zoo</em>.</p> <dl class="last docutils"> <dt>Syntax:</dt> <dd><pre class="code text first last literal-block"> @@ -1203,206 +1438,13 @@ This rule can be written as a single word, <em>zoo</em>. Such stubs are called </dd> </dl> </dd> -<dt>Comments</dt> -<dd>start with <strong>#</strong>. There are a few exceptions to that, the <em>#deptype</em> and -<em>#! NOPARSE</em> keywords. Comments inside rule blocks are not allowed and -will be read as normal <em>dependency strings</em>.</dd> </dl> </div> </div> </div> -<div class="section" id="expected-overlay-result-structure-of-the-generated-overlay"> -<h2><a class="toc-backref" href="#id27">4.3 Expected Overlay Result / Structure of the generated overlay</a></h2> -<p>Assuming that you're using the default configuration (where possible) and -the <em>R-packages</em> eclass file, the result should look like:</p> -<pre class="code text literal-block"> -<overlay root>/ -<overlay root>/eclass -<overlay root>/eclass/R-packages.eclass -<overlay root>/profiles -<overlay root>/profiles/categories -<overlay root>/profiles/repo_name -<overlay root>/profiles/use.desc -<overlay root>/sci-R/<many directories per R package> -<overlay root>/sci-R/seewave/ -<overlay root>/sci-R/seewave/Manifest -<overlay root>/sci-R/seewave/metadata.xml -<overlay root>/sci-R/seewave/seewave-1.5.9.ebuild -<overlay root>/sci-R/seewave/seewave-1.6.4.ebuild -</pre> -</div> -</div> -<div class="section" id="depres-console"> -<h1><a class="toc-backref" href="#id28">5 DepRes Console</a></h1> -<p>As previously stated, the <em>DepRes Console</em> is only meant for <strong>testing</strong>. -It's an interactive console with the following features:</p> -<ul class="simple"> -<li>resolve dependencies</li> -<li>create new dependency rules (<strong>only single line rules</strong>)</li> -<li>create dependency rules for each R package found in a directory</li> -<li>load rules from files</li> -<li>save rules to a file</li> -</ul> -<p>Rules are managed in a set. These so-called <em>rule pools</em> are organized in -a <em>first-in-first-out</em> data structure that allows -to create or remove them easily at runtime.</p> -<p>Running <tt class="docutils literal">roverlay depres_console</tt> will print a welcome message that -lists all available commands and a few usage hints.</p> -<p>For reference, these commands are currently available:</p> -<table border="1" class="docutils"> -<colgroup> -<col width="29%" /> -<col width="71%" /> -</colgroup> -<thead valign="bottom"> -<tr><th class="head">command</th> -<th class="head">description</th> -</tr> -</thead> -<tbody valign="top"> -<tr><td>help, -h</td> -<td>lists all commands</td> -</tr> -<tr><td>help --list, -h --list</td> -<td>lists all help topics for which help is available</td> -</tr> -<tr><td>help <em><cmd></em>, -h <em><cmd></em></td> -<td>prints a command-specific help message</td> -</tr> -<tr><td>load <em><file|dir></em>, -l <em><file|dir></em></td> -<td>loads a rule file or a directory with rule files -into a new <em>rule pool</em></td> -</tr> -<tr><td>load_conf, -lc</td> -<td>loads the rule files listed in the config file -into a new <em>rule pool</em></td> -</tr> -<tr><td>addrule <em><rule></em> -+ <em><rule></em></td> -<td>creates a new rule and adds it to the topmost, -i.e. latest <em>rule pool</em>. This command uses -<a class="reference internal" href="#rule-file-syntax">Rule File Syntax</a>, but multi line rules are -not supported.</td> -</tr> -<tr><td>add_pool, -<<</td> -<td>creates a new <em>rule pool</em></td> -</tr> -<tr><td>unwind, ->></td> -<td>removes the topmost <em>rule pool</em> and all of its -rules</td> -</tr> -<tr><td>resolve <em><dep></em>, -? <em><dep></em></td> -<td>tries to resolve the given dependency string and -prints the result</td> -</tr> -<tr><td>print, p</td> -<td>prints the rules of the topmost <em>rule pool</em></td> -</tr> -<tr><td>print all, p all</td> -<td>prints the rules of all <em>rule pools</em></td> -</tr> -<tr><td>write <em><file></em>, -w <em><file></em></td> -<td>writes the rules of the topmost <em>rule pool</em> into -<em><file></em></td> -</tr> -<tr><td>cd <em><dir></em></td> -<td>changes the working directory, also creates it if -necessary</td> -</tr> -<tr><td>scandir <em><dir></em>, -sd <em><dir></em></td> -<td>creates dependency rules for each R package found -in <em><dir></em></td> -</tr> -<tr><td>set, unset</td> -<td>prints the status of currently (in)active modes</td> -</tr> -<tr><td>set <em><mode></em>, -unset <em><mode></em></td> -<td>set or unsets <em><mode></em>. There's only one mode to -control, the <em>shlex mode</em> which controls how -command arguments are parsed</td> -</tr> -<tr><td>mkhelp</td> -<td>verifies that each accessible command has a help -message</td> -</tr> -<tr><td>exit, qq, q</td> -<td>exit the <em>DepRes Console</em></td> -</tr> -</tbody> -</table> -<dl class="docutils"> -<dt>Example Session:</dt> -<dd><pre class="code first last literal-block"> -== depres console == -Run 'help' to list all known commands -More specifically, 'help <cmd>' prints a help message for the given -command, and 'help --list' lists all help topics available -Use 'load_conf' or 'lc' to load the configured rule files - -commands: load, unwind, set, help, >>, load_conf, <<, cd, mkhelp, -resolve, lc, add_pool, addrule, h, +, l, li, write, p, r, ?, w, print, -sd, unset, scandir -exit-commands: q, qq, exit - -cmd % + ~dev-lang/R :: R language -new rules: -~dev-lang/R :: R language ---- --- -command succeeded. - -cmd % ? R language -Trying to resolve ('R language',). -Resolved as: ('dev-lang/R',) - -cmd % ? R language [ 2.15 ] -Trying to resolve ('R language [ 2.15 ]',). -Resolved as: ('>=dev-lang/R-2.15',) - -cmd % ? R -Trying to resolve ('R',). -Channel returned None. At least one dep couldn't be resolved. - -cmd % p -~dev-lang/R :: R language - -cmd % >> -Pool removed from resolver. - -cmd % p - -cmd % ? R language -Trying to resolve ('R language',). -Channel returned None. At least one dep couldn't be resolved. - -cmd % exit -</pre> -</dd> -</dl> -</div> <div class="section" id="configuration-reference"> -<h1><a class="toc-backref" href="#id29">6 Configuration Reference</a></h1> -<div class="section" id="dependency-rules"> -<h2><a class="toc-backref" href="#id30">6.1 Dependency Rules</a></h2> -<p><merge with basic..overview::deprules></p> -</div> -<div class="section" id="repositories"> -<h2><a class="toc-backref" href="#id31">6.2 Repositories</a></h2> -<p><merge with basic..overview::repo></p> -</div> -<div class="section" id="main-config"> -<h2><a class="toc-backref" href="#id32">6.3 Main Config</a></h2> -<p>The main config file uses "<option> = <value>" syntax, comment lines start +<h1><a class="toc-backref" href="#contents">7 Configuration Reference</a></h1> +<p>The main config file uses '<option> = <value>' syntax, comment lines start with <strong>#</strong>. Variable substitution ("${X}") is not supported. Quotes around the value are optional and allow to span long values over multiple lines. Whitespace is ignored, file <strong>paths must not contain whitespace</strong>.</p> @@ -1413,7 +1455,7 @@ Whitespace is ignored, file <strong>paths must not contain whitespace</strong>.< <em>WARNING</em>, <em>ERROR</em> and <em>CRITICAL</em>.</dd> <dt>bool</dt> <dd><p class="first">Value is a string that represents a boolean value.</p> -<p>This table illustrates which values strings are accepted:</p> +<p>This table illustrates which value strings are accepted:</p> <table border="1" class="last docutils"> <colgroup> <col width="59%" /> @@ -1454,7 +1496,7 @@ restrictions. It's better to comment out options.</dd> </dl> <p>The following sections will list all config entries.</p> <div class="section" id="misc-options"> -<h3><a class="toc-backref" href="#id33">6.3.1 misc options</a></h3> +<h2><a class="toc-backref" href="#contents">7.1 misc options</a></h2> <dl class="docutils" id="distfiles"> <dt>DISTFILES</dt> <dd>Alias for <a class="reference internal" href="#distfiles-root">DISTFILES_ROOT</a>.</dd> @@ -1473,7 +1515,7 @@ location (see <a class="reference internal" href="#repo-config-options">repo con </dl> </div> <div class="section" id="overlay-options"> -<h3><a class="toc-backref" href="#id34">6.3.2 overlay options</a></h3> +<h2><a class="toc-backref" href="#contents">7.2 overlay options</a></h2> <dl class="docutils" id="eclass"> <dt>ECLASS</dt> <dd>Alias to <a class="reference internal" href="#overlay-eclass">OVERLAY_ECLASS</a>.</dd> @@ -1520,7 +1562,7 @@ per R package. All others will be removed.</p> </dl> </div> <div class="section" id="other-config-files"> -<h3><a class="toc-backref" href="#id35">6.3.3 other config files</a></h3> +<h2><a class="toc-backref" href="#contents">7.3 other config files</a></h2> <p>Some config config options are split from the main config file for various reasons:</p> <ul class="simple"> @@ -1569,13 +1611,11 @@ much without dependency resolution.</p> </dl> </div> <div class="section" id="logging"> -<h3><a class="toc-backref" href="#id36">6.3.4 logging</a></h3> +<h2><a class="toc-backref" href="#contents">7.4 logging</a></h2> <dl class="docutils" id="log-date-format"> <dt>LOG_DATE_FORMAT</dt> -<dd><p class="first">The date format used in log messages.</p> -<p class="last">Defaults to <em>%F %H:%M:%S</em>. -(<<explain the date format by referencing to date(1) -or python's logging->? module>>)</p> +<dd><p class="first">The date format (ISO8601) used in log messages.</p> +<p class="last">Defaults to <em>%F %H:%M:%S</em>.</p> </dd> </dl> <dl class="docutils" id="log-enabled"> @@ -1596,7 +1636,7 @@ level set will use this value.</p> </dd> </dl> <div class="section" id="console-logging"> -<h4><a class="toc-backref" href="#id37">6.3.4.1 console logging</a></h4> +<h3><a class="toc-backref" href="#contents">7.4.1 console logging</a></h3> <dl class="docutils" id="log-console"> <dt>LOG_CONSOLE</dt> <dd><p class="first">Enables/Disables logging to console. The value has to be a <em>bool</em>.</p> @@ -1617,7 +1657,7 @@ level set will use this value.</p> </dl> </div> <div class="section" id="file-logging"> -<h4><a class="toc-backref" href="#id38">6.3.4.2 file logging</a></h4> +<h3><a class="toc-backref" href="#contents">7.4.2 file logging</a></h3> <dl class="docutils" id="log-file"> <dt>LOG_FILE</dt> <dd><p class="first">Sets the log file. File logging will be disabled if this option does not @@ -1676,7 +1716,7 @@ files will be kept.</p> </div> </div> <div class="section" id="options-for-debugging-manual-dependency-rule-creation-and-testing"> -<h3><a class="toc-backref" href="#id39">6.3.5 options for debugging, manual dependency rule creation and testing</a></h3> +<h2><a class="toc-backref" href="#contents">7.5 options for debugging, manual dependency rule creation and testing</a></h2> <dl class="docutils" id="description-dir"> <dt>DESCRIPTION_DIR</dt> <dd><p class="first">A directory where all description data read from an R package will be @@ -1709,10 +1749,11 @@ This will pass '--bwlimit=<value>' to all rsync commands.</p> </dl> </div> </div> -<div class="section" id="id2"> -<h2><a class="toc-backref" href="#id40">6.4 Field Definition</a></h2> -<p>The field definition file uses <a class="reference external" href="http://docs.python.org/library/configparser.html">ConfigParser</a> syntax. For an example, see -<a class="reference internal" href="#default-field-definition-file">default field definition file</a>.</p> +<div class="section" id="field-definition-config"> +<span id="id2"></span><h1><a class="toc-backref" href="#contents">8 Field Definition Config</a></h1> +<p>The field definition file uses <a class="reference external" href="http://docs.python.org/library/configparser.html">ConfigParser</a> syntax and defines +how an R package's DESCRIPTION file is read. +See the next section, <a class="reference internal" href="#default-field-definition-file">default field definition file</a>, for an example.</p> <p>Each information field has its own section which declares a set of options and flags. Flags are case-insensivitve options without a value - they're enabled by listing them.</p> @@ -1761,8 +1802,8 @@ The default behavior is to merge lines.</dd> </dl> <dl class="docutils" id="field-flag-islist"> <dt>isList</dt> -<dd>Declares that a field's value is a list whose values are separated by -by ',' or ';'.</dd> +<dd>Declares that a field's value is a list whose values are separated +by ',' and/or ';'.</dd> </dl> <dl class="docutils" id="field-flag-iswhitespacelist"> <dt>isWhitespaceList</dt> @@ -1781,19 +1822,16 @@ False (e.g. is an empty string).</dd> <dl class="docutils" id="field-flag-ignore"> <dt>ignore</dt> <dd>Declares that a field is known but entirely ignored. Unknown fields -are ignored, too, the main difference is the log message.</dd> +are ignored, too, the main difference is the emitted log message if +such a field is found.</dd> </dl> </blockquote> <div class="note"> <p class="first admonition-title">Note</p> <p class="last">It won't be checked whether a flag is known or not.</p> </div> -</div> -</div> -<div class="section" id="appendix"> -<h1><a class="toc-backref" href="#id41">7 Appendix</a></h1> -<div class="section" id="default-field-definition-file"> -<h2><a class="toc-backref" href="#id42">7.1 Default Field Definition File</a></h2> +<div class="section" id="example-the-default-field-definition-file"> +<span id="default-field-definition-file"></span><h2><a class="toc-backref" href="#contents">8.1 Example: The default field definition file</a></h2> <p>This is the default field definition file (without any ignored fields):</p> <pre class="code ini literal-block"> <span class="k">[Description]</span> @@ -1828,6 +1866,168 @@ are ignored, too, the main difference is the log message.</dd> </pre> </div> </div> +<div class="section" id="dependency-resolution-console"> +<span id="depres-console"></span><h1><a class="toc-backref" href="#contents">9 Dependency Resolution Console</a></h1> +<p>As previously stated, the <em>DepRes Console</em> is only meant for <strong>testing</strong>. +It's an interactive console with the following features:</p> +<ul class="simple"> +<li>resolve dependencies</li> +<li>create new dependency rules (<strong>only single line rules</strong>)</li> +<li>create dependency rules for each R package found in a directory</li> +<li>load rules from files</li> +<li>save rules to a file</li> +</ul> +<p>Rules are managed in a set. These so-called <em>rule pools</em> are organized in +a <em>first-in-first-out</em> data structure that allows +to create or remove them easily at runtime.</p> +<p>Running <tt class="docutils literal">roverlay depres_console</tt> will print a welcome message that +lists all available commands and a few usage hints.</p> +<p>For reference, these commands are currently available:</p> +<table border="1" class="docutils"> +<colgroup> +<col width="29%" /> +<col width="71%" /> +</colgroup> +<thead valign="bottom"> +<tr><th class="head">command</th> +<th class="head">description</th> +</tr> +</thead> +<tbody valign="top"> +<tr><td>help, +h</td> +<td>lists all commands</td> +</tr> +<tr><td>help --list, +h --list</td> +<td>lists all help topics for which help is available</td> +</tr> +<tr><td>help <em><cmd></em>, +h <em><cmd></em></td> +<td>prints a command-specific help message</td> +</tr> +<tr><td>load <em><file|dir></em>, +l <em><file|dir></em></td> +<td>loads a rule file or a directory with rule files +into a new <em>rule pool</em></td> +</tr> +<tr><td>load_conf, +lc</td> +<td>loads the rule files listed in the config file +into a new <em>rule pool</em></td> +</tr> +<tr><td>addrule <em><rule></em> ++ <em><rule></em></td> +<td>creates a new rule and adds it to the topmost, +i.e. latest <em>rule pool</em>. This command uses +<a class="reference internal" href="#rule-file-syntax">Rule File Syntax</a>, but multi line rules are +not supported.</td> +</tr> +<tr><td>add_pool, +<<</td> +<td>creates a new <em>rule pool</em></td> +</tr> +<tr><td>unwind, +>></td> +<td>removes the topmost <em>rule pool</em> and all of its +rules</td> +</tr> +<tr><td>resolve <em><dep></em>, +? <em><dep></em></td> +<td>tries to resolve the given dependency string and +prints the result</td> +</tr> +<tr><td>print, p</td> +<td>prints the rules of the topmost <em>rule pool</em></td> +</tr> +<tr><td>print all, p all</td> +<td>prints the rules of all <em>rule pools</em></td> +</tr> +<tr><td>write <em><file></em>, +w <em><file></em></td> +<td>writes the rules of the topmost <em>rule pool</em> into +<em><file></em></td> +</tr> +<tr><td>cd <em><dir></em></td> +<td>changes the working directory, also creates it if +necessary</td> +</tr> +<tr><td>scandir <em><dir></em>, +sd <em><dir></em></td> +<td>creates dependency rules for each R package found +in <em><dir></em></td> +</tr> +<tr><td>set, unset</td> +<td>prints the status of currently (in)active modes</td> +</tr> +<tr><td>set <em><mode></em>, +unset <em><mode></em></td> +<td>sets or unsets <em><mode></em>. There's only one mode to +control, the <em>shlex mode</em> which controls how +command arguments are parsed</td> +</tr> +<tr><td>mkhelp</td> +<td>verifies that each accessible command has a help +message</td> +</tr> +<tr><td>exit, qq, q</td> +<td>exits the <em>DepRes Console</em></td> +</tr> +</tbody> +</table> +<dl class="docutils"> +<dt>Example Session:</dt> +<dd><pre class="code first last literal-block"> +== depres console == +Run 'help' to list all known commands +More specifically, 'help <cmd>' prints a help message for the given +command, and 'help --list' lists all help topics available +Use 'load_conf' or 'lc' to load the configured rule files + +commands: load, unwind, set, help, >>, load_conf, <<, cd, mkhelp, +resolve, lc, add_pool, addrule, h, +, l, li, write, p, r, ?, w, print, +sd, unset, scandir +exit-commands: q, qq, exit + +cmd % + ~dev-lang/R :: R language +new rules: +~dev-lang/R :: R language +--- --- +command succeeded. + +cmd % ? R language +Trying to resolve ('R language',). +Resolved as: ('dev-lang/R',) + +cmd % ? R language [ 2.15 ] +Trying to resolve ('R language [ 2.15 ]',). +Resolved as: ('>=dev-lang/R-2.15',) + +cmd % ? R +Trying to resolve ('R',). +Channel returned None. At least one dep couldn't be resolved. + +cmd % p +~dev-lang/R :: R language + +cmd % >> +Pool removed from resolver. + +cmd % p + +cmd % ? R language +Trying to resolve ('R language',). +Channel returned None. At least one dep couldn't be resolved. + +cmd % exit +</pre> +</dd> +</dl> +</div> +<div class="section" id="implementation-overview"> +<h1><a class="toc-backref" href="#contents">10 Implementation Overview</a></h1> +<p id="dynamic-selfdep-rule-pool"><span id="dependency-resolution"></span><<work in progress; not part of the html doc yet>></p> +</div> </div> </body> </html> |