| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
All repo configuration is currently removed unconditionally. Gentoo
itself doesn't really need this, but derivatives probably do.
Closes: https://github.com/gentoo/catalyst/pull/16
Signed-off-by: James Le Cuirot <chewi@gentoo.org>
|
|
|
|
| |
Signed-off-by: Ben Kohler <bkohler@gentoo.org>
|
|
|
|
|
|
|
| |
This is functionality better implemented in fsscripts outside of
catalyst.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
The Python community is coalescing around tomli, and tomli is likely to
be integrated into the standard library per PEP680.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
checksum format is simple one, identical to one
sha256sum from coreutils produces, lines starting with # are ignored.
example:[1]
# SHA256 HASH
xxxx..... stage3-....tar.xz
systemd upstream calls it suse-style .sha256 files.[0]
infra already supports inline signing of files.
Bug: https://bugs.gentoo.org/821568
[0] https://github.com/systemd/systemd/blob/aedec452b9e5dd197881f2164fb205dfe8bfdcec/src/import/pull-common.c#L236
[1] https://mirrors.edge.kernel.org/opensuse/distribution/leap/15.0/iso/openSUSE-Leap-15.0-DVD-x86_64.iso.sha256
Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
|
|
|
|
|
|
|
| |
We don't use ISOLINUX since the commit listed below.
Fixes: 0a27a7a3 ("targets: Use GRUB for BIOS boot")
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit renames the config option 'portage_overlay' to 'repos'.
Also, it renames the method 'portage_overlay' to 'process_repos'
and 'set_portage_overlay' to 'set_repos'. Motivation:
1) The Gentoo wiki states 'repository' as the preferred term,
and 'overlay' only as a colloquial
(https://wiki.gentoo.org/wiki/Ebuild_repository),
reducing the distinction between the main repository
and other 'overlay' repositories.
3) This avoids confusion with filesystem overlays
(via the config option 'overlay' and 'root_overlay').
2) The plural form highlights the possibility of specifying
multiple repositories, continuing 2861038a.
Signed-off-by: Felix Bier <felix.bier@rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
This commit renames the config option 'repos' to 'repos_storedir'.
This is done as preparation for renaming 'portage_overlay' to 'repos'.
Signed-off-by: Felix Bier <felix.bier@rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
With --enter-chroot, after the mounts and environment are set up,
catalyst will drop you into a shell inside the chroot. Useful for
hacking or debugging.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
The code already supports multiple repositories. This commit adapts the
documentation to make this more explicit.
Signed-off-by: Felix Bier <felix.bier@rohde-schwarz.com>
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
distcc_hosts are independent of the build itself, and therefore should
be configured system-wide in catalyst.conf and not in each spec file.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We currently have two mechanisms of setting MAKEOPTS: in spec files and
in catalystrc.
Setting makeopts in spec files doesn't make sense. The spec should
describe the thing that's being built and not contain options that are
specific to the build system.
Setting makeopts via catalystrc is better, but it only applies to the
actual build system invocations, leaving emerge to run jobs serially or
again requiring configuration specific to the build machine to be put
into the spec file. For example:
update_seed_command: ... --jobs 5 --load-average 5
With jobs and load-average specified in catalyst.conf, catalyst has the
information required to configure both emerge and the build systems
emerge executes.
This removes the undocumented makeopts spec file option and replaces it
with jobs and load-average settings in catalyst.conf.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
grub-mkrescue produces an ISO that is bootable from EFI as well as from
the BIOS, so isolinux isn't necessary to accomplish that. The benefits
are that (1) we reduce the number of dependencies required to make the
ISO, (2) the same GRUB set up works for EFI and BIOS systems, and (3)
grub-mkrescue replaces a whole lot of code that we no longer need to
maintain.
This patch has the side effect of removing the F[2-7] help texts and the
memtest86+ option, since those were done via isolinux. I feel okay
removing those since isolinux was only used on BIOS systems and not on
EFI, so none of those things were available to the vast majority of our
users.
It also has the side-effect of removing Fedora's signed UEFI shim
(sys-boot/shim) which could be used to add a hash for the live CD's grub
binary so that you could install under Secure Boot. That doesn't offer
any protection, and the author of the code indicated to me that the
value in offering this feature is that it allows installation on systems
with broken firmware under which important hardware (like the NVMe
drive) isn't exposed unless using Secure Boot. After quite a bit of
searching the internet, I can't find any mentions of this being a
problem.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
| |
This confusingly named option allowed you to specify and entire spec
file on the command line. It seems that the addition of the --snapshot /
-s option in commit ac746eff5363 (new -s option for creating snapshots)
in 2004 removed all known uses, so let's remove it.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
Depends on media-gfx/splashutils which was removed from Gentoo in
November 2018.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
It made sense to have this as a separate section when there were
multiple configuration options (digests and hash_function) but
hash_function is now gone.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
| |
There are very few EAPI < 5 ebuilds left.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
| |
Fixes: c7688c666567 (doc/catalyst-config.5.txt: Add man page for catalyst.conf)
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This removes support for various file systems from the embedded target
(a target for producing images for embedded systems, as far as I
understand) and for ISOs. For ISOs, squashfs is great and everyone uses
it -- it provides better performance from a CD than the alternatives
like zisofs, normal, or noloop. See [1] for performance data of squashfs
vs other methods.
For embedded, it's unclear whether the target is used at all. There are
some very old specs in releng.git that use 'rel_type: embedded', but I'm
not sure if the target is used at all these days. To that end, I've
asked in #gentoo-embedded if anyone uses it. I've removed what I believe
to be the file system options that don't provide any value, leaving
jffs2 for now.
[1] https://elinux.org/Squash_Fs_Comparisons
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
We're using tar2sqfs from squashfs-tools-ng, so let's replace the usage
of mksquashfs (from squashfs-tools) with gensquashfs.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
| |
Its purpose was very confused.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
With sqfs snapshots, there's now no dependence on the build system's
portdir.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There were a number of problems with catalyst's snapshot system. It was
built around using the build system's portdir and had no control over
what was in that portdir or when it was updated.
As a result, when a stage build failed, it was difficult to tell what
the snapshot consistet of precisely or whether it contained a particular
recent fix.
With snapcache disabled, ebuild repo snapshots were tar'd and compressed
and then unpacked into the stage chroot which is an unnecessarily
expensive process. Moreover, a porttree has more than 100k small files,
which are stored extremely inefficiently on most file systems—a whole
porttree is usually around 700M on disk. Just removing all of those
files during the cleaning stage is an expensive operation.
Instead, we make a compressed squashfs image and mount it in the build
chroot. The porttree has many duplicate files, and squashfs deduplicates
the files and then compresses, so the result is very efficiently packed:
~38M with gzip -9 compression.
The snapshot target has been modified to generate a squashfs image from
a bare ebuild git repo. Piping git-archive to tar2sqfs generates the
squashfs image in less than 10 seconds on a modern system. The git repo
is fetched with --depth=1 to minize bandwidth and disk usage, and git gc
is run after fetch to minimize disk usage. Storage requirements for the
git ebuild repo with metadata are ~70M.
The squashfs snapshot is stored in /var/tmp/catalyst/snapshots/ by
default with a name <repo_name>-<git sha1>.sqfs. With this convention,
we know the exact point in history that the snapshot was taken. The
catalyst-auto script can use the sha1 to get a deterministic timestamp,
so that it is independent on when `catalyst -s` was run, but is instead
the timestamp of the commit date of the repo's git SHA1.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
Among the problems with snapcache were broken locking, and still needing
to pack and unpack snapshot tarballs. In a few commits, I'm going to
replace it with something significantly better.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
| |
Leave the machinery in place, but remove the default settings from the
configuration file, since basically no one is going to need to change
these and in practice they just cover up mistakes in defaults.py.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Other algorithms give better compression ratios, but the difference is
not meaningful for a 2MiB text file. In my testing bzip2 gave a better
compression ratio of 15:1 vs gzip's 11:1, but that ends up being only a
size difference of 50KiB (136 vs 187) which is only an additional 2.5%
savings from the uncompressed input.
Choose gzip because transparent decompression is widely supported by web
servers and clients.
Closes: https://bugs.gentoo.org/630284
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
| |
Switch to SHA1, which is plenty fast. The next commit switches from the
external shash/b2sum tools to Python's hashlib, and the removal of this
config option will simplify that since the crc32 hash is in a different
python module (zlib).
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This option seems completely absurd and on top of that has been
completely broken, since contents() has been passed 'j', which is the
mode, as the positional argument 'destination'. This only worked because
destination is not used in the contents generation and the actual mode
parameter has a default argument of 'auto'.
So the config option has not worked in a long time and no one has
noticed.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
The only change I reverted was from the reformatting of the
HASH_DEFINITIONS table in catalyst/hash_utils.py.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
Evidently this was a target last used 15 years ago to produce a Gentoo
LiveCD ISO with *a* game on it.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
spind was deleted from livecd-tools in 2011.
See: https://gitweb.gentoo.org/proj/livecd-tools.git/commit/?id=7d4d363bae7313e46de960a45888907059b0f2d6
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
| |
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
Unused, and other successful tinderbox systems exist and are in active
use.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
| |
Unused for 12 years.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
| |
Catalyst has been python 3-only for a while.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
| |
They're not useful and out of date to boot.
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
| |
GRUB replaced yaboot on PPC in commit e9c0a0394d15 (ppc: Generate Grub
entries)
Signed-off-by: Matt Turner <mattst88@gentoo.org>
|
|
|
|
|
|
|
|
|
|
|
| |
The locations approved by the council:
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
PORTDIR="/var/db/repos/gentoo"
Bug: https://bugs.gentoo.org/684908
Signed-off-by: Zac Medico <zmedico@gentoo.org>
|
| |
|
|
|
|
|
| |
per report by hanetzer on irc, this shebang work for prefix installs and
per floppym it is valid and correct
|
|
|
|
| |
allow livecd to keep build deps or just not run depclean at all
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The generation of cachedirs up to this point includes the version_stamp
from the specfile.
If the version is being automatically incremented (eg daily builds),
then it generates unique cache directories for each build, and doesn't
reuse old cache.
Example versioned cache directories:
/release/buildroot/amd64-dev/packages/default/stage1-amd64-20171203
/release/buildroot/amd64-dev/packages/default/stage1-amd64-20171204
/release/buildroot/amd64-dev/packages/default/stage1-amd64-20171206
This should be using a common cache directory:
/release/buildroot/amd64-dev/packages/default/stage1-amd64
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Brian Dolbec <dolsen@gentoo.org>
|
| |
|