mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
feat(images): carry the zfs userland on all distros
dockerd's zfs graphdriver resolves `zfs` on PATH and opens /dev/zfs before it
accepts a data root (moby daemon/graphdriver/zfs/zfs.go:54,59), so the userland
has to sit in the image, not on the host. Only the apt images carried it, which
is why resolve.sh's userland_for() kept zfs out of every entry containing arch,
centos or fedora - that is every automatic swarm sweep, so the zfs data root and
the zfs branch of the backup launcher were never once exercised. Run
30781723102 shows it: "zfs left out: not every distro of this entry carries the
userland".
scripts/install/zfs.sh joins the three existing install scripts in the image
build and takes a different route per distro, each one run in a real container
of the matching base image before landing here:
debian trixie 2.3.2 zfsutils-linux, contrib enabled first
ubuntu 26.04 2.4.1 zfsutils-linux
arch 2.4.3 the AUR PKGBUILD, which configures --with-config=user
fedora 44 2.4.3 zfsonlinux repo plus a dummy zfs-kmod provider
centos 10 2.2.10 the same, from the epel path
The dummy is what keeps the RPM route usable: the `zfs` package carries an
ungated `Requires: %{name}-kmod`, so a plain `dnf install zfs` resolves through
zfs-dkms and installs 389 extra packages and 1.44 GB of kernel and toolchain to
build a module against a kernel the container never boots. Providing zfs-kmod
from an empty package keeps the same install at 60 packages with no kernel
package at all, and the transaction is checked afterwards so a regression fails
the build instead of bloating the image. The repository is removed again once
the userland is in, because the dummy pins one version and an upgrade would
otherwise re-resolve the kmod at deploy time.
Version drift against the host module is not a concern: zpool create enables a
feature only when the loaded module reports it through /sys/module/zfs/features
.pool (cmd/zpool/zpool_main.c, module/zcommon/zfeature_common.c), and libzfs
compares no versions at all (lib/libzfs/os/linux/libzfs_util_os.c:123-134), so a
newer userland degrades rather than breaking.
With every image carrying it, userland_for() and every_distro_carries() have
nothing left to decide and go. Two defects that only became reachable once zfs
could be drawn are fixed in the same move:
* /dev/zfs is a misc device, so the old lookup in /proc/devices could never
find its major and the node was never created. A node container's /dev is
populated at creation and udevd is masked, so nothing else fills the gap
either: the first bring-up of every job would have declined, blaming a
module that modprobe had just loaded. The minor now comes from /proc/misc.
* Pools live in the kernel the lab nodes share with the runner, while the
teardown only removes containers, and the node hostnames repeat across the
distro sweep. From the second distro on, zpool create hit the previous
iteration's pool. It is destroyed and its loop detached first.
The node image drops its own apt-only zfs branch and instead masks the six boot
units the userland enables by preset, which on a node can only fail: the pool is
created after boot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
65d2aa028e
commit
ed1121053b
10 files changed
+285
-50
No files matched your search
+3
-2
@@ -30,10 +30,11 @@ COPY . ${INFINITO_SRC_DIR}
|
||||
|
||||
# hadolint ignore=DL3008,DL3033,DL3041,SC1090
|
||||
RUN set -euo pipefail; \
|
||||
source <(grep -hE '^INFINITO_(PYTHON|DOCKER_CLI|PACKAGE)_INSTALL_SCRIPT=' "${INFINITO_SRC_DIR}/default.env"); \
|
||||
source <(grep -hE '^INFINITO_(PYTHON|DOCKER_CLI|PACKAGE|ZFS)_INSTALL_SCRIPT=' "${INFINITO_SRC_DIR}/default.env"); \
|
||||
/bin/bash "${INFINITO_SRC_DIR}/${INFINITO_PYTHON_INSTALL_SCRIPT:?}"; \
|
||||
/bin/bash "${INFINITO_SRC_DIR}/${INFINITO_DOCKER_CLI_INSTALL_SCRIPT:?}"; \
|
||||
/bin/bash "${INFINITO_SRC_DIR}/${INFINITO_PACKAGE_INSTALL_SCRIPT:?}"
|
||||
/bin/bash "${INFINITO_SRC_DIR}/${INFINITO_PACKAGE_INSTALL_SCRIPT:?}"; \
|
||||
/bin/bash "${INFINITO_SRC_DIR}/${INFINITO_ZFS_INSTALL_SCRIPT:?}"
|
||||
|
||||
RUN set -euo pipefail; \
|
||||
systemctl mask systemd-firstboot.service first-boot-complete.target || true; \
|
||||
|
||||
@@ -17,6 +17,12 @@
|
||||
# AppArmor profile then denies the setuid helper CAP_DAC_READ_SEARCH and every
|
||||
# PAM account lookup fails; a hard 0600 instead of u+rw breaks debian's setgid
|
||||
# helper.
|
||||
#
|
||||
# The zfs userland arrives with the parent image and enables six units by
|
||||
# preset. All of them run before scripts/tests/deploy/utils/filesystem/
|
||||
# docker_dataroot.sh creates the pool, so on a node they can only fail and
|
||||
# leave systemd degraded; they are masked here rather than in the parent, which
|
||||
# a real host also boots.
|
||||
ARG INFINITO_IMAGE
|
||||
FROM ${INFINITO_IMAGE}
|
||||
|
||||
@@ -24,13 +30,6 @@ RUN set -eux; \
|
||||
if command -v apt-get >/dev/null 2>&1; then \
|
||||
apt-get update -qq; \
|
||||
apt-get install -y -qq --no-install-recommends dnsmasq openssh-server cryptsetup e2fsprogs elfutils kmod; \
|
||||
. /etc/os-release; \
|
||||
if [ "${ID:-}" = debian ] && [ -n "${VERSION_CODENAME:-}" ]; then \
|
||||
echo "deb http://deb.debian.org/debian ${VERSION_CODENAME} contrib" >/etc/apt/sources.list.d/contrib.list; \
|
||||
apt-get update -qq || rm -f /etc/apt/sources.list.d/contrib.list; \
|
||||
fi; \
|
||||
apt-get install -y -qq --no-install-recommends zfsutils-linux || \
|
||||
echo "zfsutils-linux unavailable; a zfs data-root pick will decline on this image" >&2; \
|
||||
rm -rf /var/lib/apt/lists/*; \
|
||||
elif command -v dnf >/dev/null 2>&1; then \
|
||||
dnf install -y dnsmasq openssh-server cryptsetup e2fsprogs elfutils kmod; \
|
||||
@@ -45,6 +44,7 @@ RUN set -eux; \
|
||||
systemctl mask systemd-resolved; \
|
||||
systemctl mask systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket systemd-udevd-varlink.socket systemd-udev-trigger; \
|
||||
systemctl mask modprobe@.service; \
|
||||
systemctl mask zfs-import-cache zfs-import-scan zfs-mount zfs-share zfs-volume-wait zfs-zed; \
|
||||
systemctl enable ssh 2>/dev/null || systemctl enable sshd; \
|
||||
chmod u+rw /etc/shadow; \
|
||||
if [ -e /etc/gshadow ]; then chmod u+rw /etc/gshadow; fi; \
|
||||
|
||||
@@ -324,3 +324,6 @@ INFINITO_WHITELIST=
|
||||
|
||||
# Master toggle for the worker SPOT chain (INFINITO_WORKER_CPU / _FETCH); falsy values: 0/false/no/off.
|
||||
INFINITO_WORKER_ENABLED=true
|
||||
|
||||
# Repo-relative path of the ZFS userland install script.
|
||||
INFINITO_ZFS_INSTALL_SCRIPT=scripts/install/zfs.sh
|
||||
@@ -6,6 +6,7 @@ This directory contains scripts that install and bootstrap host-side tooling use
|
||||
- 🐧 Install ad-hoc APT packages for CI and workflow jobs via `apt.sh`
|
||||
- 📦 Build and install distro package metadata via `package.sh` (Arch/Debian/Ubuntu/Fedora)
|
||||
- 🏖️ Install OS-level Claude Code sandbox dependencies (`bubblewrap`, `socat`) via `sandbox.sh`, covering Debian, Ubuntu, Fedora, CentOS/RHEL/Rocky/Alma, and Arch; also exposed as `make install-agent`
|
||||
- 🧊 Install the OpenZFS userland via `zfs.sh` (apt on Debian/Ubuntu, the AUR PKGBUILD on Arch, the zfsonlinux repository on Fedora/CentOS); the kernel module comes from the host
|
||||
- 🐍 Bootstrap virtual environments and editable project dependencies
|
||||
- 🛠️ Keep host preparation separate from lint, build, and test execution
|
||||
|
||||
|
||||
Executable
+213
@@ -0,0 +1,213 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bake the OpenZFS userland into the distro image. The kernel module always
|
||||
# comes from the host - dockerd's zfs graphdriver only needs the `zfs` binary on
|
||||
# PATH and an openable /dev/zfs, so a pool the host prepared is unusable without
|
||||
# these packages inside the container.
|
||||
#
|
||||
# Three trip-wires this script exists to route around:
|
||||
#
|
||||
# * The OpenZFS `zfs` RPM carries an ungated `Requires: %{name}-kmod`, so a
|
||||
# plain `dnf install zfs` resolves through zfs-dkms and drags in
|
||||
# kernel-devel, kernel-debug-core and a DKMS build against a kernel the
|
||||
# container never boots: measured at 389 extra packages and 1.44 GB. A
|
||||
# dummy package that only *provides* zfs-kmod keeps the same install at 60
|
||||
# packages with no kernel package at all. Its version must equal the exact
|
||||
# %{version} dnf resolves for `zfs`; a mismatch silently falls back to DKMS.
|
||||
# Because that version is pinned, the repository is removed again once the
|
||||
# userland is in: leaving it enabled means the next `dnf upgrade` finds a
|
||||
# newer zfs whose kmod dependency the dummy no longer satisfies, and the
|
||||
# whole DKMS pull returns at deploy time rather than at build time.
|
||||
# * Debian keeps zfsutils-linux in contrib, which the base image does not
|
||||
# enable. Without the extra source apt reports no installation candidate.
|
||||
# * Arch has no zfs userland in core/extra. archzfs publishes a prebuilt
|
||||
# zfs-utils, but only on a rolling release tag whose asset filename carries
|
||||
# the version, so any pinned URL 404s on the next upstream bump. Building
|
||||
# the AUR PKGBUILD (which configures --with-config=user, userland only) is
|
||||
# the reproducible route and costs about six minutes. That build runs with
|
||||
# --skippgpcheck rather than --skipinteg: the release tarball's sha256 and
|
||||
# b2 sums stay verified, and only the detached OpenZFS signature is skipped,
|
||||
# because checking it means carrying the release key in the builder's
|
||||
# keyring while the same PKGBUILD already supplies both the sums and the
|
||||
# list of accepted keys, so the signature adds no trust the sums do not.
|
||||
set -euo pipefail
|
||||
|
||||
RETRIES=3
|
||||
AUR_BUILDER=pkgbuild
|
||||
ZFS_RELEASE_FEDORA="3-1 3-0 2-8"
|
||||
ZFS_RELEASE_EL="3-0 2-8 2-3"
|
||||
|
||||
log() { echo "[zfs] $*"; }
|
||||
|
||||
# Param: $@ command to run, retried on failure with a widening backoff
|
||||
retry() {
|
||||
local attempt=1
|
||||
until "$@"; do
|
||||
if [ "${attempt}" -ge "${RETRIES}" ]; then
|
||||
log "ERROR: '$*' failed after ${attempt} attempts"
|
||||
return 1
|
||||
fi
|
||||
log "attempt ${attempt} of '$1' failed; retrying"
|
||||
sleep $((attempt * 5))
|
||||
attempt=$((attempt + 1))
|
||||
done
|
||||
}
|
||||
|
||||
install_apt() {
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
local apt_opts=(-o Acquire::Retries="${RETRIES}" -qq)
|
||||
|
||||
if [ "${ID:-}" = debian ]; then
|
||||
log "enabling contrib for ${VERSION_CODENAME:?}, which is where zfsutils-linux lives"
|
||||
echo "deb http://deb.debian.org/debian ${VERSION_CODENAME} contrib" \
|
||||
>/etc/apt/sources.list.d/contrib.list
|
||||
fi
|
||||
|
||||
retry apt-get "${apt_opts[@]}" update
|
||||
retry apt-get "${apt_opts[@]}" install -y --no-install-recommends zfsutils-linux
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
}
|
||||
|
||||
install_pacman() {
|
||||
retry pacman -Sy --noconfirm --needed base-devel git
|
||||
|
||||
if ! id "${AUR_BUILDER}" >/dev/null 2>&1; then
|
||||
useradd -m -s /bin/bash "${AUR_BUILDER}"
|
||||
fi
|
||||
install -d -m 0755 /etc/sudoers.d
|
||||
printf '%s ALL=(ALL) NOPASSWD: /usr/bin/pacman\n' "${AUR_BUILDER}" \
|
||||
>/etc/sudoers.d/99-"${AUR_BUILDER}"-pacman
|
||||
chmod 0440 /etc/sudoers.d/99-"${AUR_BUILDER}"-pacman
|
||||
|
||||
local build_root=/tmp/zfs-utils-build
|
||||
rm -rf "${build_root}"
|
||||
install -d -o "${AUR_BUILDER}" -g "${AUR_BUILDER}" "${build_root}"
|
||||
|
||||
retry su "${AUR_BUILDER}" -c \
|
||||
"git clone --depth 1 https://aur.archlinux.org/zfs-utils.git '${build_root}'"
|
||||
su "${AUR_BUILDER}" -c \
|
||||
"cd '${build_root}' && makepkg --syncdeps --noconfirm --cleanbuild --skippgpcheck"
|
||||
|
||||
local pkg
|
||||
pkg="$(find "${build_root}" -maxdepth 1 -type f -name 'zfs-utils-*.pkg.tar.*' \
|
||||
! -name '*-debug-*' | head -n1)"
|
||||
if [ -z "${pkg}" ]; then
|
||||
log "ERROR: makepkg produced no zfs-utils package"
|
||||
return 1
|
||||
fi
|
||||
|
||||
pacman -U --noconfirm "${pkg}"
|
||||
rm -rf "${build_root}" /var/cache/pacman/pkg
|
||||
}
|
||||
|
||||
# Param: $1 path segment of the zfsonlinux repository, `fedora` or `epel`
|
||||
# Param: $2 space-separated zfs-release versions to try, newest first
|
||||
release_install() {
|
||||
local base="$1" candidates="$2" dist version url
|
||||
dist="$(rpm --eval '%{dist}')"
|
||||
for version in ${candidates}; do
|
||||
url="https://zfsonlinux.org/${base}/zfs-release-${version}${dist}.noarch.rpm"
|
||||
if dnf install -y "${url}"; then
|
||||
log "zfs-release: ${url}"
|
||||
return 0
|
||||
fi
|
||||
log "no zfs-release ${version} for ${dist}"
|
||||
done
|
||||
log "ERROR: none of the zfs-release versions '${candidates}' exists for ${dist}"
|
||||
return 1
|
||||
}
|
||||
|
||||
install_dnf() {
|
||||
case "${ID:-}" in
|
||||
fedora) retry release_install fedora "${ZFS_RELEASE_FEDORA}" ;;
|
||||
*) retry release_install epel "${ZFS_RELEASE_EL}" ;;
|
||||
esac
|
||||
|
||||
local zfs_version
|
||||
zfs_version="$(dnf --quiet repoquery --qf '%{version}\n' zfs | sort -V | tail -n1)"
|
||||
if [ -z "${zfs_version}" ]; then
|
||||
log "ERROR: the zfs-release repository offers no zfs package"
|
||||
return 1
|
||||
fi
|
||||
log "zfs userland offered by the repository: ${zfs_version}"
|
||||
|
||||
if ! command -v rpmbuild >/dev/null 2>&1; then
|
||||
retry dnf install -y rpm-build
|
||||
fi
|
||||
|
||||
local topdir=/tmp/zfs-kmod-container
|
||||
rm -rf "${topdir}"
|
||||
mkdir -p "${topdir}/SPECS"
|
||||
cat >"${topdir}/SPECS/zfs-kmod-container.spec" <<SPEC
|
||||
Name: zfs-kmod-container
|
||||
Version: ${zfs_version}
|
||||
Release: 1
|
||||
Summary: Satisfies the zfs kernel module dependency inside a container
|
||||
License: CDDL
|
||||
BuildArch: noarch
|
||||
Provides: zfs-kmod = ${zfs_version}
|
||||
|
||||
%description
|
||||
The container never loads a zfs module; the host supplies it and the container
|
||||
reaches it through /dev/zfs. This package exists only so the zfs userland
|
||||
resolves without pulling a DKMS toolchain and a kernel it will never boot.
|
||||
|
||||
%files
|
||||
SPEC
|
||||
|
||||
rpmbuild --define "_topdir ${topdir}" -bb "${topdir}/SPECS/zfs-kmod-container.spec"
|
||||
dnf install -y "${topdir}"/RPMS/noarch/zfs-kmod-container-*.rpm
|
||||
retry dnf install -y zfs
|
||||
|
||||
if rpm -qa 'kernel-devel*' 'kernel-debug*' 'zfs-dkms' | grep -q .; then
|
||||
log "ERROR: the transaction pulled kernel or DKMS packages, so the dummy did not take"
|
||||
rpm -qa 'kernel-devel*' 'kernel-debug*' 'zfs-dkms' >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
dnf remove -y zfs-release
|
||||
rm -rf "${topdir}"
|
||||
dnf clean all
|
||||
}
|
||||
|
||||
verify() {
|
||||
local tool
|
||||
for tool in zfs zpool; do
|
||||
if ! command -v "${tool}" >/dev/null 2>&1; then
|
||||
log "ERROR: ${tool} is missing after the install"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
|
||||
if ldd "$(command -v zpool)" | grep -q 'not found'; then
|
||||
log "ERROR: zpool has unresolved shared libraries"
|
||||
ldd "$(command -v zpool)" | grep 'not found' >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if command -v dnf >/dev/null 2>&1 && dnf repolist --enabled 2>/dev/null | grep -q '^zfs'; then
|
||||
log "ERROR: a zfs repository is still enabled, so an upgrade would re-resolve the kmod"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "installed $(command -v zfs) and $(command -v zpool); all libraries resolved"
|
||||
}
|
||||
|
||||
main() {
|
||||
# shellcheck source=/dev/null
|
||||
. /etc/os-release
|
||||
|
||||
if command -v apt-get >/dev/null 2>&1; then
|
||||
install_apt
|
||||
elif command -v pacman >/dev/null 2>&1; then
|
||||
install_pacman
|
||||
elif command -v dnf >/dev/null 2>&1; then
|
||||
install_dnf
|
||||
else
|
||||
log "ERROR: no supported package manager (apt/pacman/dnf)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
verify
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -11,7 +11,8 @@ bridge (`192.168.244.0/24`, MTU 1400): three swarm nodes (1 manager +
|
||||
2 workers), a non-swarm NFSv4 server serving the shared volume storage,
|
||||
and a non-swarm backup host for the DR drill. The node image
|
||||
([`compose/swarm/Dockerfile`](../../../../compose/swarm/Dockerfile))
|
||||
bakes python3 + dnsmasq; the containers and the lab network are declared
|
||||
bakes python3 + dnsmasq and masks the zfs boot units the parent image's
|
||||
userland enables; the containers and the lab network are declared
|
||||
in [`compose/swarm/compose.yml`](../../../../compose/swarm/compose.yml) (project
|
||||
`${SWARM_NAME}`, backup host behind the `drill` profile) and started by
|
||||
one `docker compose up` in `routine/01_bootstrap.sh`. Lab DNS is provisioned
|
||||
|
||||
@@ -23,6 +23,7 @@ IMAGE=/var/tmp/docker-fs.img
|
||||
POOL_HOST="$(hostname 2>/dev/null || echo unknown)"
|
||||
POOL="infinito_ci_$(printf '%s' "${POOL_HOST}" | tr -dc 'A-Za-z0-9_.:-')"
|
||||
DAEMON=/etc/docker/daemon.json
|
||||
MISC_MAJOR=10
|
||||
|
||||
report() { echo "docker-dataroot-filesystem: $*"; }
|
||||
|
||||
@@ -63,6 +64,28 @@ arm_autoclear() {
|
||||
report "WARNING: ${loop} did not arm autoclear (AUTOCLEAR='${armed}')"
|
||||
}
|
||||
|
||||
# Param: $1 pool whose leftovers from an earlier bring-up have to go
|
||||
#
|
||||
# Pools live in the kernel, which the lab nodes share with the runner and with
|
||||
# every later distro of the same sweep, while the container teardown only removes
|
||||
# containers. A pool of this name is therefore still imported when the next
|
||||
# iteration starts, and `zpool create` refuses the name rather than reusing it.
|
||||
# Its vdev is a loop device backed by an image the dead container already
|
||||
# unlinked, so that space stays claimed until the loop is detached as well.
|
||||
reclaim_stale_pool() {
|
||||
local pool="$1" vdev
|
||||
zpool list -H -o name "${pool}" >/dev/null 2>&1 || return 0
|
||||
vdev="$(zpool list -vHP "${pool}" | awk 'NR == 2 {print $1}')"
|
||||
report "destroying ${pool}, left imported on this kernel by an earlier bring-up"
|
||||
zpool destroy -f "${pool}"
|
||||
if [ -b "${vdev}" ]; then
|
||||
losetup -d "${vdev}"
|
||||
report "detached its vdev ${vdev}"
|
||||
else
|
||||
report "WARNING: its vdev '${vdev}' is no block device; that loop stays attached"
|
||||
fi
|
||||
}
|
||||
|
||||
decline() {
|
||||
if [ "${REQUIRED}" = true ]; then
|
||||
report "FAIL: ${FSTYPE} was stated but cannot be delivered: $*"
|
||||
@@ -135,11 +158,16 @@ zfs)
|
||||
if ! modprobe_out="$(modprobe zfs 2>&1)"; then
|
||||
report "modprobe zfs failed: ${modprobe_out}"
|
||||
fi
|
||||
zfs_minor="$(awk '$2 == "zfs" {print $1}' /proc/misc)"
|
||||
[ -n "${zfs_minor}" ] ||
|
||||
decline "the zfs kernel module is not loaded on the host: ${modprobe_out}"
|
||||
if [ ! -c /dev/zfs ]; then
|
||||
zfs_major="$(awk '$2 == "zfs" {print $1}' /proc/devices | head -n1)"
|
||||
[ -n "${zfs_major}" ] && mknod /dev/zfs c "${zfs_major}" 0
|
||||
report "the module is loaded but this /dev predates it; creating the node from /proc/misc"
|
||||
mknod /dev/zfs c "${MISC_MAJOR}" "${zfs_minor}"
|
||||
fi
|
||||
[ -c /dev/zfs ] || decline "the zfs kernel module is not loaded on the host: ${modprobe_out}"
|
||||
[ -c /dev/zfs ] ||
|
||||
decline "zfs holds misc minor ${zfs_minor} but /dev/zfs could not be created"
|
||||
reclaim_stale_pool "${POOL}"
|
||||
;;
|
||||
esac
|
||||
|
||||
|
||||
@@ -2,18 +2,21 @@
|
||||
# Resolve which filesystem one matrix entry runs its docker data root on and
|
||||
# record the decision for the steps that follow.
|
||||
#
|
||||
# A random pick draws from what this kernel serves, measured here, narrowed to
|
||||
# what every distro of the entry carries a userland for. A stated pick is
|
||||
# honoured even where it is unsupported, and the applying step then fails. The
|
||||
# choice goes to the step summary, since reproducing a red run means re-stating
|
||||
# it verbatim.
|
||||
# A random pick draws from what this kernel serves, measured here. Every distro
|
||||
# image carries the userland for all three, the zfs one baked by
|
||||
# scripts/install/zfs.sh, so the pool depends on the kernel alone. A stated pick
|
||||
# is honoured even where it is unsupported, and the applying step then fails.
|
||||
# The choice goes to the step summary, since reproducing a red run means
|
||||
# re-stating it verbatim.
|
||||
#
|
||||
# Arguments:
|
||||
# $1 STATED ext4 | btrfs | zfs; 'auto' or empty for a random pick
|
||||
# $2 LABEL matrix entry the pick belongs to, e.g. compose/web-app-gitea
|
||||
# $3 DISTROS space-separated distributions the entry deploys on
|
||||
# $4 SCOPE node every distro in DISTROS must carry the userland too
|
||||
# runner DISTROS is ignored; only the kernel decides
|
||||
# $3 DISTROS space-separated distributions the entry deploys on, recorded
|
||||
# with the decision so a red run can be restated verbatim
|
||||
# $4 SCOPE runner | node; names where the pick gets applied. The kernel
|
||||
# measured here is the runner's either way, because the node
|
||||
# containers share it.
|
||||
set -euo pipefail
|
||||
|
||||
STATED="${1:-}"
|
||||
@@ -44,35 +47,12 @@ kernel_serves() {
|
||||
return 1
|
||||
}
|
||||
|
||||
userland_for() {
|
||||
case "$1" in
|
||||
centos | arch | fedora) echo "ext4 btrfs" ;;
|
||||
*) echo "ext4 btrfs zfs" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
every_distro_carries() {
|
||||
local fs="$1" distro
|
||||
[ "${SCOPE}" = node ] || return 0
|
||||
for distro in ${DISTROS}; do
|
||||
case " $(userland_for "${distro}") " in
|
||||
*" ${fs} "*) ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
candidates() {
|
||||
local kept="" fs
|
||||
for fs in ${POOL}; do
|
||||
if ! kernel_serves "${fs}"; then
|
||||
continue
|
||||
fi
|
||||
if ! every_distro_carries "${fs}"; then
|
||||
note "${fs} left out: not every distro of this entry carries the userland"
|
||||
continue
|
||||
fi
|
||||
kept="${kept} ${fs}"
|
||||
done
|
||||
kept="${kept# }"
|
||||
@@ -91,7 +71,7 @@ else
|
||||
REQUIRED=false
|
||||
fi
|
||||
|
||||
echo "filesystem for ${LABEL}: ${PICKED} (${ORIGIN})"
|
||||
echo "filesystem for ${LABEL} on '${DISTROS}': ${PICKED} (${ORIGIN})"
|
||||
|
||||
if [ -n "${GITHUB_ENV:-}" ]; then
|
||||
{
|
||||
|
||||
@@ -10,7 +10,9 @@ one is not, a stated pick overrides the pool, and a pick is always drawn from a
|
||||
pool the target can serve.
|
||||
|
||||
The pool contents are policy, not contract, and are deliberately not asserted -
|
||||
the per-distro expectation is derived from the resolver's own answers.
|
||||
the expectation is derived from the resolver's own answers. What is asserted is
|
||||
that the pool depends on neither the distro set nor the scope, because every
|
||||
distro image carries all three userlands.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -122,10 +124,15 @@ class TestFilesystemResolve(unittest.TestCase):
|
||||
pool = resolve("", " ".join(ALL_DISTROS), scope).pool
|
||||
self.assertTrue(set(absent).isdisjoint(pool))
|
||||
|
||||
def test_the_node_pool_also_drops_what_a_distro_cannot_install(self) -> None:
|
||||
runner = set(resolve("", " ".join(ALL_DISTROS), "runner").pool)
|
||||
node = set(resolve("", " ".join(ALL_DISTROS), "node").pool)
|
||||
self.assertTrue(node.issubset(runner))
|
||||
def test_the_distro_set_does_not_narrow_the_pool(self) -> None:
|
||||
every = resolve("", " ".join(ALL_DISTROS), "node").pool
|
||||
for distro in ALL_DISTROS:
|
||||
self.assertEqual(resolve("", distro, "node").pool, every)
|
||||
|
||||
def test_both_scopes_draw_from_the_same_pool(self) -> None:
|
||||
runner = resolve("", " ".join(ALL_DISTROS), "runner").pool
|
||||
node = resolve("", " ".join(ALL_DISTROS), "node").pool
|
||||
self.assertEqual(node, runner)
|
||||
|
||||
def test_a_missing_scope_is_a_hard_error(self) -> None:
|
||||
proc = subprocess.run(
|
||||
|
||||
Vendored
+1
@@ -60,6 +60,7 @@ STATIC_KEYS = (
|
||||
"INFINITO_DOCKER_CLI_INSTALL_SCRIPT",
|
||||
"INFINITO_PACKAGE_INSTALL_SCRIPT",
|
||||
"INFINITO_PYTHON_INSTALL_SCRIPT",
|
||||
"INFINITO_ZFS_INSTALL_SCRIPT",
|
||||
"INFINITO_ALIAS_REPOSITORY",
|
||||
"INFINITO_FORK_REPOSITORY_NAME",
|
||||
"INFINITO_PLAYWRIGHT_KEEP",
|
||||
|
||||
Reference in new issue
Block a user