The five DinD lab nodes run privileged with cgroupns=host, a writable host cgroupfs and seccomp/apparmor off, and carried no resource limit of any kind. A runaway node therefore competes with the Actions runner agent itself, and the platform reports that only as 'lost communication', never as an OOM.
Only pids_limit for now. A memory cap needs a measured per-node peak, and every measurement available is a failure-time snapshot, i.e. a lower bound: 4.4 GiB on the busiest node of a variant-0 drill. Sizing the cap off that risks OOM-killing a legitimate container instead of the runaway node. The pid ceiling has no such problem: 8192 against a measured 981 is 8.3x.
The value is a static default.env key on purpose. A handler-derived key is absent from .env on a fresh runner, and the topology export step runs before the install step that generates it, so compose interpolation would have failed every swarm job before anything deployed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The LUKS step of the DR drill died on `The file /dev/mapper/usbdrill does
not exist and no size was specified.` -- from mkfs, not from cryptsetup,
because luksOpen had exited 0. The node did have a udev, and it had
created the mapper entry: as a symlink to ../dm-1. Modern udev no longer
mknods, it only manages symlinks, and the /dev/dm-N target it points at is
a devtmpfs node that a container's tmpfs /dev never receives. Without a
udev libdevmapper creates the node itself, which is why the debian nodes
passed -- their image ships no udevd at all, while arch cannot install
systemd without one.
Masking udevd in the node image removes the whole class rather than one
call site. DM_DISABLE_UDEV rides on the node containers instead of the
script because the drill opens the volume a second time through the
recover CLI in a separate docker exec, which would not inherit an export.
The block-device check then fails loudly with both prerequisites named,
and closes the mapping first so an abort cannot leak it onto the runner --
device-mapper is not namespaced and every distro round shares the VM.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every DinD node is attached to a second network by the bootstrap cache connect, so the embedded DNS returns two A records per node name in undefined order; workers that resolved the manager onto the cache network could not reach the registry bound to the lab IP and rejected locally built images with 'No such image', pushing all replicas onto the manager. Pin a fixed lab ipv4_address per node and every node name via extra_hosts (files-first resolution wins over DNS); the subnet and IPs live in default.env as the INFINITO_SWARM_* SPOT consumed by compose interpolation and the topology exports. Proven by A/B: the identical friendica variant-2 swarm job went from worker rejects to fully green including the drain test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Node image now builds FROM the distro infinito image (python, ansible, CLI baked in) instead of installing a per-node .deb, so there is no per-node bootstrap or timeout; the swarm CI job depends on the distribution build. Move compose/swarm.yml to compose/swarm/compose.yml and relocate the swarm matrix orchestrator from cli.administration.deploy.swarm to utils.tests.swarm (test code out of the shipped CLI).
Back the NFS export root with a disk-backed named volume instead of a RUNNER_TEMP bind: local-act RUNNER_TEMP resolves onto tmpfs and ganesha FSAL_VFS cannot export a tmpfs, so it loaded zero exports and the controller mount got ENOENT. Add optional compose/swarm/cache.override.yml wiring swarm nodes to the package-cache frontend when present, leaving CI without a cache unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5