A role-level `placement: manager` pins every service of the role and keeps
every volume node-local, so a role needing one node-bound service had to give
up NFS for all of them. GitLab's gitaly is that case: its repositories volume
is node-local (`nfs: false`, gitaly does not support NFS) and a reschedule
left the data behind on the old node.
`deploy.yml.j2` now resolves `services.<service>.placement` first and keeps the
role-wide list as the fallback, so gitaly carries the constraint alone while
webservice, sidekiq, workhorse, shell, migrations and gitlabmcp stay
schedulable and `gitlab_shared`, `gitlab_uploads` and `gitlab_builds` keep
their NFS backing. Role placement still comes from the primary entry, which is
what drives that backing.
Two things had to come with it. The swarm drill decided that a role is
manager-pinned by grepping the whole `meta/services.yml`, so a per-service pin
would have skipped GitLab's drain and reschedule steps while six of its
services still run on workers; it now reads the primary entry. And
`resource_filter` returns a present-but-null value rather than the hard
default, which rendered `node.role == None`, a constraint no node satisfies.
`test_no_placement_manager.py` now inspects every `placement` value instead of
only `manager`, requires the justification for each, and rejects a value that
is not a swarm node role, since a misspelled one hangs the converge wait
rather than failing the deploy. Requirement 023 records the shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 🐳 Workspace (ubuntu:latest) job on main (run 38879556215, 1b36ef6120)
failed test-unit-python in the hook's second make test:
AssertionError: Regex didn't match: '^ localhost=ok/\d+ms$'
not found in ' localhost=ok/-1612931482679767779ms'
ubuntu:latest is Ubuntu 26.04, whose date is uutils coreutils. uutils
0.7.0 ignores the field width in %3N and prints all nine nanosecond
digits, so `date +%s%3N` yields a 19-digit value instead of epoch
milliseconds. Against main's sampler, uutils 0.7.0 date reports
localhost=ok/7081859ms, and an unpadded nanosecond value turns the
difference negative.
answer_once now takes both timestamps from /proc/uptime, which the
kernel prints with two decimals whatever date the image ships. The
sampler already reads /proc/net/udp, so it was Linux-only before. The
resolution is 10 ms; the same probe reports 10 to 20 ms with uutils
date on PATH.
The new test puts a date on PATH that does not understand %3N; main's
sampler dies on it, the fixed one does not call date for the latency.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Manjaro workspace job failed test-unit-ruby with "cannot load such
file -- minitest". Debian vendors minitest in libruby, but Arch ships it
as ruby-minitest and Fedora and CentOS Stream as rubygem-minitest, none
of which ruby pulls in. ensure_ruby_present now probes
`require "minitest"` and installs the distro package when the probe
fails, then fails loudly if minitest is still missing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Fedora and CentOS Stream workspace jobs failed TLS to pypi.org,
registry.npmjs.org and repo.packagist.org (CERTIFICATE_VERIFY_FAILED,
curl error 7). package-frontend/ca.sh put the frontend CA into the Arch
anchor dir /etc/ca-certificates/trust-source/anchors, which p11-kit on
the RedHat family never reads.
Probing for the anchor dir does not fix it: with-ca-trust.sh creates
both /etc/pki/ca-trust/source/anchors and
/etc/ca-certificates/trust-source/anchors on every update-ca-trust host,
so the probe in registry-ca.sh already picks the RedHat dir on a reused
Arch runner. Both scripts now install into both dirs, rebuild only when
one of them changed, and take CA_ROOT so unit tests run them against a
fake root.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The main merge joined two correct rules into one broken script. The
architecture gate republishes the highest version when :latest misses an
architecture, because the published install instructions pull the image
untagged and only a release of that version rewrites latest. The backfill
floor refuses any tag below v9.0.2, which cannot build from its own tree.
The gate returned before the floor was ever consulted, so an incomplete
:latest would have asked the backfill for a tag it cannot build. Both
paths now ask below_floor, which also replaces the comparison the loop
carried inline.
The picker's own test could not have caught it: its docker stub answers
tags without emitting a manifest, and architectures.sh reads an empty
manifest as every architecture missing, so the gate fired on every case
and printed the newest tag. The stub now serves the whole pool, which is
the state the floor cases assume anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
down_act_outer.sh removed every container named act-*, so starting or
releasing a swarm drill in one worktree killed the act container of a
drill another worktree had running.
workflow.sh now labels act's job containers with
infinito.worktree=${INFINITO_CONTAINER} through the default
ACT_CONTAINER_OPTIONS, and down_act_outer.sh removes only containers
carrying the own label. Both scripts refuse to run without
INFINITO_CONTAINER, so the filter never widens to every act container;
make exports BASH_ENV, which loads it from .env.
Worktrees without a custom.env share the default container name and
therefore still share one label among themselves.
Closes#933
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The loader copies INFINITO_DISTRO into a loader-local variable before it
compares the value with the one in .env. With INFINITO_DISTRO read directly in
that condition, `shellcheck -x scripts/meta/resolve/apps.sh` reported SC2153
for INFINITO_DISTROS, which apps.sh takes from the sourced .env; with the copy
it reports nothing, as on main.
The loader test checks that .env is unchanged with filecmp against the file it
was copied from, instead of reading it twice with Path.read_bytes, which
test_no_uncached_file_reads rejects.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The release backfill dispatches the highest version tag whose images are
missing. Every tag from v15.0.0 down to v9.0.2 has its images now, so since
2026-10-05 it picks v9.0.1 every day, and each of the nine runs failed in
"Build missing CI images / Image" at
`entry.sh: line 19: infinito: command not found`. The v9.0.1 tree cannot
build its own image; c680be12ea fixed that build in v9.0.2, and a tag cannot
take the fix.
highest_release_tag.sh now stops at v9.0.2: no tag below it is picked, so the
backfill reports nothing missing instead of retrying v9.0.1. v9.0.0 has the
same Dockerfile, Makefile and scripts/docker as v9.0.1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make lint runs install-lint once and then dispatches its fifteen targets
in parallel. scripts/lint/mermaid.sh provisioned chrome-headless-shell
inside that parallel phase with `npx puppeteer browsers install
--install-deps`, whose `apt-get satisfy` carries no lock timeout. In a
fresh stack container another target held the apt lock, the shared
libraries stayed missing, the script only warned, and all 289 Markdown
files then failed on libnspr4.so (#795).
utils/install/lint/mermaid_cli.py now provisions the browser right after
mmdc, inside the serial install-lint step, as root with --install-deps
and as a user without it, because puppeteer refuses to install libraries
without root. A failed install deletes the chrome-headless-shell cache
folder and retries once under a 900 second timeout per attempt; a cut-off
download otherwise leaves a folder without its executable, and puppeteer
refuses every later install of that build. The install-lint stamp counts
as fresh only while the browser executable exists, so a recreated
container provisions it again.
mermaid.sh no longer calls apt-get or npx. It renders one minimal diagram
first and, when that fails, exits with mmdc's output instead of reporting
every Markdown file.
Verified: tests/unit/python/scripts/lint/test_mermaid.py stubs mmdc,
npx and apt-get. With a browser that cannot load libnspr4.so the script
exits 1 with that line, before any Markdown file renders, and mmdc is the
only tool it calls. Against main's mermaid.sh the same test fails: it
calls apt-get and npx and reports the files one by one. The installer and
stamp cases pass, shellcheck is clean.
Not verified: a full make lint in a recreated stack container. The race
reproduced in one of two runs in the issue.
feature/store-i18n changed the provisioning in mermaid.sh with 2f5f52bd1a
and dc67bce5f0 (timeout, deleting a cut-off browser). Both behaviours
moved into mermaid_cli.py here, so its merge takes this version of
mermaid.sh.
Closes#795
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A compose deploy job labelled centos or arch built its stack image from
pkgmgr-debian and ran it as infinito_nexus_debian (#794). distros.sh
exports INFINITO_DISTRO per iteration and sources load.sh again, but the
loader returned early because INFINITO_ENV_LOADED and the .env mtime were
unchanged. A full reload would not have helped either: the debian values
were already exported, so the restore step kept them as caller intent.
load.sh now records the distro it loaded for. Another INFINITO_DISTRO
defeats the early return and drops INFINITO_CONTAINER,
INFINITO_PARENT_IMAGE and INFINITO_IMAGE, the three keys the env handlers
derive from the distro. When the requested distro also differs from the
one .env was generated for, the loader sources a set that
`python -m cli.meta.env --output` generates for it in a private temp
directory and leaves .env untouched, so a local multi-distro run does not
switch the checkout's distro. A caller value set for the loaded distro
still wins.
Verified: tests/unit/python/scripts/meta/env/test_load.py and the new
distros.sh case pass. Against main's loader both fail with the symptom of
the issue: INFINITO_DISTRO=centos next to infinito_nexus_debian and
pkgmgr-debian:stable, and distros.sh handing debian, arch and centos the
same debian parent. shellcheck is clean.
Consequence: CI jobs labelled arch, ubuntu, fedora or centos now build
and deploy that distro, so a failure that shows up there was masked
before.
feature/store-i18n carries 3f973eb137, which resolves the parent image
for scripts/image/push.sh on its own, and 05e4dbafcb, which inserts the
INFINITO_INSTANCE slot check into the block this commit restructures. The
merge keeps the slot check ahead of the distro check.
Closes#794
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lint-playwright stages every role spec under a fresh mktemp directory and
runs `playwright test --list` there. Playwright keys its compilation cache
by source path and writes it to $TMPDIR/playwright-transform-cache-<uid>,
so with a new stage path per run the cache never hit and every run added a
full copy. In the dev container that tmpfs reached 12 GB and the quality
gate failed with "No space left on device".
PWTEST_CACHE_DIR now points into the stage dir, which the EXIT trap already
removes. Deploy-time Playwright runs are not affected; the export only
reaches this script's `npx playwright` calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The drop and truncate helpers ended with `[[ $rc -ne 0 ]] && warn`,
which returns 1 when the command succeeded. The per-stack subshell runs
on the left of `||`, where set -e is suspended, so its status is that
of the last helper: every successful MariaDB drop printed "DB purge
failed (rc=1)" and the script exited 1. The helpers now end with an if
statement, which returns 0 when nothing failed and still warns when
something did.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A gallery run took every view twice, once with the injected snippets
stripped. The comparison is only needed when the operator asks for it
for one role, so captureDesignGallery now captures the "after" side
alone and adds the plain side when PLAYWRIGHT_GALLERY_BEFORE is true.
make design-gallery passes it through as before=true; gallery.sh
refuses any other value, and rerun-spec.sh hands the variable into the
Playwright container. The deploy-driven run sets no such switch, so it
captures the designed side only.
The design loop page and the requirement describe the gallery as the
designed state and name before=true for a comparison.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The policy admits an Exception comment when it names a concrete trip-wire.
Neither of these did.
The compose one argued why the task delegates over query(node_hosts) instead
of guarding on not IS_STACK_HOST; the loop with run_once already says every
node, and the guard it warns about is not in the file to be misread. The
dnf-mirrors one explained why the SIG repository needs its own sed, directly
above a sed whose own path carries SIGs/<stream>/extras/<arch>/extras-common.
Both came from 155b35ec54 and 59840c3362 earlier today. The reasoning stays in
those commit messages, which is where it belongs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A role sync runs the whole role, several hundred tasks, although a
design carrier usually lives in one task file. Iterating on that file
cost minutes per try.
`make compose-role-sync role=<app> variant=<idx> task=<file>` now runs
only that task file of the role against the running stack. The sync
playbook gets a third scope `task` next to `design` and `role`: it
places the shared services like a role sync, loads the token store and
the proxy handlers, includes the role with `tasks_from` and flushes
its handlers.
The task file gets the variables of the role and nothing an earlier
task of the role would have set, so it has to stand alone. The script
accepts a task path of plain path characters only, because the value
travels into a command line inside the container.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The before/after gallery of a role had three gaps that cost a full
rerun or produced a wrong "before" screenshot without a failure:
- A view behind a redirect or a hash route kept the injected snippets
on its plain side, because neither passes page.route(). The helper
now loads such a document once more by the URL it ended on and fails
the view when the plain side still carries the snippets.
- A state that only exists with the injected snippets had no plain
side to capture. A view marked `afterOnly: true` skips it.
- Looking at one view meant a temporary test or the whole gallery.
`make design-gallery app=<role> views=a,b` runs the gallery test
alone, captures only the named views and adds them to the existing
screenshots.
The view names travel as PLAYWRIGHT_GALLERY_VIEWS into the Playwright
container. The name stays out of the INFINITO_ registry on purpose: it
is a call parameter, and a default in .env would silently narrow every
later gallery run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The comments policy admits a doc comment directly above a function; the same
text inside the body is a mid-code comment, which is valid only as a marked
exception, so the line was flagged for deletion. Every other shell helper in
the repository already carries its `# Param:` lines above the declaration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverses the reasoning of 7a836f934c. That commit read the metalink as the
outlier because two mirrors agreed with each other, and dropped its digest
check. The metalink was right: it advertises 0ebc31a075dcba98... for x86_64
BaseOS, which is byte-identical to the sha512 of the repomd.xml that
mirror.stream.centos.org serves today. The two mirrors that agreed were both
stale, and the check that was removed is the one that had been refusing them.
Without it baseos and appstream resolve independently, so they can land on
mirrors at different sync generations. A stale appstream then offered
python3-devel-3.12.12-3.el10 while a current baseos carried python3 at -5 and
-6; the canonical repositories list identical release sets and no -3.el10 at
all. sys-pip could not satisfy the dependency, on x86_64 at 01:35 and on
aarch64 at 13:29 of the same run, each after a full dnf upgrade had already
run.
The three repositories the image enables are pinned to the canonical upstream
so the set always shares one sync. baseos and appstream take the
<repo>/<arch>/os layout; extras-common is a SIG repository and needs its own
shape. Every other repository keeps the metalink upstream ships, which is why
the digest check returns for them.
Verified against the repository files the image ships: both spellings of the
stream segment rewrite, the disabled repositories stay untouched, and all
three pinned paths answer 200 for x86_64 and aarch64.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dnf-mirrors.sh shipped without the executable bit in 7a836f934c. The
Dockerfile calls it as `/bin/bash "<script>"`, so nothing was broken, but the
file has a shebang and test_executable requires the bit on every .sh.
files_linkeditor declared `# nocheck: dynamic-flag`, and no lint reads it:
test_dynamic_flags walks `role_dir / ROLE_FILE_META_SERVICES` and never opens
meta/addons/, and no other rule carries that name. It was the only such
marker in any addon declaration, and copying it is how the same dead marker
nearly reached metadata.yml.
Its TODO stays. The condition it records -- an upstream release using
addNewFileMenuEntry -- is unverified, so the line is the only trace of an
open obligation rather than an explanation of finished code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make compose-entity-purge removed the containers, volumes, databases
and vhosts of an app but left its group in devices.yml. Inventory
provisioning only merges groups, and the webserver health unit builds
its domain list from the groups of the host, so every later full deploy
of another app ended with failed=1: the unit kept probing the purged
domains and the service terminator gave up after 25 retries.
The make target now passes retire to the purge wrapper. With it the
wrapper hands the inventory directory into the container, and the app
orchestrator removes the purged application ids from devices.yml of the
base inventory and of every round inventory next to it.
The deploy flows that purge before they deploy the same apps again
(purge=true, the purge between matrix rounds, the reinstall selection)
call the wrapper without retire and keep the groups, because an update
deploy needs the app in the inventory.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both image jobs of run 37494541850 fail building the centos image:
Downloading successful, but checksum doesn't match.
Calculated: 76995e25878232ab... Expected: 0ebc31a075dcba98...
Error: Failed to download metadata for repo 'baseos':
Cannot download repomd.xml: All mirrors were tried
[filesystem] ERROR: 'dnf install -y epel-release' failed after 3 attempts
The retry added in d0b0b464e7 does not help and the log says why: the
Calculated digest is byte-identical on all three attempts, so dropping the dnf
metadata cache between them changes nothing. The stale bytes were never local.
Probing the sources directly settles where they come from. The metalink
advertises 0ebc31a075... for x86_64 BaseOS, matching the job's Expected value
hours later, while mirror.plusline.net and mirrors.xtom.de both serve a
repomd.xml hashing to 76995e25..., matching the job's Calculated value. The
mirrors agree with each other; the metalink is the outlier, and no retry can
reconcile a digest that no mirror serves.
The metalink exists to carry that digest alongside the mirror list, which
means the bytes and the hash for the same file arrive from two sources that
can drift apart. The mirrorlist offers the same mirrors without the second
source, so the failure class disappears rather than being retried. Mirror
redundancy is unchanged; no host is pinned.
Scoped to mirrors.centos.org. Fedora ships the same metalink shape, is not
affected by the desync, and keeps its repositories as upstream set them.
Distros without /etc/yum.repos.d exit before the rewrite.
Verified against the files the image actually ships, extracted from
centos-stream-repos-10.0-18.el10.noarch.rpm: all 23 metalink lines convert, 9
in centos.repo and 14 in centos-addons.repo. The three repositories that are
enabled there -- baseos, appstream and extras-sig-extras-common -- all resolve
through the mirrorlist endpoint.
This gives up the metalink's integrity check on repomd.xml itself. Packages
remain GPG-verified through gpgcheck.
No image has been built from this and no CI run has exercised it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both image-build jobs of run 37494541850 failed, and only centos broke; arch,
debian, ubuntu and fedora built. The headline reads like an outage:
Error: Failed to download metadata for repo 'baseos':
Cannot download repomd.xml ... All mirrors were tried
The line above it says otherwise:
Errors during downloading metadata for repository 'baseos':
- Downloading successful, but checksum doesn't match.
Calculated: 76995e25878232aba76bc22834280007be1baf40...
The download succeeds; the repomd.xml served does not match the checksum the
metalink advertises. The decisive detail is that this digest is byte-identical
on attempts 1, 2 and 3. A real re-fetch reaches a different mirror and yields a
different digest, so attempts 2 and 3 fetched nothing: dnf had written the bad
metadata to /var/cache/dnf and retry only slept and re-invoked the same command
against the same file. Three attempts, one download, and the same for all seven
build attempts.
retry_dnf is retry plus a metadata clean before each re-run, so the next
attempt is a fetch and can land on a mirror that is not mid-sync. All ten dnf
call sites use it, not just the one that failed: the zfs builddep and install
paths carry the identical weakness. apt and pacman keep retry, having no
equivalent cache trap.
The success path is unchanged, so the common case behaves identically. On the
failure path the script runs under set -e, which aborts if the clean itself
returns non-zero: measured, and it fails closed and loud rather than continuing
against a stale cache.
This fixes the retry, not the mirror. If the EL mirrors served stale metadata
for the whole window, three real fetches still fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conflicts:
- roles/web-app-pihole/meta/services.yml: main dropped the unused redis
entry, this branch changed the javascript entry directly below it. The
redis entry is gone and the javascript entry keeps the flag bound to
the SSO flag of the role and its nocheck reasons.
- README.md: the generated role table keeps the renamed web-svc-design
role and lists Pi-hole without Redis and without the CDN.
The role diagram of Pi-hole is regenerated, because both sides removed
an edge and the index of the red link moved.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make design-palette renders a deployed app with another base color, runs
its design assertions and renders the base of the inventory again through
an EXIT trap, also when a render or an assertion fails. sync=design
re-renders the static design files, sync=role re-runs the app role for
values it bakes at deploy time.
make design-gallery takes pw= to replace the default pattern. Such a run
adds its screenshots to the existing ones, so a single view is recaptured
without a full run.
The role pass now runs the gate once after its last edit, checks the
palette with blue only, repeats the full deploy at its end only when it
changed provisioning and has a wall-clock budget per class. The
verification decision of the requirement records the same rule.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The shared design helper gains tokenValue, assertToken and
assertLightAndDark. assertToken polls the computed property of an
element until it equals the value of a token. assertLightAndDark
demands that the surface behind an element is darker in dark mode than
in light mode.
assertReadable switches CSS transitions off, waits two frames and the
finite animations per mode, reads every color back from a canvas pixel,
so any CSS color syntax is measured, and blends a translucent text color
with its background. A selector that matches nothing visible fails
unless it is passed as { selector, optional: true }, and a call that
measured no element fails.
make design-gallery runs the design tests without retries, so a failing
test is reported after one attempt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make compose-role-sync runs only the role of one deployed application,
with its proxy vhost and injected snippets. It marks every shared
service of the stack as converged and skips the stack-wide service
stop. With pw= it stages the Playwright spec of the role again, renders
its env and runs it. make design-sync keeps re-rendering only the
shared CSS, the role stylesheet and the branding assets. Both run
through one playbook-sync.yml with a scope, which replaces
playbook-design.yml.
The host placement of the proxy and the shared services moves from the
constructor stage into tasks/utils/constructor/placement.yml, so the
role sync can include it.
base='<hex>' on both targets passes DESIGN_BASE_COLOR. The development
inventory references that variable for the design base color and keeps
#FFA500 as its default, so a deployed role can be rendered and checked
with another palette.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The wait job picks the newest privileged run on the head commit of the
PR. Two fork PRs that share a head repository, branch and commit but
target different bases each start such a run, and the commit alone cannot
tell them apart. A failed run of the other PR would then end the wait of
this one while its images are still being built.
The run name of the workflow carries the PR number, so the match now
requires it next to the head commit. A run without that name matches
nothing, and the wait does what it does whenever no run is found: it
keeps polling for the images until they exist or the wait times out.
Raised by the CodeRabbit review of this PR.
Refs #784
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
