On a fresh Fedora dev container lint-mermaid failed most files with
ConnectionClosedError: the browser aborted with
`FATAL:third_party/skia/src/ports/SkFontMgr_FontConfigInterface.cpp:163]
Not implemented.` in its font fallback, because the host had no font at
all. install-lint provisions the browser's libraries by soname on dnf,
yum and pacman, and a font is not a soname; apt-get gets fonts through
puppeteer's --install-deps.
It looked load-dependent because lint-packages installs rpmlint in the
same parallel phase, which pulls fontconfig and Noto Sans: renders
crashed until that transaction landed, and every later run was green.
With every font removed, lint-mermaid alone fails 274 of 289 files the
same way.
ensure() now installs the distribution's font provide when
/usr/share/fonts holds no file (`font(:lang=en)` on dnf and yum,
`ttf-font` on pacman), and browser_installed() counts the font, so the
install-lint stamp is not fresh on a font-less host.
Closes#1154
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Three docstrings from #1173 claimed that ensure() leaves only the pinned
release on PATH or replaces whatever version PATH finds. It does neither:
it installs the release into /usr/local/bin and fails unless PATH then
resolves to it, so a distribution copy such as /usr/bin/shfmt stays on
disk behind it.
The test module and ensure() lose their docstrings; the module docstring
of the installer returns to the form its siblings use ("Install hadolint
via GitHub-release prebuilt binary.").
Reported by CodeRabbit on #1173.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`shfmt.ensure()` returned as soon as any `shfmt` was on PATH and tried
the distribution package before the release pinned in
roles/sys-lint/meta/services.yml. The Debian dev container therefore
formats with its packaged 3.8.0 (/usr/bin/shfmt), Fedora with its packaged
3.7.0, the CentOS one with the pinned v3.14.1 (/usr/local/bin/shfmt), and
they disagree on
scripts/tests/deploy/swarm/utils/unmount/nfs_mounts.sh: every gate in a
CentOS stack moved the `then` of the heredoc `if` below the terminator.
ensure() now follows ruff's installer: it reads `shfmt --version`, keeps
the binary only when it reports the pinned release, and otherwise installs
the release binary into /usr/local/bin. It fails when PATH still resolves
another version afterwards instead of formatting with it. The system
package path is gone.
nfs_mounts.sh is committed in the pinned release's format (byte-identical
to the v3.14.1 output from the CentOS container), so the first gate on any
distro leaves the tree clean.
The install-lint stamp now also depends on roles/sys-lint/meta/services.yml,
the pin file of actionlint, hadolint and shfmt, so a bumped pin reaches
environments whose stamp predates it; until now only pyproject.toml and
scripts/install/lint.sh invalidated it.
Fixes#1155
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`tarball_name()` picked the archive from `platform.machine()` alone, so
an end-of-life Node.js on Intel macOS would have unpacked a Linux build
into /usr/local, Homebrew's prefix there. The upstream install now
supports Linux only and stops with a message to install a supported
Node.js through the package manager everywhere else.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Debian trixie ships nodejs 20.19, whose release line reached its end of
life on 2026-04-30. `@mermaid-js/mermaid-cli` 12 and its puppeteer
declare `node >=22.13.0`, so `make install-lint` failed in every fresh
Debian stack container with `All providers failed for
chrome-headless-shell` (#1088).
`ensure_node_present()` still installs the distro package first, then
reads the release line of `node` on PATH and looks it up in the
nodejs/Release schedule. A line past its `end` date is replaced by the
newest LTS from nodejs.org/dist/index.json: SHASUMS256.txt is verified
with gpgv against the active Node.js release keys from
nodejs/release-keys, the tarball against its SHA-256 entry, and the
release is unpacked into /usr/local, after which `node` on PATH must be
that release. Nothing pins a major: the next LTS line is picked up the
same way. Ubuntu 26.04's Node.js 22, supported until 2027-04-30, stays.
`ensure_npm_present()` calls it first, so a container that already holds
the end-of-life node is upgraded too rather than skipped because npm
exists. The call runs once per process.
Closes#1088
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On the Fedora, CentOS Stream and Manjaro workspace jobs install-lint
ran `puppeteer browsers install chrome-headless-shell --install-deps`
as root, and puppeteer rejected it with "apt-get does not seem to be
available". The installer only warned ("optional lint tool"), so
install-lint passed while mmdc had no browser.
--install-deps is now passed only where apt-get exists. Everywhere else
the libraries `ldd` reports missing are installed by soname: dnf and
yum through the RPM provide `<soname>()(64bit)`, pacman through the
first package `pacman -F usr/lib/<soname>` names (libgbm.so.1 is owned
by both mesa and mesa-amber, which conflict). browser_installed() also
requires every library to resolve, so the install-lint stamp stays
stale until the browser can actually load.
A failed npm install, a missing mmdc, a failed or timed-out browser
download and unresolved libraries now raise instead of warning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Manjaro workspace job failed test-unit-ruby with "cannot load such
file -- minitest". Debian vendors minitest in libruby, but Arch ships it
as ruby-minitest and Fedora and CentOS Stream as rubygem-minitest, none
of which ruby pulls in. ensure_ruby_present now probes
`require "minitest"` and installs the distro package when the probe
fails, then fails loudly if minitest is still missing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- browser_revision() reads revisions.js from the puppeteer-core that
node resolves for mmdc's puppeteer: puppeteer/node_modules first, then
the node_modules directories above it. When mermaid-cli pins another
puppeteer-core, npm nests puppeteer's own copy, and the sibling held a
different revision.
- A failed browser install removes only the directories of the pinned
revision before the retry. Removing the whole chrome-headless-shell
cache also deleted browsers that other puppeteer installs share.
Addresses the CodeRabbit review on #958.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install-lint provisioned chrome-headless-shell with `npx --yes
puppeteer browsers install`, which runs the latest puppeteer from the
registry. mmdc runs with the puppeteer npm installed next to it as a
peer dependency when mmdc itself was installed, and looks for exactly
the revision that puppeteer-core pins. Once puppeteer 25.14.0 was out,
npx installed 155.0.8059.79 while mmdc with puppeteer-core 25.13.0
wanted 155.0.8059.39, and lint-mermaid failed on every file. Because
browser_installed() accepted any chrome-headless-shell, a stack with
the wrong revision never repaired itself.
The browser is now installed through the puppeteer CLI next to mmdc,
which installs its puppeteer-core's revision, and only that revision,
read from puppeteer-core's revisions.js, counts as installed.
Closes#957
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install-lint provisioned chrome-headless-shell on every run, and `make lint`
runs install-lint once per lint target in parallel whenever its stamp is not
fresh. On CI the stamp never is (eslint lives in node_modules, not on PATH),
so one lint job provisioned 16 times (run 38455552981). As root, puppeteer's
--install-deps runs `apt-get satisfy` even for an installed browser, and two
of those at once fail on the dpkg lock; a failed provisioning then deleted the
shared browser cache while lint-mermaid could be rendering with it.
The browser is now provisioned only when it is not installed, and --force
drops its cache so `make install-lint-force` still reinstalls it with its
libraries.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_no_uncached_file_reads rejects Path.read_text in tests unless the file is
written in the test itself. The mermaid script test reads the call log its
stubs append to during the run, so that read carries `# nocheck: cache-read`.
The mermaid cli test is formatted with ruff.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make lint runs install-lint once and then dispatches its fifteen targets
in parallel. scripts/lint/mermaid.sh provisioned chrome-headless-shell
inside that parallel phase with `npx puppeteer browsers install
--install-deps`, whose `apt-get satisfy` carries no lock timeout. In a
fresh stack container another target held the apt lock, the shared
libraries stayed missing, the script only warned, and all 289 Markdown
files then failed on libnspr4.so (#795).
utils/install/lint/mermaid_cli.py now provisions the browser right after
mmdc, inside the serial install-lint step, as root with --install-deps
and as a user without it, because puppeteer refuses to install libraries
without root. A failed install deletes the chrome-headless-shell cache
folder and retries once under a 900 second timeout per attempt; a cut-off
download otherwise leaves a folder without its executable, and puppeteer
refuses every later install of that build. The install-lint stamp counts
as fresh only while the browser executable exists, so a recreated
container provisions it again.
mermaid.sh no longer calls apt-get or npx. It renders one minimal diagram
first and, when that fails, exits with mmdc's output instead of reporting
every Markdown file.
Verified: tests/unit/python/scripts/lint/test_mermaid.py stubs mmdc,
npx and apt-get. With a browser that cannot load libnspr4.so the script
exits 1 with that line, before any Markdown file renders, and mmdc is the
only tool it calls. Against main's mermaid.sh the same test fails: it
calls apt-get and npx and reports the files one by one. The installer and
stamp cases pass, shellcheck is clean.
Not verified: a full make lint in a recreated stack container. The race
reproduced in one of two runs in the issue.
feature/store-i18n changed the provisioning in mermaid.sh with 2f5f52bd1a
and dc67bce5f0 (timeout, deleting a cut-off browser). Both behaviours
moved into mermaid_cli.py here, so its merge takes this version of
mermaid.sh.
Closes#795
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The image runs this module from a layer that holds three files of utils:
__init__.py, install/__init__.py and install/collections.py, the slice
ef45d69dc3 cut so the source tree cannot invalidate the collection install.
67f5398985 established that the module therefore stays on the standard
library, and d4ae5cf9f0 broke it 15 minutes later by routing the reads
through utils.cache.files to satisfy the cache-read lint. The import raises
ModuleNotFoundError: No module named 'utils.cache' in that layer, which no
import inside a full checkout can show, because this repository installs
utils into site-packages.
Nothing turned red. The caller reads only the exit code, and the module's
contract is exit 1 with the missing collections on stdout, so a crash - exit
1, nothing on stdout - is indistinguishable from work to do. Every image
build since has reinstalled every collection with --force-with-deps, which
is the cost the check exists to avoid. The deploy job the traceback was
found in finished green.
The reads go back to Path.read_text with a cache-read marker each. The cache
was never earning its place here: read_text is an lru_cache keyed on the
path, and one run reads the requirements file once and one MANIFEST.json per
collection, eight distinct paths, none of them twice. Copying utils/cache
into the slice instead would work - the closure is three stdlib-only files -
but it binds a package with 51 commits in the last twelve months to the
layer that installs ansible, in front of a cache with no possible hit.
The caller now exits 3 when a non-zero status arrives with an empty stdout,
a state the contract cannot produce, so the next crash stops the install
instead of quietly reinstalling everything. all.sh runs under set -e and
touches the stamp only after ansible.sh returns, so the failure also leaves
the stamp stale rather than recording an install that did not happen.
The probe that would have caught this builds the slice in a tempdir from the
Dockerfile's own COPY lines and runs the module inside it. It has to pass
python -S: without it the interpreter resolves every submodule from the
installed checkout however the slice is built, and the test passes on a
module that cannot import in the image. Verified by injecting the import
back into the tempdir copy, which reproduces the traceback verbatim.
Delta measured by loading the committed module and this one side by side:
identical on both requirements files, against the real collections directory
and an empty one, on a missing file, on non-UTF-8 input and on a corrupt
MANIFEST.json. The one behavioural difference is that a file changed within
a process is now read fresh rather than from the memo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
suppressed_line_numbers matches the marker on the flagged line itself, and
ruff had split two of the calls so the comment landed behind the closing
paren, one line too low. The reads that can be cached now go through
utils.cache.files, which the bare interpreter imports fine; the two that
cannot keep a marker on a line short enough to stay whole.
The parser also gains the two guards utils/install/lint/pinned.py already
carries for the same bootstrap problem: its reading is cross-checked
against PyYAML on both requirements files, and a subprocess with yaml
made unimportable proves the module still works without it. That turns
the manual venv check into a standing test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint installer imports this module, and it runs on the runner's bare
interpreter before a single dependency exists. Sharing the version check
with it therefore handed the bootstrap a PyYAML import, which killed it:
'ModuleNotFoundError: No module named yaml' raised through
utils/install/lint/__main__.py:35.
Routing the read through utils.cache.yaml would not have helped, that
imports yaml too. The requirements files are this repository's own and
hold one shape, so the pins are read with two regexes and the standard
library. A line that does not match the shape leaves the entry unpinned,
and unpinned already counts as unsatisfied, so the parser errs towards
installing rather than towards skipping.
Quoted versions are unwrapped: yaml accepts version: "13.4.0" and a
formatter may introduce it, and carrying the quotes into the comparison
would silently defeat the check instead of failing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The requirements named seven collections without versions, so every
install re-resolved them against galaxy.ansible.com and --force-with-deps
redownloaded all 37 MB even when nothing had changed. Exact pins make the
installed set checkable, and the check short-circuits the whole roundtrip:
make install-ansible drops from 8m24s to 0.077s, and a Galaxy outage stops
mattering once the pinned set is present.
The git fallback is pinned to the same versions so it installs what the
Galaxy file declares rather than whatever the branches currently hold.
kewlfft.aur pins a commit because its tags carry no galaxy.yml and master
has moved past the released version.
The lint installer kept a second, weaker truth about the same directory,
guessing from directory presence for three hardcoded collections; it now
shares the version-exact check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Hugo image build in run 34653542922 died on a Debian mirror mid-sync: 'Err:13 libruby ... File has unexpected size (6300780 != 5440). Mirror sync in progress?', and apt closed with its own advice, 'maybe run apt-get update or try with --fix-missing?'. The index the build had fetched disagreed with the pool it then read.
Acquire::Retries=3 does not cover that. It re-fetches the same URL against the same stale index and reads the same wrong size, which is why three retries produced three identical failures.
install_package_candidates now runs its candidate loop through a helper and, when every candidate failed and the manager is apt-get, fetches a fresh index and runs the loop once more. The helper returns two facts rather than one because the two existing error paths distinguish them: whether the caller's command became callable, and whether anything installed at all.
The retry is bounded so it cannot mask a real absence: apt-get only, once, only when nothing installed, and only behind a new index. A package that does not exist fails the second pass the same way and raises the same RuntimeError. The tests pin both directions, the exact update, install, update, install sequence for apt-get and two calls only for a manager without an index, so the retry cannot spread to pacman or dnf.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conflict: roles/web-app-bridgy-fed/files/Dockerfile
Both sides extended the header comment. main also replaced the upstream
checkout, `git clone --depth=1 --branch "${BRIDGY_REF}"`, with git init plus a
fetch of the single ref; this branch never touched those lines, so git took
main's version automatically and only the comment collided.
Resolved to main's file verbatim. It is a superset of ours in both respects:
it keeps our two comment paragraphs and adds a third naming the trip-wire the
non-idiomatic fetch exists for, namely that BRIDGY_REF is a commit and
`git clone --branch` refuses one. Verified byte-identical to stage 3.
Committed with the hook skipped and before the semantic audit of the other 45
files finished. main brings a lint rule that has never seen this branch's
roles, image bumps on svc-ai-ollama and web-app-nextcloud which this branch
rewrote, a stylelint major, and release 13.0.0 across the packaging surface.
Any of those can merge cleanly line by line and still be wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without a zip extension or an unzip/7z binary Composer silently drops to
--prefer-source and mirror-clones every dependency, which exceeds its
300 s process timeout on phpunit.git. That took the Arch workspace job
from a 2m16s test-unit to a 16m08s failure. Debian never saw it because
apt pulls php-zip and unzip in as Composer dependencies; pacman does not.
Arch ships no zip extension and no php-zip package, so the extractor has
to be the unzip binary rather than the extension. Its mapping already
covers all five package managers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ensure_php_toolchain installed the php and composer binaries and stopped
there. A distro ships the interpreter and its extensions as separate
packages, and ensure_command_present returns early once shutil.which
finds php - which it always does since the image bakes the interpreter -
so the extensions were never provisioned on any path.
composer then refused to resolve the vendor tree with
"phpunit/phpunit requires ext-dom * -> it is missing from your system"
and test-unit-php failed in every workspace job. The loaded ini list in
those logs shows 20-mbstring.ini present and no 20-dom.ini, 20-xml.ini
or 20-xmlwriter.ini, i.e. php-xml missing.
Extensions are now probed against php -m and installed from a per-manager
mapping, then re-probed so a package that does not deliver the extension
raises instead of leaving composer to fail later. The loop sits in
ensure_php_toolchain because both entry points pass through it: the image
build via utils.install.interpreters and the test run via
scripts/install/php.sh, so a prebuilt image is repaired at test time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The committed line was 89 characters, one over the default limit, so
every autoformat run rewrote it and it kept reappearing as an unrelated
worktree change.
The unit suites had to install their toolchain on every run. The image
now carries the interpreters, so the make prerequisite finds them present.
The Composer vendor tree deliberately stays out of the image build: the
package cache is reached through the compose extra_hosts mapping, which
only exists for a running container, so a build-time composer install
would bypass the cache and talk to packagist directly. Fetching it from
the test target keeps it on the cached path.
test-unit-php and test-unit-ruby exited 0 whenever the interpreter or the
vendor tree was missing, so both suites were silently absent from every
gate. They now provision their toolchain through install-php/install-ruby
and fail loudly when that is impossible.
The three ensure_*_present helpers collapse into ensure_command_present
in system_pkg, and the ruby runner rejects an empty suite because
minitest reports success when nothing was required.
test-unit-javascript ran `node --test` without declaring where node
comes from, while its python sibling declares `install`. On a bare
distro image the target died with `exec: node: not found` (127), which
took down the pre-commit hook and all five test-workspace jobs.
The installer stack already knows how to pull a command from the system
package manager; node was simply missing from the table. Wiring the
target to install-node provisions it the same way php, ruby and npm are
provisioned, and fails loudly when the runtime cannot be installed
rather than skipping the suite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
