Commit Graph
14682 Commits
Author SHA1 Message Date
evangelostsak 9a173ab710 feat(swarm): advertise on the mesh address instead of the underlay
swarm.manager.advertise_addr already existed as an empty hook. Resolving it
from the mesh the cross-host writer put in the inventory is what actually moves
swarm's control, gossip and overlay ports onto the tunnel; left empty, swarm
binds the underlay and the tunnel carries nothing it was built for.

Guarded on the mesh being present rather than assumed, so a compose deploy and
any swarm inventory written before the mesh pass still resolve to the previous
empty default.
2026-09-18 09:51:52 +02:00
evangelostsak d887deac52 feat(wireguard): declare the two meshes and the subnet pool
One SPOT the mesh writer reads and Ansible resolves. 10.100.0.0/16 is chosen to
clear every range the platform already allocates: Docker holds 10.208.0.0/12,
nested Docker 192.168.67.0/24 and 192.168.68.0/22, the swarm lab
192.168.244.0/24, and per-role subnets sit in the lower 192.168.0.0/16. An
overlap surfaces as unroutable peers long after the deploy reports green.

The data mesh selects its backup spoke by the svc-bkp- prefix rather than by
name, mirroring IS_BACKUP_HOST, so a new backup role joins without editing this
declaration.

Two MTU values are pinned rather than discovered. The overlay stacks VXLAN
inside WireGuard: 1500 on the underlay leaves 1420 after the WireGuard header
and 1370 after VXLAN, and a path that exceeds it drops large payloads only,
which reads as flaky CI rather than as a misconfiguration.
2026-09-18 09:51:35 +02:00
evangelostsak 0b8f8ac46f test(inventory): exercise the mesh CLI as the swarm routine invokes it
Covers the entry point rather than the library: one invocation writes every
member across both inventory files, a missing host_vars directory fails instead
of silently writing nothing, and a rerun is a no-op end to end.
2026-09-18 09:51:17 +02:00
evangelostsak 33dbd1ed8c test(inventory): pin what the mesh writer may put on disk
Asserted against real files in a tempdir, because the guarantees are about
files: which host_vars a secret may appear in, and whether a second run of an
unchanged mesh leaves them alone.

Two cases carry the security argument. A private key must reach exactly one
host_vars file, which is what keeps one host's inventory from impersonating
another, and it must never appear there in plaintext. A third covers the
inverse of the idempotence shortcut: a public key whose credential has gone
missing is worse than no key, because peers accept it while its owner cannot
use it, so that member is reported unkeyed and the pair is reminted together.

A worker is asserted NOT to receive the data mesh. Carrying it would peer the
worker directly with the NFS server and bypass the routing through the hub that
this topology depends on.
2026-09-18 09:29:35 +02:00
evangelostsak 10f0b94877 test(inventory): pin mesh planning, which fails silently when wrong
Every failure mode here is invisible at deploy time. A spoke missing from the
hub's peer list configures cleanly and never handshakes. A public key that does
not answer for its private half looks identical until traffic is attempted. A
partial rotation leaves the hub holding an old key while the spoke presents a
new one, and the handshake is refused with nothing logged on either side.

The load-bearing case is the hub filter: the manager is a member of
svc-swarm-node as well as the hub group, so left unfiltered it peers with
itself, which WireGuard accepts and which then routes the hub's own traffic
into its own tunnel.

The subnet guards are asserted against the shipped declaration too, so the
configuration proves its own collision checks rather than only the synthetic
ones.
2026-09-18 09:29:16 +02:00
evangelostsak c629fce2b7 test(inventory): add the mesh test package root
Resolves PROJECT_ROOT for the mesh suites, which read the shipped mesh
declaration rather than restating it.
2026-09-18 09:27:31 +02:00
kevinveenbirkenbachandClaude Opus 5 bde617af95 fix(bigbluebutton): take dockerize from the registry that publishes both arches
da665ab699 pinned the helper to jwilder/dockerize:v0.11.0 on Docker Hub, which
is the one tag in that repository's history published for arm64 alone. The
runners are amd64, so generate-compose died on 'exec /bin/dockerize: exec
format error' at bootstrap_compose_project.yml:13 and took the deploy with it.
Not only BigBlueButton's own: Nextcloud#11 failed on the same line, because BBB
sits in its closure.

The mirror is not at fault. Both the mirrored copy and the upstream tag are
single manifests, so the copy is faithful; Docker Hub simply has no amd64 for
that tag. Measured twice by different means, the registry API reading the
config blob and a pull reading the daemon's view, agreeing on all eighteen
rows: v0.11.0 is linux/arm64, v0.10.0 is linux/amd64, and the fifteen
alangecker/bbb-docker-* images are linux/amd64 without exception.

GHCR publishes what Docker Hub does not: ghcr.io/jwilder/dockerize carries a
manifest list with linux/amd64 and linux/arm64 for v0.11.0 and for v0.15.1, and
its tags run four minor versions further than Docker Hub's, which upstream's own
issue #320 describes as six years stale. ghcr.io is already in
_MIRRORABLE_REGISTRIES and eight other roles pin through it, so nothing in the
mirror chain changes.

The jump from v0.11.0 to v0.15.1 is covered by what the script actually uses:
generate-compose invokes dockerize with -template and nothing else, and
v0.15.1 documents that flag unchanged, single-argument form included.

Verified by deploying it. web-app-bigbluebutton variant 1 in compose, the
combination CI failed on, with make test green beforehand: the pre-pull takes
ghcr.io/jwilder/dockerize:v0.15.1, the task that used to die finishes in 0.60s,
'exec format error' appears zero times in 104k lines of log, and the play ends
ok=1178 changed=88 failed=0.

This does not make BigBlueButton run on arm64 and does not claim to. Fifteen of
its sixteen images are amd64-only; dockerize was the single arm64 one in the
set, which is why it broke an otherwise uniformly amd64 stack. Variant 0 and the
swarm mode are unexercised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:23:07 +02:00
evangelostsak dae11dd57e feat(inventory): add cross-host credential generation
Every credential generator in this package is single-host scoped.
credentials_generator shells out once per (app_id, host_vars_file) pair, and
that subprocess sees one host's inventory with no knowledge of any other. A
mesh credential breaks the assumption: the value one host receives is derived
from a secret minted for a different host, because a peer holds the public half
of a key whose private half lives elsewhere.

This package is the missing writer that sees the whole host set at once, so a
mesh is written complete or not at all. It is specified as cross-host
credential generation rather than as a WireGuard utility -- the application id
is a parameter and a mesh is a named hub-and-spoke grouping of inventory hosts,
so a later consumer needing paired secrets reuses it unmodified.

Planning is a pure function of the inventory, the spec and the public keys that
already exist, so mesh completeness, key pairing and rotation are provable
without a deploy. Keys are minted in-process with Curve25519 in the encoding
WireGuard expects, so the controller needs no wg binary to plan a mesh it never
joins, and a public key is always derived from its private half rather than
stored, because a stored pair can drift into a mesh that looks configured and
never handshakes.

Group membership is read across every inventory file it is spread over: the
swarm lab keeps its backup host in a sibling backup.yml, so a mesh spanning
both planes is only complete when both files have been read.

Vault encryption is salted, so a member that already holds a key is left
untouched rather than re-encrypted; otherwise every run would rewrite the file
and no deploy could be a no-op.
2026-09-18 09:06:52 +02:00
evangelostsak f3f6ac0f60 docs(requirements): add 037, WireGuard transport for swarm deployments
Two hub-and-spoke meshes over the five hosts default.env already names, with
the manager as the only member of both and therefore the router between the
swarm and data planes. Records the three consequences of that shape as accepted
rather than hiding them: the hub carries worker-to-worker overlay traffic, it
carries all worker storage traffic because workers are not data-plane members,
and losing its tunnel stops running containers in a way plain swarm does not.

The subnet-collision criterion comes from the topology rather than the design:
the lab underlay is 192.168.244.0/24, Docker holds 10.208.0.0/12, and per-role
subnets sit in the lower 192.168.0.0/16, so a mesh has to be allocated clear of
all three and checked before a deploy.
2026-09-18 09:06:15 +02:00
kevinveenbirkenbachandClaude Opus 5 94429e8fc5 feat(matrix): take the Element release that has the invites section
v1.12.28 is the newest tag vectorim/element-web publishes. Read at the artifact
rather than at the tag: /app/version inside the image reads 1.12.28, and the
control on v1.12.27 reads 1.12.27, so the tag does not lie here. /usr/bin/wget
is a busybox applet in it, which the wget healthcheck flavor needs.

It carries 'Room list: add an Invites section' (#34736), the section
9661800cb0 wrote its selector for while the pin stood at v1.12.27 and rendered
no such thing. df19dd103 made that selector independent of the wrapper, so the
expansion loop beside it stops being a no-op here instead of the tile becoming
unreachable again. It also carries a fix for GHSA-wqmv-r2qj-2j9p.

What the bump can reach is bounded and was counted rather than assumed. Seven
of the twelve matrix specs hold no DOM selector at all and speak only to
synapse, which stays at v1.160.0. The exposure is _shared.js with 32, this
role's DM spec with 9 and the seaweedfs spec with 2. Of the shared helper's
selectors, ten carry a fallback chain and three are single-point, none of those
three on an internal mx_ class. meta/csp.yml grants element unsafe-inline,
unsafe-eval and connect-src *, with no hash pin, so a changed asset bundle
cannot break the policy.

Not verified: no deploy ran. The same release reworks the room list three ways
-- the invites section, collapsing sections on drag, a default people section --
and that is the area the DM spec drives, so the specs that touch it are the ones
to read first if the next round is red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:52:26 +02:00
kevinveenbirkenbachandClaude Opus 5 df19dd103c fix(matrix): match the invite tile on the button, not on a row that is absent
9661800cb0 scoped the tile to [role="row"][aria-level="2"], taken from
upstream's test helpers for Element 1.12.28. meta/services.yml pinned v1.12.27
then and has never held 1.12.28 on any ref, so the selector described markup
the deployment does not render.

Read out of biber's DOM in the trace of the failing job: role="row" occurs zero
times, aria-expanded zero times, aria-label="Toggle " zero times, and every
aria-level in the document is "1" on a heading. The only aria-level="2" in the
whole trace is inside the test's own injected script. The invite is a bare
button in a virtuoso list:

  DIV[data-testid="virtuoso-item-list"] > DIV[data-index="0"]
    > BUTTON[aria-label="Open room Administrator Infinito.Nexus invitation."]

So the tile was never clicked, biber stayed on the welcome screen, and the poll
spent its 120 seconds on a page that could not change. The screenshots of all
three attempts show the same state: the invite row visible in the list, a
device-verification prompt beside it, no room open.

Matching the button directly drops the dependency on the wrapper. The label
prefix and the partner match are unchanged, and "Open room settings" is the
only other button carrying the prefix, which the partner match excludes.

The section expansion stays. It is a no-op against 1.12.27, where nothing
reports aria-expanded, and it is what upstream's helpers describe for the
release after it.

Not verified: the accept step behind the tile. biber never reached the invite
view in any attempt, so which of "Start chatting", "Accept" or "Join" 1.12.27
renders there is unobserved, and the anchored regex covering all three is
carried over untested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:49:39 +02:00
kevinveenbirkenbachandClaude Opus 5 4b0c7423cb fix(lint): credit curl to nginx rather than to Alpine
cc6da24174 measured Alpine on nginx:1.31.6-alpine alone and recorded curl as
something the family ships. nginx installs it: /usr/bin/curl is there at 272432
bytes, and at 321880 bytes in the Debian nginx:1.31, while redis:8-alpine and
node:26-alpine measured the same way have no curl at all. The table therefore
passed a curl flavor on any alpine image, which is the failure the rule exists
to catch, spelled with the rule's own name on it.

The family now holds what busybox provides, sh, wget and nc. What one image
adds on top of its family goes in IMAGE_PROVIDES, keyed on the image name with
the registry prefix dropped, and nginx is its only entry today.

The Debian half was re-measured while checking this, over /usr/bin, /bin,
/usr/local/bin and /sbin rather than one path: debian:13-slim, node:26-slim and
python:3.13.15-slim carry /bin/bash at 1298416 bytes and none of curl, wget or
nc. bash is Essential in Debian and the other three are not, which is the whole
difference between the stripped tag and the full one.

Nine cases ran against the predicate. The two that matter are curl on
redis:8-alpine and on node:26-alpine, which passed before this commit and flag
after it; curl on nginx still passes, and the six alpine declarations the tree
actually holds stay green, fediwall and nextcloud on nginx's curl, hugo and
littlejs on busybox wget, taiga on either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:34:28 +02:00
kevinveenbirkenbachandClaude Opus 5 cc6da24174 test(lint): judge a healthcheck against what its base was measured to hold
c31c62efa checked four of the repository's 86 healthcheck declarations. It read
the probe's CMD form only, scoped itself to slim, and counted bash as a shell
builtin, which is where most of the gap sat: tcp, connect and http_status all
render CMD bash -c over /dev/tcp, so 31 declarations were skipped as needing
nothing. It now examines 18, and what it still skips it skips for a reason.

bash is a program like any other here. Its absence is not hypothetical: alpine
has no /bin/bash at all, and its busybox /bin/sh does not implement /dev/tcp,
so the redirection those three flavors are built on needs the real shell rather
than any shell. No role pairs them today, which is why this is prevention and
not a fix.

The CMD-SHELL form is parsed instead of ignored: || reads as alternatives, &&
and ; as a sequence needing all its parts, and shell builtins are dropped so a
trailing || exit 1 cannot make every requirement look satisfiable. That brings
in nc, http and the msmtp compositions, six declarations the old shape could
not see. http is the one that needed the alternatives: it runs wget or curl and
is satisfied by either.

alpine moves from a docstring exemption into the scope with a measured table
behind it. Read out of the images with docker create and docker cp, which
reports an absent path rather than a shell's opinion of it: debian slim carries
/bin/bash at 1298416 bytes and none of curl, wget or nc; nginx alpine carries
curl at 272432 bytes as a real binary, wget and nc as busybox applets, and no
bash. Adding a family means measuring it the same way.

Vendor images stay out. Their contents are not knowable from the tree and most
carry no Dockerfile of ours, so the rule's remedy has nowhere to land; skipping
them honestly beats a table that rots.

Sixteen cases were run against the predicate, one per defect class and its
satisfied counterpart. Two corrected the work: nc is a busybox applet on alpine,
which the first table omitted and which made nc-on-alpine flag wrongly, and this
file's own prose named a Debian codename, which test_no_debian_codename_tags
caught in the gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:28:12 +02:00
kevinveenbirkenbachandClaude Opus 5 c31c62efaf fix(images): install the curl the slim base dropped under the healthcheck
10c5b35e88 moved simpleicons and roulette-wheel from node:26-bookworm to
node:26-slim. Both declare healthcheck flavor curl, which renders
CMD curl -f <url>, and neither Dockerfile installed anything. Measured with
docker create and docker cp: node:26-bookworm carries /usr/bin/curl at 280800
bytes, node:26-slim carries neither curl nor wget. The probe therefore fails
from the first beat and the container never reports healthy.

sys-svc-compose waits on the database container alone in compose and on every
stack task in swarm, so the consequence splits by mode. Under swarm the task
dies with non-zero exit (137): dockerexec: unhealthy container, is rescheduled,
dies again, and the converge wait fails after 600 attempts. Five jobs across
the two runs on a534d5d46 died that way, and none of them on the role they are
named after: ERPNext#0, Decidim#0, Hugo#0, Nextcloud#9 and Nextcloud#12 all
carry web-app-dashboard in their closure, dashboard enables simpleicons, and
the failure reads 'not converged: simpleicons_simpleicons'.

All five are swarm and tor, which reads like the tor axis until XWiki#1 is put
beside them: swarm, tor, green, and simpleicons is not in its closure. The
discriminator is the service, not the axis.

Of the seven roles 10c5b35e88 rebased, bridgy-fed and postmarks probe over tcp,
and flowise and bookwyrm install curl themselves. These two were the remainder.
roulette-wheel had not been deployed yet when this was written, so it is the
same defect caught before it was observed. The install goes into the runtime
stage of the multi-stage build; the builder stage would have left the runtime
image unchanged and nothing would have gone red.

The lint reads the binary out of the probe the declaration renders rather than
restating a flavor-to-binary table, so a flavor that changes its command moves
the check with it, and it locates the flavor line per service so one marker
cannot silence a sibling. Its scope stops at slim: nginx:1.31.6-alpine was
measured too and ships curl as a real 272 KB binary plus busybox wget, so the
four alpine roles that declare those flavors are served by their base, and
widening the rule to them needs that measurement per image rather than an
assumption from the word alpine.

Not built. The image with the new layer has not been produced, and whether a
second fault waits behind the healthcheck is decided by the next deploy. The
apt path itself is not blind: BookWyrm#1 installs curl the same way on
python:3.11-slim and is green in this run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:18:37 +02:00
kevinveenbirkenbachandClaude Opus 5 86f7d8c4f5 fix(install): keep the collection check inside the slice that runs it
The image runs this module from a layer that holds three files of utils:
__init__.py, install/__init__.py and install/collections.py, the slice
ef45d69dc3 cut so the source tree cannot invalidate the collection install.
67f5398985 established that the module therefore stays on the standard
library, and d4ae5cf9f0 broke it 15 minutes later by routing the reads
through utils.cache.files to satisfy the cache-read lint. The import raises
ModuleNotFoundError: No module named 'utils.cache' in that layer, which no
import inside a full checkout can show, because this repository installs
utils into site-packages.

Nothing turned red. The caller reads only the exit code, and the module's
contract is exit 1 with the missing collections on stdout, so a crash - exit
1, nothing on stdout - is indistinguishable from work to do. Every image
build since has reinstalled every collection with --force-with-deps, which
is the cost the check exists to avoid. The deploy job the traceback was
found in finished green.

The reads go back to Path.read_text with a cache-read marker each. The cache
was never earning its place here: read_text is an lru_cache keyed on the
path, and one run reads the requirements file once and one MANIFEST.json per
collection, eight distinct paths, none of them twice. Copying utils/cache
into the slice instead would work - the closure is three stdlib-only files -
but it binds a package with 51 commits in the last twelve months to the
layer that installs ansible, in front of a cache with no possible hit.

The caller now exits 3 when a non-zero status arrives with an empty stdout,
a state the contract cannot produce, so the next crash stops the install
instead of quietly reinstalling everything. all.sh runs under set -e and
touches the stamp only after ansible.sh returns, so the failure also leaves
the stamp stale rather than recording an install that did not happen.

The probe that would have caught this builds the slice in a tempdir from the
Dockerfile's own COPY lines and runs the module inside it. It has to pass
python -S: without it the interpreter resolves every submodule from the
installed checkout however the slice is built, and the test passes on a
module that cannot import in the image. Verified by injecting the import
back into the tempdir copy, which reproduces the traceback verbatim.

Delta measured by loading the committed module and this one side by side:
identical on both requirements files, against the real collections directory
and an empty one, on a missing file, on non-UTF-8 input and on a corrupt
MANIFEST.json. The one behavioural difference is that a file changed within
a process is now read fresh rather than from the memo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:03:37 +02:00
kevinveenbirkenbachandClaude Opus 5 a534d5d46e fix(hlth-csp): take the csp-checker build whose 204 no longer races
3.0.3 has a tag and no image. Its release build failed the same 204 fixture
3.0.2 failed on, because the handling read the status the moment page.goto
rejected while the listener sat on a second CDP session, and the protocol
orders messages within a session rather than between two of them. The status
now gets awaited, so the verdict no longer depends on which session delivers
first, and the page close that the wait pushes into the navigation's commit
window is bounded rather than awaited forever.

A third number rather than a rebuilt 3.0.3, for the reason 3.0.3 itself got
one: the mirror copies by tag, so an overwritten tag turns every copy that has
already seen it into a silent lie, and the deploys pull through that mirror.

The reachability check answers 404 for 3.0.4 until the release publishes, the
way it did for the two before it. That clears when the release runs.

Worth stating plainly: the runner's failure was never reproduced locally. The
ordering hole is closed by construction and two full suites are green, but
whether it was the cause is decided by the release build, not here. If that
build fails on the 204 fixture again, this pin is worth as little as the last
one and the job log is what to read next.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 01:00:02 +02:00
evangelostsak a1cae7a42a fix(sys-svc-container): drop the storage-driver indirection the merge kept
Resolving the merge I kept this branch's SYS_SVC_CONTAINER_STORAGE_DRIVER on the
grounds that it routes a value through a variable and nothing superseded it. The
unit suite disagrees: upstream added TestDaemonStorageDriver, which renders
daemon.json.j2 against a context that deliberately does not define the variable
and asserts the literal, so six tests failed with UndefinedError.

Nothing ever set the variable -- not even the wireguard role this branch removed
-- so it was a knob with no consumer, and upstream has since pinned the literal
with a test. Both files now match upstream exactly.
2026-09-18 00:52:46 +02:00
evangelostsak d783d83560 Merge feature/web-app-openbao into feature/svc-net-wireguard
Brings this pre-swarm branch up to the current swarm-era codebase: 3681 commits
across 7221 files. Our side contributed almost nothing by comparison, so the
conflict surface stayed at the 17 files both sides had touched.

Resolutions:
- The 15 legacy svc-net-wireguard-{core,firewalled,plain} files conflicted as
  modify/delete and rename/delete (upstream renamed 01_core.yml to 00_core.yml).
  All resolved as deletions: this branch is starting wireguard over. Five more
  files of those roles came back silently because upstream had not touched them
  since the merge base, and were removed after the merge.
- sys-svc-container/templates/daemon.json.j2 took upstream's structure, which
  adds default-address-pools and json-file log limits and makes storage-driver
  conditional on DOCKER_IN_CONTAINER. Our unconditional version would have set
  fuse-overlayfs on bare-metal hosts too, so theirs is the correct semantics;
  only the value now routes through SYS_SVC_CONTAINER_STORAGE_DRIVER.
- sys-svc-container/vars/main.yml keeps that variable, which upstream never had
  and nothing supersedes, and adopts upstream's removal of the MTU comment.

Follow-up the merge itself required: the root README role table was regenerated
and the privileged_control_plane list in requirement 035 dropped from 24 to 21,
both of which still enumerated the deleted roles.
2026-09-18 00:41:03 +02:00
evangelostsak b60e3fddfb chore(claude): adopt the upstream agent permission policy
This branch never edited .claude/settings.json, so the incoming version is a
straight fast-forward of it. Staged from the upstream blob directly because the
sandbox bind-mounts this path read-only and git cannot unlink it, which aborts
the merge before it starts. The recorded content is identical to what a normal
merge would produce; the on-disk copy stays behind until it is refreshed outside
the sandbox.
2026-09-18 00:36:38 +02:00
evangelostsak c40d7aa6da chore(wireguard): remove the role and its references to start fresh
The consolidated svc-net-wireguard role, its requirement document and the three
references that only existed to support it go away together: the lint ignore for
a load failure inside its tasks, the requirements index entry, and the tunnel
subnet variable nothing else consumed.

Left in place deliberately: the WireGuard mention in the svc-net category
description, which is upstream prose about networking roles in general rather
than anything this role owned.
2026-09-18 00:30:39 +02:00
kevinveenbirkenbachandClaude Opus 5 b8c651be3b fix(hlth-csp): take the csp-checker build where the 204 handling fires
3.0.2 never produced an image. Its release build failed the E2E suite on the
204 fixture itself: the handling listened for Network.responseReceived, and a
204 is never committed, so Chromium emits no such event and the checker kept
reporting the host as unreachable. The pin therefore addressed a tag that the
registry answers with 404, which the image-reachability check has been saying
since the bump landed.

csp-checker d827de1 reads the status from Network.responseReceivedExtraInfo,
the event that carries an uncommitted navigation's status, correlated with the
Document request id. Reproduced and fixed in one E2E run, with the download
fixture still failing so the change distinguishes a 204 rather than blessing
every aborted navigation.

A new number rather than a rebuilt 3.0.2, because the mirror copies by tag: an
overwritten tag turns every copy that already saw 3.0.2 into a silent lie, and
the deploys pull through that mirror.

Until the release publishes, the image does not exist and the reachability
check answers 404 for 3.0.3 the way it did for 3.0.2. That clears when the
release runs, not by a change here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 00:04:21 +02:00
kevinveenbirkenbachandClaude Opus 5 ee582762b0 feat(lmstudio): fetch the model ourselves instead of trusting one client shot
The preload is the only artifact this repository pulls raw from the public
internet. Everything else, including the LM Studio image itself, arrives
through the mirror. `lms get` gets exactly one attempt under async, cannot
resume, and wedges its own daemon when a transfer dies, so one dropped socket
costs a whole job. XWiki#2 of run 35152776835 died 28 seconds into the
transfer at 3 MB/s and rising, with 3600 seconds of budget left.

The model is now fetched on the hosting node with get_url_retry, which gives
13 attempts and verifies a pinned sha256, and handed to the container with
`lms import`. Both steps are best effort: when the warm-up fails, `lms get`
runs exactly as before, so the change adds a faster path without removing the
old one. The pattern is web-app-mediawiki's extension install, down to the
`container cp` in the script.

Verified against the binary the role actually deploys, extracted from
lmstudio/llmster-preview:cpu and built 2025-08-28, not against upstream HEAD:
`lms import` takes `<file-path>`, `-y, --yes` and `--user-repo <user/repo>`;
the destination is join(modelsFolderPath, user, repo, basename(path)) under
join(lmstudioHome, "models"); the image ships no settings.json, so
downloadsFolder cannot move that target. The already-present branch of
`lms get` carries no process.exit, so it returns 0 and the task's existing
`until: rc == 0` accepts it.

The pin carries file and sha256 next to the source URL, and the digest is
quoted because an all-digit hash would otherwise parse as an integer and fail
the shape check for the wrong reason. tests/external guards the pin against
what HuggingFace publishes, comparing digests through the API without
transferring the file, and reports a network error as unverified rather than
failing a good pin on a firewalled runner.

Two limits worth stating. `lms get` still queries HuggingFace before it
reaches the download step, so this removes 531 MB of exposure rather than the
network dependency. And the tolerated substring 'already downloaded' in the
task's until, failed_when and reap list matches no output this version emits,
which says "You already have this model."; that branch is dead today and is
left untouched here because the return code carries the contract.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:46:27 +02:00
kevinveenbirkenbachandClaude Opus 5 1941f0a689 docs(csp): draw how an application's policy is assembled and where it lies
One directive is fed from four places that each look authoritative on their
own: the role's own flags, the whitelist, the hashes the front injectors
register, and one branch that reads a service flag. Which of them ends up in
the header is only visible when the order is written down, and that order is
what the socialhome failure turned out to depend on.

The diagram is scoped to the header build and carries the three trip-wires on
the edges where they fire. The logout branch adds 'unsafe-inline' and thereby
closes the hash gate directly below it, so every registered hash is discarded,
including the one the logout injector computes for its own snippet. The flag
it reads derives from group_names, so the script policy follows the round
rather than the application. And the branch is absent from the smart-defaults
docstring, which names logout only for frame-ancestors.

Counted at this commit: 83 roles declare a logout service. 59 declare their
own script flag, 16 pin logout to false and have never had the grant, and 8
rely on it with nothing of their own. That last group is what a reader needs
the diagram for.

Node labels carry file and line so a policy that surprises someone can be
traced in one jump instead of a read through 460 lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:22:03 +02:00
kevinveenbirkenbachandClaude Opus 5 152c7b16ae fix(socialhome): declare the inline permission the app's own page needs
The served document runs an inline script at line 38. It is not ours: in the
failing variant the Lua body filter renders an empty snippet list, so nothing
this repository injects can account for it.

Until now the only token that admitted that script came from csp_filters.py,
which grants 'unsafe-inline' to script-src-attr and script-src-elem whenever
services.logout is enabled. socialhome derives that flag from 'web-svc-logout'
in group_names, so the app's own page worked or broke depending on whether an
unrelated service shared the round.

Measured in run 35152776835. Job 105253846799 reports `Blocked URI: inline`,
`Directive: script-src-elem`, `Source: /:38:0`. The two rendered headers differ
in exactly that token: variant 0 emits `script-src-elem 'self' 'unsafe-inline'
...`, variant 1 emits `script-src-elem 'self' ...`. csp-checker exits with the
violation count, so this single violation failed the hlth-csp unit and
sys-service-terminator took the job down after 25 attempts.

Rendering build_csp_header against HEAD and against this change confirms the
delta is bounded: with logout off the directive gains the token, with logout
on nothing moves. script-src-attr stays undeclared because no artifact reports
a violation there.

An inline hash was considered and dropped. No role in this repository declares
meta/csp.yml hashes, the hash would have to byte-match the served document,
and csp_filters.py discards every registered hash whenever 'unsafe-inline' is
present, which is the state this app is in as soon as logout is deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:04:01 +02:00
kevinveenbirkenbachandClaude Opus 5 6e94e5b213 docs(cache): draw the SPOT lookup path and the three ways it lies
The applications SPOT is read through four places that each look
authoritative on their own: the process-global dict, the per-host carrier
fact, the templar render, and the key that decides between them. Which one
answers a given lookup is only visible when the order is written down, and
the order is what c4556fe72 turned out to depend on.

The diagram is scoped to that path and carries the three trip-wires on the
edges where they fire rather than in a list underneath: the process cache is
consulted before the host's own fact, the strategy's main process renders a
task name for the first host of a batch and every worker forked afterwards
inherits it, and carrier.py:68 stringifies only one side of its key
comparison, so a non-scalar key component silently stops matching.

Node labels avoid Jinja braces because GitHub's Mermaid renderer treats them
as node-shape syntax and drops the block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 21:16:27 +02:00
kevinveenbirkenbachandClaude Opus 5 c4556fe72d fix(cache): key the applications SPOT by the groups its payload reads
A service flag may be written as `{{ '<role>' in group_names }}`; 114
meta/services.yml files do exactly that, and it is the only host-dependent
variable any of them uses. The rendered payload therefore differs per host
while the cache key did not: it was built from applications, users,
DOMAIN_PRIMARY and SYSTEM_EMAIL_DOMAIN alone, and in the failing job all
five host_vars files were byte-identical.

113113e553 moved the render out of every task on the premise that Ansible
forks a fresh worker per task, so the in-process cache is always cold. That
holds for workers and not for the strategy's main process, which renders a
task's name once for the first host of a batch. Anything cached there is
inherited by every worker forked afterwards, and applications.py consults
that process cache before the host's own carrier fact, so an inherited
entry outranks the correct answer.

Pretix#2 of run 35152776835 shows both sides in one task: `skipping:
[web-app-pretix-swarm-mgr-01]` against `included: dedicated.yml for
wrk-01, wrk-02`, eleven seconds before mgr-01 resolved its database to
`pretix_database`. The swarm was running the central `postgres_postgres`,
so the wait never converged and the job died after 600 attempts. Variants 0
and 1 pin the flag to a literal and both pass; only variant 2 inherits the
Jinja.

The groups join the key as a sorted comma-joined string rather than a
tuple, because carrier.py stringifies only one side of its key comparison.
A non-scalar component would make the carrier silently stop matching and
every lookup re-render.

domains derives from the applications view and inherits the same
dependency, so this closes a second instance that had not surfaced yet.
users reads group_names nowhere. No extra rendering results: group_names is
constant inside a worker, and the one process that sees several hosts is
the one the split is for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 21:12:42 +02:00
kevinveenbirkenbachandClaude Opus 5 c693f4f619 style(quality): apply what the gate wrote back into the tree
Nothing here was typed by hand. The docs generator rebuilt the MCP audit
table against pins that had already landed, so fider reads v0.36.0 instead
of stable, jenkins 2.568.3-lts instead of lts, pretix 2026.7.0 instead of
stable, and flowise 24-slim instead of 24-bookworm-slim. That is the table
catching up with da665ab69 and 10c5b35e8, reading the same declaration the
updater reads.

Autoformat wrapped three test files from the same series: one condition
pulled onto a single line, three assertEqual calls broken across lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:35:05 +02:00
kevinveenbirkenbachandClaude Opus 5 9b5d4875f7 fix(hlth-csp): take the csp-checker build that stops failing on a 204
n8n ships isbot.spawn(['bot']) unconditionally and answers any user agent
matching /bot/i with a bodyless 204. This checker calls itself
CSP-CheckerBot, so the probe never reached the application. Chromium
cancels a navigation on 204, page.goto threw net::ERR_ABORTED, and the
checker reported the host unreachable. In CI job 105068914482 the unit
failed 25 times over 1551 seconds and took the job with it.

3.0.2 ends the visit as reachable when the document response is exactly
204, and prints that it checked nothing, because page.url is about:blank
afterwards. Exactly 204, not the 2xx range: a 200 carrying
Content-Disposition aborts the same way and is a download, not an empty
document. The checker's e2e suite pins both directions, with the download
fixture as the control that fails if the condition is ever widened.

Same shape as df4b7757bc took for 304: a status the checker's own contract
calls healthy, rejected at runtime over how the browser reacts to it.

The user agent keeps its name. The checker is a bot and a bot filter
blocking it is correct; what changes is that the consequence is now stated
rather than hidden. A vhost answering 204 reports that its CSP went
unchecked instead of reporting nothing at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:29:32 +02:00
kevinveenbirkenbachandClaude Opus 5 badfe72a4f test(update): build the release-group fixture from the role-file constant
test_file_spot failed the gate on the entry fixture, which spelled
meta/services.yml out by hand. The path now comes from utils.roles.mapping
as ROLE_FILE_META_SERVICES, following 4a0fca2bcb, which fixed the same
lint on the addon fixture next door.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:26:41 +02:00
kevinveenbirkenbachandClaude Opus 5 0c018fa318 test(update): require every image pin to be orderable or declared
The ranking the last commits built toward is now enforced: a semver tag,
else a digest, and nothing else without a written reason. Two states fail.
A pin that is neither names something the registry can move underneath us,
and nothing here can tell whether it still points where it did when it was
written. A digest on an upstream that does publish semver takes the
fallback for no gain and hides the entry from the update job.

Thirteen entries stood in the way when the check was first run. Four were
resolved by the channel grammar, which made the Jitsi components orderable
without touching their pins. Three needed nothing once a moving tag could
declare itself with 'docker-version'. Two are fixed here and four are
declared.

LiteLLM leaves 'main-v1.77.3.dynamic_rates'. That tag exists exactly once
among 2609, so it is a dead end no updater can follow, and it misreports
itself: the image inside answers 1.77.2. The pin moves to
'main-v1.77.7-stable', five patches along the same minor and onto a channel
with 134 builds. Started against this repository's config shape, 1.77.7
registers both model entry forms and accepts drop_params and the
os.environ master key.

Pixelfed leaves 'latest' for '20260917'. The upstream stamps its dates as
one eight-digit number, which is depth 1 and orderable with no tooling
change at all, and the two tags resolve to the same digest, so nothing
about the deployed image moves.

The four declarations carry the count that justifies them rather than a
claim: LM Studio publishes two tags and neither is a version, the MIG
registry carries one, and the two Nextcloud Talk images publish 174 and 104
AIO build stamps between them.

Both branches of the gate were negative-controlled. The digest branch
skips in this repository because no digest pin exists at all, so it was
exercised by introducing one and watching it fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:50:27 +02:00
kevinveenbirkenbachandClaude Opus 5 e14e73fba0 refactor(docker): let a moving tag declare itself with the marker it already has
A pin outside semver was given its own 'docker-ref' marker on the theory
that every class needs one. That holds for a digest and not for a tag. A
digest pin declares something about the registry, that bytes move under a
standing tag, which no other choice says. A tag outside semver declares
that upstream publishes nothing orderable, which is exactly what
'docker-version' already means and what minio and the act runner already
carry on their version lines.

Keeping both would have put '# nocheck: docker-version, docker-ref' on
every such line: two keys, two checks, one sentence said twice. 'docker-ref'
is dropped instead. It lived in this one mapping, was never entered in the
suppression catalog, and no check ever read it.

The semver entry goes with it, because a semver pin never needs a
declaration. The test that asserted three distinct rules is replaced by two
that pin the separation where it is real: a digest marker differs from a
tag marker, and a semver pin has no marker at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:47:02 +02:00
kevinveenbirkenbachandClaude Opus 5 e66005ab07 fix(update): move a role's images as one release or not at all
Seven roles pin several images at one shared version, because those images
are one product: the four Jitsi components, six GitLab CNG images, eight
BigBlueButton services, three Penpot, plus Mastodon, Funkwhale and
Nextcloud Talk. The bump job walks entries one by one and takes each
image's own newest tag, so a run landing between upstream's first and last
push raises the components already there and leaves the rest behind. What
deploys afterwards is a mixed set, and for Jitsi a mixed set breaks video.

The window is real rather than theoretical. Measured on Docker Hub,
'stable-9457' exists for jitsi/prosody, jitsi/jicofo and jitsi/jvb but not
for jitsi/web, one build out of sixty.

Candidates are now grouped by role and current version, and the group is
held to the newest tag the intersection of all their tag lists still
offers. A group with nothing newer in common yields no update at all rather
than a partial one. A lone entry passes through untouched, so nothing
changes for the large majority.

The key is deliberately the role and the version, not the image namespace.
A namespace rule does not survive the data: prom/* carries four components
at four different versions, opentalk/* three, and minio, taigaio and
markoshust each two. Grouping by namespace would hold back pins that are
correctly independent.

Five of the seven roles have carried this exposure all along; the channel
grammar in the previous commit added the sixth by making Jitsi orderable at
all. This closes all of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:45:28 +02:00
kevinveenbirkenbachandClaude Opus 5 23fcad5ddf feat(update): read a release line written before the number as a family
Upstreams that ship several channels in parallel put the channel first:
LiteLLM publishes 'main-v1.77.7-stable' next to 'main-v1.77.7-nightly',
Jitsi ships 'stable-9646'. The grammar only allowed a leading 'v', so both
shapes parsed as nothing at all, sat at depth 0 next to 'latest', and the
bump job never looked at them.

The numeric core now takes an optional leading channel, and the channel
folds into the flavor the same way the patch letter already does.
Everything that is not the number is family; only the number is order. That
is what keeps '-stable' away from '-nightly' and 'stable-9646' away from a
bare '9646', which would leave the channel and change what the role
deploys.

The change is additive. Only a tag starting with letters followed by a dash
can match the new group, and every such tag failed the grammar outright
before, so no tag that parses today parses differently. The regression case
pins the four shapes that already worked, and the new cases pin the two
directions that must not happen along with the tags that must stay refused:
'act-latest', 'bookworm-slim', 'main-latest' and 'buildx-stable-1' carry a
channel but no version.

Two pins in this repository become orderable as a result: the four Jitsi
components and the LiteLLM gateway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:43:13 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 10c5b35e88 feat(images): move seven roles off the Debian codename base onto slim
A '-bookworm' tag names its Debian release by codename, which carries no
ordering, so nothing can raise it across a Debian major and the base stays
frozen on that release for as long as the pin stands. Moving to '-slim'
drops the codename and takes the smaller image at the same time.

The tag hides that this is two changes at once: node:26-slim and
python:3.x-slim are trixie today, so those six roles cross Debian 12 to 13
and lose the toolchain the full buildpack-deps image carried. node:24-slim
is still bookworm and its manifest is byte-identical to
node:24-bookworm-slim, so flowise changes nothing today; what it gains is a
tag that will follow the rebase instead of pinning Debian 12 forever.

Each image was built with --no-cache and the steps read back rather than
the exit code: postmarks installs git 2.47.3-0+deb13u1 and 304 npm
packages, bookwyrm builds through its runtime stage so the libjpeg62-turbo,
libsass1, libffi8, libxslt1.1 and libmagic1 it hardcodes do exist on
trixie, roulette-wheel compiles its webpack bundle. Two were additionally
exercised at runtime: sharp loads its prebuilt binding in the simpleicons
image, and bridgy-fed imports secp256k1 and constructs flask_app.app.
web-svc-cdn has no Dockerfile, so its build script's seven tools were
checked for presence instead.

bridgy-fed needs pkg-config, autoconf, automake and libtool to survive the
move. secp256k1 vendors libsecp256k1 and builds it from source: without
pkg-config the metadata step refuses outright, and with it autogen.sh exits
127. python:3.14-bookworm is the full buildpack-deps image and ships both;
slim does not. They land in the build stage only, so the runtime image
stays slim.

The flowise min-storage marker still claimed a bookworm-slim base and now
names what is actually built.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:00:51 +02:00
kevinveenbirkenbachandClaude Opus 5 da665ab699 feat(services): give fifteen moving pins a name the registry can order
A pin like 'alpine', 'stable', 'lts' or 'buildx-stable-1' names no version,
so nothing can compare it to anything: the bump job skips it and the pin
drifts with whatever upstream last pushed under that name. Each of these
fifteen now names its release, which is the precondition for the freshness
pipeline to see it at all.

Every conversion was placed against the tag it replaces, and none of them
is a downgrade. Six resolve to a byte-identical digest and are pure
relabelling (memcached, redis, the two debian bases, jenkins, xwiki); four
are genuinely newer than what the moving tag carried, so those roles will
deploy a later version than before (nginx moves past the 1.31.5 behind
'alpine', buildkit past the v0.32.2 behind 'buildx-stable-1', dockerize
past a 'latest' older than v0.10.0); openresty and pretix keep their
version and differ only by build.

fider is the one that loses something: 'stable' carries a rebuild pushed
twelve days after v0.36.0, which is still the newest versioned tag
upstream publishes. An unnamed rebuild cannot be tracked, so the pin takes
the named release and the updater can raise it when v0.37.0 lands.

The flavor invariant keeps the families apart: jenkins stays on '-lts' and
xwiki inside '-postgres-tomcat', so no bump can cross into a different
runtime.

svc-runner loses a 'nocheck: docker-version' that has become dead. The
marker suppresses two checks at once, so while it stood buildkit was held
out of the daily freshness warning as well; removing it puts the role back
in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:58:40 +02:00
kevinveenbirkenbachandClaude Opus 5 51dc06ad09 feat(update): order vendor patch counters and five-component versions
Two tag shapes in this repository name a fixed release yet failed the
semver gate, so the bump job skipped them and nothing could tell whether a
newer one existed: the Checkmk counter '2.4.0p32' and the Collabora
five-component '26.04.2.1.1'. Both returned depth 0 and fell into the
unorderable residue next to 'latest' and 'alpine'.

The numeric core now takes up to five components, and a counter written as
a letter plus a number is parsed into its own group. The letter joins the
flavor and the number joins the ordering key, which is what keeps
'2.4.0p32' in a different family from the four-component '2.4.0.32': a
bump between them would silently change what the role deploys. The number
in the key also makes p9 order below p10 instead of above it.

version_flavor reads the parse result positionally, so the wider tuple
leaves it untouched. Its contract does change for counter tags, which now
report the letter as their flavor; every shape the grammar already handled
keeps its classification, pinned by a regression case in the new test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:44:28 +02:00
kevinveenbirkenbachandClaude Opus 5 be6a25517c feat(docker): classify image pins and build pull references from the class
A digest pin was emitted as `name:sha256:...`, which no registry accepts:
image_source('ghcr.io/.../csp-checker', 'sha256:d559...') returned
'ghcr.io/.../csp-checker:3.0.0'. The separator depends on the pin class, so
the class has to be decided before the reference is built.

utils/docker/image/pin.py names the three classes in preference order
(semver, then digest, then a general ref that no registry can order) and
maps each to the nocheck marker that declares it. Both discovery entry
points now end in pull_reference(), so '@' versus ':' is decided in one
place instead of at every call site.

The semver pin lint points a digest at 'docker-digest' instead of
'docker-version', so the two fallbacks stay distinguishable in the
suppression catalog. It stays warn-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:39:16 +02:00
kevinveenbirkenbachandClaude Opus 5 f3b624e2d9 fix(swarm-lab): close the second consumer of the broken interface list
5aa7506e36 traced the enumeration bug: a veth that disappears between the RTM_GETADDR snapshot and the SIOCGIFFLAGS ioctl makes the AF_INET6 pass return 0, and network.c:836 then skips the AF_INET pass entirely. That commit removed bind-dynamic, which took the listener collector out of the blast radius. It left the other consumer of daemon->interface_addrs in place.

forward.c:1669 opens a rejection gate under OPT_LOCAL_SERVICE and walks the same list. With the IPv4 half missing, every IPv4 source is rejected, loopback included, and the once-only warning at forward.c:1695 makes a permanent outage look like a single incident. Penpot job 105068912535 and MediaWiki job 105068912565 each lost one lab node this way: PID 332 ran with --local-service and is the PID that logged the rejection; the sampler shows 179 consecutive failures over 47 minutes with the socket still bound.

Debian's init-system-common appends --local-service unconditionally, so no packaging knob removes it. option.c:6240 resets it when any of if_names, if_except, if_addrs or authserver is set, no matter which config file carried the directive. interface=* fills if_names and, through wildcard_match, still admits every interface including a docker_gwbridge that does not exist at render time. bind-interfaces and bind-dynamic are absent from that predicate and would not have helped.

Measured against the rendered template with --local-service on argv as Debian appends it: the DNS service limited to local subnets line is gone and dig still answers from the lab zone. The nodes run Debian and Ubuntu, so the task name's previous claim about fedora and centos, and about local-service implying bind-interfaces, was wrong twice over and is replaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:12:24 +02:00
kevinveenbirkenbachandClaude Opus 5 d9cc2e2833 test(swarm): make a dnsmasq that stops answering describable
Penpot job 105068912535 and MediaWiki job 105068912565 both died on one lab node whose dnsmasq kept its :53 socket bound and rejected every query with 'ignoring query from non-local network', once for a container address and once for 127.0.0.1. Rejecting loopback as non-local only makes sense if --local-service is active and dnsmasq's own interface list went stale, but the bundle records neither the flag nor where it would come from.

Capture the process command line, /etc/default/dnsmasq, the unit including its drop-ins, and the config that actually landed on disk. Addresses and listening sockets were already collected and are not duplicated.

Read-only. Every call falls back to a text line when the file or the unit is absent, so a node without dnsmasq reports that rather than failing the collector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 12:35:00 +02:00
kevinveenbirkenbachandClaude Opus 5 ed7a012faf fix(socialhome): declare the font origins the app itself loads
socialhome's own page head links a Google Fonts stylesheet. The deployment CSP did not know the origin, so the browser blocked it, csp-checker reported the violation, the hlth-csp unit failed and sys-service-terminator took the job down with it.

Job 104991069038 carries the only violation in the whole bundle, 52 times: [CSP DOM] Blocked URI: https://fonts.googleapis.com/css2?family=Fraunces:... with Directive: style-src-elem. Fraunces appears nowhere in this repository, so the reference comes from upstream socialhome rather than from our theming.

font-src for fonts.gstatic.com rides along: the stylesheet pulls the font files from there, so declaring only the stylesheet origin would have surfaced the next violation instead of clearing the check. That half is derived from how @font-face resolves and from the four sibling roles that declare the same pair, not observed, because the stylesheet never loaded.

This keeps a clearnet origin reachable from an onion deployment, which the mirror exists to avoid. The sibling roles carry a standing TODO to move such assets to the mirror; socialhome joins that debt rather than adding a new kind of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 12:23:02 +02:00
kevinveenbirkenbachandClaude Opus 5 ada2e4d311 fix(mirror): keep the origin prefix when an upstream redirects
The mirror proxies /<origin>/<path> to https://<origin>/<path> and relayed the upstream Location header untouched. A root-relative Location then resolved against the mirror instead of the origin, the prefix that names the origin was lost, the follow-up request matched no location block and fell into the catch-all 404. Chromium discards a 404 served for a script request, so the page lost the asset with net::ERR_BLOCKED_BY_ORB.

MIG job 105068912392 shows both halves, 28 times each: GET /unpkg.com/3d-force-graph/dist/3d-force-graph.min.js answered 302, then GET /3d-force-graph@1.80.0/dist/3d-force-graph.min.js answered 404. unpkg redirects an unversioned path to the resolved version and expresses the target as an absolute path.

The forms move into a redirect_sources filter so the template states the intent once and the list is unit-tested. Measured against a two-container nginx fixture: an absolute https Location, an absolute http one and a root-relative one all come back as /<origin>/target. A Location naming a third host stays untouched, which is unchanged behaviour and still leaks past the mirror.

A network-path form was written first and then dropped: nginx normalises such a target in return, and routing one through add_header emits two Location headers of which the proxy relays only the first, so the case could not be produced and the rule stayed unproven.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 12:21:41 +02:00
kevinveenbirkenbachandClaude Opus 5 df4b7757bc fix(hlth-csp): take the csp-checker build that stops failing on a 304
csp-checker 3.0.0 narrowed the healthy set to 2xx by failing every status at or above 300, which swept in 304. The health service then died on the first revalidated vhost: in CI job 104991065851 it reported element.<onion> unreachable with Status 304, and sys-service-terminator retried the unit 25 times over 1792 seconds before the job failed.

3.0.1 takes 304 out of the failure branch. The checker's own e2e suite verifies it in both directions: with the fix the second visit to a revalidating fixture reports 304 and stays healthy, without it the fixture reproduces the CI line verbatim.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 11:49:50 +02:00
kevinveenbirkenbachandClaude Opus 5 9661800cb0 fix(matrix): drive the DM invite through Element's new room list
Element 1.12.28 rewrote the room list. Pending invites now sit in their own section that renders collapsed, so no room row exists in the DOM until it is expanded. The predicate iterated an empty list and the poll spent its whole budget with no reachable path to success.

Expand every collapsed section first, then match the row by the shape upstream's own test helpers document at revision fe37d199: role=row, aria-level=2, and a button whose aria-label starts with 'Open room'. Of the four previous selectors, three matched nothing in this build and the fourth reached only the Spaces navigation. Match aria-label as well as text, because Element renders the DM partner sometimes as a display name and sometimes as a raw MXID.

role=gridcell is deliberately not among them: the cell precedes its own button in document order, so break would land on the DIV while Element's handler sits on the button.

The accept regex is unchanged. Element 1.12.28 labels the DM accept 'Start chatting' and still ships 'Accept' and 'Join'; all three are covered and Decline stays excluded. Every click stays a raw DOM click because a persistent device-verification toast overlaps the room list and would intercept an actionability-checked click.

Verified on a local compose deploy of variant 0 with sso and tor against 1.12.28: test-dm-admin-biber.js passed in 4.2m, 7 passed, no failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:18:11 +02:00
kevinveenbirkenbachandClaude Opus 5 7623c1680e fix(matrix): pin the MDAD upstream to a commit instead of a branch
ref: master was correct while repo pointed at a controlled fork (251ce17c23). When abee277216 moved repo back to the canonical spantaleev upstream it left the branch ref behind, so every deploy took whatever that branch held at that moment.

MDAD pins the images it deploys. Its commit cb58394faaef (2026-09-16 20:09Z) raised element-web to v1.12.28, which ships the rewritten room list, and the matrix DM spec went red between two CI runs with no diff of our own. The deployed container carried org.opencontainers.image.version v1.12.28; both the mirror and docker.io still served correct v1.12.27 bytes for the tag this role pins, so neither was the source.

40179e6d4a36 is that branch head at the time of this change, so the checkout is byte-identical to what master would give today and only future drift is removed. MDAD still expects migration_validated_version v2026.05.18.0 at that commit, so its migration gate stays satisfied.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:15:48 +02:00
kevinveenbirkenbachandClaude Opus 5 329f43097c fix(ci): make the image job prove the images exist before it reports success
For a fork PR the build step is skipped -- the images come from the
privileged run -- so the job went green in 29 seconds while that run was
still pushing. Everything gated on it inherited the lie: the DNS test
pulled arch 3m30s too early on one run and 53s too early on the next,
both times 'manifest unknown', and the four deploy chunks were spared
only by starting later.

The job now polls until every distro image for its tag is in the
registry. It runs unconditionally: where the job built the images itself
the first poll passes, and a push that silently failed is caught too.

The poll drives scripts/meta/resolve/missing/ci_images.sh, which already
owned that check for the release workflow, so the ref shape is not
written a fourth time. A resolver that cannot run aborts with 2 rather
than reading as 'not yet pushed' and hiding behind the full budget.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:31:33 +02:00
kevinveenbirkenbachandClaude Opus 5 b7c89f05bd fix(hlth-csp): let a declared status reach the vhost's onion twin
The checker matches accepted codes by hostname, and a clearnet vhost
shares only its leftmost label with its onion twin, so --accept-status
mirror.<primary>=404 never applied to mirror.<host>.onion. That vhost
serves the same by-design 404, so the probe called it unreachable and
systemd failed the service -- with the identical message the missing
flag produced, while the status had already moved from 2/INVALIDARGUMENT
to 1/FAILURE.

--skip-domain carried this derivation all along, in is_skipped_domain;
replacing it with --accept-status dropped it. The expansion now runs on
the same label rule, in the wrapper rather than the lookup, because the
onion address does not exist yet when the unit is rendered -- only the
wrapper sees both names, and it reads them from the nginx configs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 23:28:02 +02:00
kevinveenbirkenbachandClaude Opus 5 2e5f70383e perf(ci): stop the deploy chunks waiting on the DNS test
The chunks gated on test-dns, so every deploy row sat behind a check that
takes minutes and rarely breaks. They now wait on build-ci-images and
validate-selection only, which is what they actually need: the images
they deploy and the selection they deploy from. validate-selection moves
into every chunk's needs, where so far only chunk 0 named it.

The DNS test keeps its teeth. It still runs against the freshly built
images, and the finish job fails the pipeline on any upstream failure or
cancellation, so a broken resolver still turns the run red -- at the end
rather than before the chunks, at the price of spending runner time on
deploys that a cheap check would have pre-empted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:07:58 +02:00
kevinveenbirkenbachandClaude Opus 5 5f6dedf7a8 perf(ci): stream the DNS run per distro instead of replaying it
The job could sit silent for its full 30-minute budget and only then
print five groups, so a hanging distro was indistinguishable from a slow
one. Each container's output is stamped with [distro][HH:MM:SS] as it
arrives, and the per-distro verdict is printed when that distro finishes
rather than at the end.

A pipeline hands $! the pid of tee rather than of the test, which would
have turned every DNS failure into a green job, so each run goes through
a subshell under pipefail. The ::error:: annotation, the per-distro logs
and the container cleanup are unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:04:28 +02:00
kevinveenbirkenbachandClaude Opus 5 d4ae5cf9f0 fix(lint): put the cache-read markers where the rule reads them
suppressed_line_numbers matches the marker on the flagged line itself, and
ruff had split two of the calls so the comment landed behind the closing
paren, one line too low. The reads that can be cached now go through
utils.cache.files, which the bare interpreter imports fine; the two that
cannot keep a marker on a line short enough to stay whole.

The parser also gains the two guards utils/install/lint/pinned.py already
carries for the same bootstrap problem: its reading is cross-checked
against PyYAML on both requirements files, and a subprocess with yaml
made unimportable proves the module still works without it. That turns
the manual venv check into a standing test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:00:30 +02:00