A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.
The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.
Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.
A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two scripts sharing a delimiter-bounded name prefix are a folder in disguise, and tests/lint/repository/structure/test_prefix_grouped_files.py refuses them. A second release_* resolver is coming, so give the group its directory now and drop the prefix from the name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ba8a72309 bounded scripts/install/apt.sh and d50ba46b0 bounded the swarm
bootstrap; these three were the remainder, and one of them was still
costing whole jobs. docker_dataroot.sh hung Nextcloud#7 (job 96179137223,
run 32284644282) for 5h40m: "installing zfsutils-linux for zpool" at
19:50:39 is the last line before "rc=124 after 20400s" at 01:30:39, with
nothing in between. distros.sh's global budget is the only ceiling below
these calls, so one stalled fetch also takes the untried distros with it.
Both update and install are wrapped, not just update - that hang sat in
the install, and apt-get install fetches from the same mirror. Constants
come verbatim from scripts/install/apt.sh:19-20 rather than being
re-derived.
host_deploy.sh runs inside a container via `docker exec ... sh -c`, where
POSIX sh has no arrays, so its options are a word-split string instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
01_bootstrap.sh ran apt-get update with no timeout, so a stalled mirror
connection hung until the drill's global budget killed it - twice in run
32284541284 (jobs 96178507396 funkwhale, 96178507443 friendica), each
after hours of silence. The journal pins it exactly: sudo[462047] opened
19:51:36 and closed 00:07:30, 4h15m54s in a single apt-get update, and
the session only ended because timeout fired. Healthy runs of the same
line in the same jobs took 3s, 21s and 41s.
The cost is not one red job. distros.sh:130 is the only ceiling and it
wraps the whole per-distro sequence in the remaining global budget, so
one stalled fetch also takes the untried distros with it: funkwhale ran
2 of 5, friendica 3 of 5, both ending at budget remaining=-19s/-20s.
-qq hid the stall and || true cannot catch a hang.
Same failure class and same remedy as ba8a72309, which bounded
scripts/install/apt.sh; this closes the sibling call site it left open.
Constants are taken verbatim from there rather than re-derived. timeout
runs under sudo so SIGTERM reaches apt-get instead of the sudo wrapper -
the 4h session above is what the other order looks like. -E stays so the
package-cache proxy variables survive.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff PLW1510 wants the subprocess call to name its check mode, and the
cache-read lint flags the calls file. The stub rewrites that file on every
_run, so the shared LRU read would serve the first case's calls to all the
others -- keep the plain read and carry the marker on its own line, where the
formatter cannot move it away from the call it exempts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The id column printed the discovery id, so every mode/tor combination of a
role repeated the same number. Count the rows instead and remap 'covered by'
onto the row number of the covering row's first occurrence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Azure Ubuntu mirror stalled during runs 32265854076 and 32265543004, apt-get update burned its 10m timeout and exited 124, and 40 deploy jobs died in the install step. Every package the wrapper is ever asked for ships with the ubuntu-24.04 runner image, so the network round trip bought nothing and cost 60s per job.
dpkg-query decides per package now; apt only runs for what is genuinely missing, which keeps the act image (no envsubst, no skopeo, no gh) working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
86389f856 added the input with a 113-character description; test_name_emoji_and_length caps a dispatch input at 80. 53e5e9ba2 set the convention this missed: the description says what the input is, the workflow README says what the values do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 95883451154 failed the converge gate with stuck: shopware_scheduler exited 255 three times in 45s because 'Table shopware.scheduled_task does not exist' - init had not written the schema yet. Swarm accepts depends_on but ignores its ordering (plugins/lookup/depends_on.py), so scheduler and worker start alongside init instead of after it.
The gate ran 12:26:24 to 12:29:25, exactly 181s. At 180s churned_past_grace flipped, and with three attempts already burned has_fatal_task called it terminal - stack_ready.sh exit 2, and 'until rc in [0, 2]' stopped both callers. Nothing was broken; the install simply takes longer than the global grace.
b384f4825 introduced that grace and consolidated eleven role overrides into one SPOT, with the rule 'patience only grows where nothing is broken'. This override argues the other half of the same commit: its own lint docstring names the failure it was written against - 'the shorter one then killed a healthy-but-slow stack'. Shopware is that stack, and keeping the override role-local leaves detection at 180s for every other role.
Eight roles depend on a completed init (bookwyrm, mastodon, erpnext, openproject, funkwhale, gitlab, shopware, zammad) and share the exposure, but only Shopware is evidenced so far. Should a second one trip the same gate, the value belongs in group_vars rather than in a second role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The DR drill in job 95883448699 could not restore the BBB instance from its own backup: baudolo-restore aborted on CREATE UNLOGGED TABLE public."user", whose GENERATED columns call public.immutable_lower_unaccent(). BBB defines that function with an unqualified unaccent() in its body, and pg_dumpall has replayed with search_path='' since PostgreSQL 10 (CVE-2018-1058). Postgres inlines the body while building the generated column, unaccent stops resolving, and ON_ERROR_STOP kills the whole replay.
The fallout was two red tests from one defect: the drill left the stack down (subjects are left down), so the svc-db-openldap CLI test five minutes later found its container gone and failed its bind.
A SET clause stops Postgres from inlining the function, so the body runs with its own resolved path and the empty search_path no longer reaches it. pg_dump carries the clause into the next dump, which is what makes the backup restorable. Reproduced and verified against postgres:17-alpine: proconfig empty before, 'search_path=public, pg_catalog' after, and the same GENERATED DDL then succeeds under search_path='' with 'Ärger' still folding to 'arger'.
The SQL file is the single spot for the logic. Its DO block is a no-op in the postgres database, so the same file first yields the database list and then patches each one - one task, one retry, no SQL in the playbook. The function is upstream BBB schema, absent from this repo, so patching it after deploy is the only lever we have.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both shared flows gate performKeycloakLogin on 'openid-connect/auth' (personas/biber.js:85, personas/admin.js:64). SuiteCRM's SSO is SAML - d15716643 moved it off the oauth2-proxy onto nbgrp/onelogin-saml-bundle - so its auth URL is /realms/<realm>/login-actions/authenticate for client <base>/saml/metadata. The gate never opens, performKeycloakLogin never runs, and the personas fail at 'did NOT reach an authenticated surface'.
The trace of job 95883075126 carries no fill() at all: goto, clearCookies x2, waitForSelector x5, click x2, waitForFunction x2, fetch. Nothing was ever typed. Keycloak's 24 LOGIN_ERROR entries are the consequence - requests to login-actions/authenticate without credentials, which is also why they carry no username attribute.
The block alone would drop coverage, so saml.spec.js takes it over where the session already exists: assertCspInjections on the authenticated surface, inAppLogout, session-status must report active:false, then assertUnauthenticatedLanding. That is what runBiberFlow asserted, minus runRoleInteraction, which SuiteCRM never passed. The SAML round trip itself stays green in the two existing tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workspace swarm track passes disable=<39 provider keys> to make swarm-zombie, which forwards it to act as a runner env var. Both deploy steps then declare 'env: disable: ${{ matrix.disable }}', and a step env beats the runner env - so for variant 0, whose matrix cell is empty, the whole set was dropped.
Effect in job 95881470584: the MariaDB-only swarm deploy also brought up web-app-keycloak (629s), web-app-mailu (295s), svc-db-openldap, web-app-matomo, web-app-dashboard, svc-bkp-volume-2-local and sys-svc-cdn, plus test-e2e-playwright (997s) across all of them - while svc-db-mariadb itself, the role under test, took 169s. sso, email, ldap, matomo, dashboard, container_backup and css all sit in the disable set that never arrived. The job died at the 345-minute workspace limit.
The input defaults to an empty string and matrix.disable still wins where it is set, so the main CI passes nothing and behaves exactly as before. Verified: act accepts --input disable= with an empty value (exit 0); the || expression already runs in call-release-version.yml and call-test-runner-smoke.yml.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b6e3368a2 added the click on Funkwhale's own confirmation ("Yes, log me out!"), which fires POST /api/v2/users/logout. c7e5b0a27 then added the return to appBaseUrl so the assertion lands on Funkwhale rather than on the logout page. Each is right on its own; together the navigation aborts the request 9ms after it starts.
Measured in all three traces of job 95883448598: the POST carries status -1 with send/wait/receive all -1 - never answered, aborted. Click at 59744ms, POST at 59876ms, goto at 59885ms. The WebSocket that follows still carries sessionid=bam0ska8z0qkbtegsp10iot028cbcsr9, and openresty logs the matching 499. Only swarm+tor is slow enough to lose the race; the other seven Funkwhale variants of that run are green.
The catch keeps those seven safe: whether their confirm click fires the same XHR is unverified, and without it a missing POST would turn them red. Promise.all rather than sequential awaits because waitForResponse has to register before the click.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The native admin spec clicked in-app logout and then waited 60s for a password field on whatever page it had landed on. That page is never Shopware: with web-svc-logout attached, the flow ends on logout.<domain>, the universal-logout status page, which is terminal by design (README: "Front Channel Logout URL ... status page with live feedback").
Evidence from job 95883075100: the error-context snapshot shows heading "Logging out" plus paragraph "You are logging out from following apps" and a list - the web-svc-logout UI, not Keycloak's end-session page, which carries a Logout button instead (that one was already handled by a30d67a63). The openresty access log confirms the chain at 03:38:58: POST /realms/.../logout/logout-confirm 200, then GET logout.infinito.example/?sid=...&iss=... 200, 2555 bytes.
Returning to ${appBaseUrl}/admin before the expectation puts the claim where it can be true or false. This is the same move c7e5b0a27 made for web-app-funkwhale, and the shared persona flow has always done it via assertUnauthenticatedLanding (personas/utils/landing.js). The assertion itself is untouched, so the test gets stricter, not more lenient: a session that survived logout now fails here instead of at a spot that said nothing about it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scripts/install/apt.sh ran apt-get without any timeout, so a stalled mirror connection hung until the 355-minute job limit killed the job - three times across both runs (95883075054, 95883075011, 95883448555), each after roughly 5h50m of silence following the last apt line, "Get:5 noble-security InRelease". The Install-jq step carries no timeout-minutes of its own, so nothing bounded it.
Acquire retries/timeouts mirror the SPOT that 07d31d900 established for deployed hosts in roles/update/tasks/debian.yml. The timeout wrapper is the backstop for what Acquire cannot cover (dpkg lock, hanging post-install), and runs under sudo so the signal reaches apt-get itself rather than the sudo wrapper; -k 30 follows up with SIGKILL. 10m is 5.5x the slowest successful run measured in these same runs (6.7s / 41.1s / 81.3s / 109.8s).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
89f53a942 landed the canonical mount surface without running autoformat,
so ruff reflowed six files the next time the quality gate ran. Pure
formatter output: expressions that fit on one line joined back up, one
import pair sorted, no executable change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3183814b9 put the marker at the end of the mock-IP line, which pushed
that line to 118 chars. `make quality` autoformats before it lints, the
formatter split the call across lines and left the comment sitting on
the closing paren -- and is_suppressed_at only honours a marker on the
construct's own line or the one above, never below. The suppression
silently detached and test-lint failed on a tree that looked annotated.
Bind the literal to a name so marker and IP share one short line that no
reflow can separate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The sync sent kcadm's rc into `|| true` and the task discarded what was
left with `failed_when: false`, so a sync that never ran reported
`changed` -- in CI run 32119014832 every sync in the outage window failed
server-side with 'Database operation failed' while the deploy stayed
green, and the first visible symptom was an unrelated group-id assert 40
seconds later.
Let the rc through and gate the task on it. Three attempts over ~220s
cover the 135s outage observed there; `ignore_errors` keeps a persistent
failure non-fatal like the sibling handler in handlers/main.yml, but it
now shows up as a failed task instead of a green one. The 60s timeout
bounds a call whose OS-level kill is SYS_TIMEOUT_CONTAINER_EXEC (7d) and
leaves 22x headroom over the slowest healthy sync measured across 111
sync tasks in both deploy modes (max 2.7s).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
89fdcbda4 gated the cid retry on `retries: 0` in compose, which Ansible
folds into a single attempt (retries = 1 + max(0, n)) and then skips the
`until` entirely -- the guard was dead code, and the swarm branch
inherited the 600x2s converge budget, so one stuck resolve could burn 20
minutes per call site against a 340-minute sweep.
Use a fixed 30x2s on both queries, sized to the observed gap between task
assignment and a running container (18:47:13 -> 18:47:18 in CI run
32119014832), and short-circuit compose in the `until` itself, where the
command is a plain echo of the id the caller passed in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The converged check and the node query are two separate service-ps
snapshots; a task killed between them leaves the replacement unassigned,
.Node comes back empty and the empty delegate_to kills the play (CI run
32119014832, swarm+tor Checkmk#2). The cid lookup has the same window
until the replacement container reaches status=running.
Retry both until the query returns a value, on the converge budget the
callers already pass in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 32118850138 left a dnsmasq that was alive with zero sockets,
nothing logged, and a 6m39s death window between two host DNS
consumers. Capture each dnsmasq's fd table and rlimits in the rescue
snapshot (closed listeners vs exhausted fd table vs lost netlink
watch), and sample the :53 listener set every 10s inside the DiD
container so the next occurrence is bounded to one bucket.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge resolver comes from ansible_facts docker0, a fact that does not exist on a first deploy: the same role installs docker, and the daemon only starts on this very template's notify. Run 1 therefore renders without the bridge and run 2 with it. Where bridge and clearnet are the same address that is one entry against two - different bytes, so the template reports changed, the handler fires, and dockerd restarts underneath a live stack. In CI run 32119014832 that restart killed postgres on the manager at 12:42:26 and the deploy unravelled from there.
Deduplicating makes both renders collide on identical bytes. Order is preserved: the bridge must stay first or dnsmasq never sees the onion names.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
inAppLogout's only failure message claimed "no in-app logout control
reachable on the current authenticated surface" even when the surface was
not authenticated at all. suitecrm variant #2 burned a full triage on
that: the app had destroyed its own session mid-journey (HTTP 500,
SCRMSESSID=deleted), rendered a blank shell, and the message pointed
everyone at a missing button that provably existed.
Before failing, the helper now checks - read-only, behind every success
path - whether a login surface is visible (the password-field marker
admin.js already uses) or the body rendered an empty shell, and says
"session was lost before the logout attempt" instead. A genuinely
missing control keeps the original message byte-identical.
Proven with a fake-page differential harness against the patched and
unpatched module: green paths never query the guard selectors, all three
red shapes produce the intended message, and the unpatched baseline
yields the old message in every case (10/10 checks).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
/health/ready rides on keycloak's database connection, so the probe
stalls whenever the DB path does. In CI run 32119014832 new connections
from the keycloak container to postgres blackholed after the postgres
restart; three probe timeouts in 75s then let swarm kill the task
mid-update.
15s/5 does not cure that outage -- those connects hung >=20s and the
blackhole outlasted two minutes -- but it keeps a container that is
merely slow from being killed, while a truly dead one is still replaced
within ~2.5min.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reset the cross-node recovery budget after a successful update pass so
later updates.yml invocations in the same deploy start with 5 fresh
attempts, and make resolve_root_group_id.sh exit non-zero when the
kcadm groups listing cannot be fetched instead of feeding a silent
empty string to the root-group assert (both surfaced by CI run
32119014832: swarm healthcheck kill mid-update).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both partners in one variant overran the 20400s swarm drill budget by
~2%; the job then hit timeout-minutes and surfaced as cancelled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overview and Features still described SSO as something clicked together
in SuiteCRM's admin panel, and the persona section routed biber through
an oauth2-proxy this role no longer declares (d15716643 replaced the
proxy with SuiteCRM's own SAML login). biber authenticates via SAML with
SAML_AUTO_CREATE; the blocked flag only gates the generic persona
journeys while the LDAP spec still signs biber in from the directory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The greedy sed took the LAST X509Certificate on the line, so a descriptor
carrying an encryption key after the signing key handed SuiteCRM the
encryption cert and every assertion signature check would fail. The
extraction now splits the document into tags and takes the first
certificate after a KeyDescriptor with use="signing" - the active key,
which Keycloak lists first under rotation - falling back to the first
certificate for IdPs that omit the use attribute (SAML metadata treats
absence as both). Proven against five descriptor shapes, including the
one the old pipeline got wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
engine_by_key merges the generation manifest's volume-to-engine entries and
its docstring promises they win, but replay() called it without the
generation it was already holding, so only repository-derived names resolved.
A dump whose names the application model does not claim aborted the drill:
matrix variant 2 seeds five mautrix bridge databases into the central
postgres, their dumps sit in postgres_data under names that are no entity,
and both Matrix#2 CI jobs died on
no database service in the repository claims volume 'postgres_data'
or database 'mautrix_meta_bridge', so its engine is unknown
tearing the stack down mid-cycle and cascading into four more CLI tests.
The drill's marker seeder has passed the generation since dd07275e8, so one
run was resolving engines from two different maps; replay now reads the same
one. A generation without a manifest contributes nothing, exactly as before,
and a corrupt manifest already fails earlier in dumps_of.
The new test pins the shape: a dump only the manifest can place is replayed.
Before the change it raised with the CI message verbatim.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test feeds daemon_json_dns() a literal {"dns": "1.1.1.1"} to prove a
string value is treated as empty; the IP is test data, not configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A Tor node rewrites resolv.conf to the local dnsmasq alone, so the
rescue bundle of run 32118850138 could not prove whether the pinned
upstream (172.30.0.53) was alive while that dnsmasq sat silent on a
dead :53. Probe the daemon.json dns servers too, deduped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every kcadm call in the per-app mapper scripts sent stderr to /dev/null, so a failing call surfaced as rc 1 with empty stdout and empty stderr - no cause at all. CI run 32119014832 burned 3h23m on exactly that empty result.
stdout redirects, the tolerated-failure guards and every failed_when stay untouched: stdout is these scripts' return channel and the callers gate on its length.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The resolve passed --filter health=healthy while the very next task waits for exactly that container id to report healthy. The node is chosen first, so the filter can only subtract candidates on an already-chosen node, never redirect to a healthy replica elsewhere; when it subtracts the last one the id is empty, and inspecting an empty id never reports healthy, so the wait burns 60x5s and fails instead of waiting.
Same defect and same one-line fix as 4b52e7131 in web-app-keycloak; aa6e156a6 relaxed the sibling resolver to status=running for the same reason. Every consumer of OPENTALK_KEYCLOAK_LOCAL_CID runs after the health wait, so nothing now touches an unproven container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kcadm keeps its session inside the container, so a swarm task that gets rescheduled comes back unauthenticated. The cross-node rescue re-resolved the node but never logged in again, so all five recovery attempts died with "No server specified" -- the recovery path could not recover, by construction. Observed in CI run 32119014832 on FusionDirectory#0 and Fediwall#2 with an identical signature; a planned replacement survives today only because the realm import re-includes login.yml right after it.
Reuse utils/login.yml for the recovery leg: it already performs the same resolve and additionally re-authenticates and waits for health.
Drop the health=healthy filter on the login resolve. It contradicted the wait-for-healthy task right after it: with no healthy container the lookup returned an empty id, and inspecting an empty id can never report healthy, turning that wait into a 300s dead end whose empty-but-defined fact also defeats the default() in vars/main.yml. It also let login target a different container than the one updates.yml resolves unfiltered a moment later. aa6e156a6 relaxed the sibling resolver to status=running for the same reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every swarm task of the tor variant died 1.4s after start with exit 8, so the
stack never converged and the drill failed at sys-svc-compose's converge wait
(CI run 32119014832, job 95662863105). The task log held exactly one line: the
entrypoint's "Fetching the IdP signing certificate" - nothing after it.
GNU wget cannot parse a socks5h:// proxy. It aborts before sending anything
with "Error parsing proxy URL: Unsupported scheme.", which src/retr.c maps to
PROXERR and src/exits.c to WGET_EXIT_SERVER_ERROR, i.e. 8 - identical in 1.21
through 1.25. Under -qO- the message is suppressed, which is why the container
log ends mid-sentence. Reproduced locally: the same URL without a proxy exits
4, with the socks5h proxy exits 8.
The proxy was injected container-wide as http_proxy/https_proxy, gated on
Keycloak's domain but applied to every destination - and the CI Tor network
has no exit nodes, so that also dead-ended every clearnet call the app makes.
The three sibling roles that solved this (wordpress, nextcloud, joomla) each
gate per destination host instead. Here the only server-side onion call is the
descriptor fetch itself: the SAML bundle is configured statically, so it needs
no backchannel, and the healthcheck already forces --noproxy '*'. The proxy
now reaches exactly the one client that needs it.
curl's timeouts are pinned so a stalled onion cannot hang the boot: 21
attempts of at most 30s bound the fetch at ~10.5min, inside the 20m
start_period and the ~28min converge budget.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.
The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four jinja lints carried a byte-identical `_is_scan_target`. They check
different rules - unguarded profiles, network_mode host, depends_on
conditions, structure - but share the question of which files they look at,
and each answered it for itself.
The predicate moves into the package they already import PROJECT_ROOT from,
named is_main_compose_template because a fifth lint in the same directory
matches a deliberately wider set: test_compose_template_no_inline_mount_section
accepts compose.override.yml.j2 and its siblings, since an inline mount section
is wrong in those too. Two different facts, now two distinguishable names
rather than one name and one near-copy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lookup read services.<type>.name only for the central provider and derived
the dedicated one from the entity, which holds as long as sys-svc-rdbms renders
the engine. bigbluebutton brings its own and declares bigbluebutton-postgres-1,
so the lookup answered bigbluebutton-database - a container that does not exist.
Nothing reads that field for bigbluebutton today, which is why the mismatch
survived: it was wrong and unused at the same time. The central branch is
untouched, where `container` is the compose service key rather than a container
name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The check ran over every mode while the drill that follows deploys swarm rows
only. Discovery assigns a row's mode by its position in the candidate list, so
with all modes in play svc-db-postgres#0 becomes a compose row - and
'svc-db-postgres#0@swarm' still passed, because the check asks whether the role
CAN take swarm, not whether this row IS assigned it.
a8c265224 pinned modes=swarm for the deploy inputs. This pins the same envelope
for the validation two lines above, so both decide from one list instead of
disagreeing about which rows exist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
path_absolute builds from playbook_dir, a controller-side value, but the task
carried no delegate_to and therefore ran on the managed node. Anywhere the
controller is not the target - every swarm node, every remote deploy - the
path does not exist, stat.exists is false and the dedicated branch is skipped
for every role, taking the env file, the initdb setup and the backup seed with
it. Silently, while the rendered service still points env_file at a file
nobody wrote.
A single-host compose run cannot see this: there the controller and the target
are the same machine, which is why 47c01259e looked green.
Delegating alone is enough. run_once would additionally hand one host's answer
to every other host of a `hosts: all` play whose include is gated per host,
making the registered result depend on host ordering rather than on the host
being configured - for a filesystem check that is identical for all of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
47c01259e named the task 87 characters wide; the convention allows 80 so the
name stays readable in a playbook run. The lint that caught it also says where
a longer explanation belongs, and it is not the task name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A dedicated database is not the same as one this role provides.
web-app-bigbluebutton declares postgres enabled and unshared, so the lookup
reports it local and the dedicated branch ran - rendering an env file, an
initdb setup and a backup row for an engine bbb-docker brings itself. Nothing
of that output is read: bigbluebutton never calls the database lookup, ships
its own env.j2 and vendors its own initdb hook.
The backup row was the one that hurt. 7d044c265 taught the dedicated branch to
seed, and it seeded credentials derived from the entity name into a postgres
that has no such role:
pg_dump -U bigbluebutton -d bigbluebutton
FATAL: role "bigbluebutton" does not exist
bigbluebutton seeds its own row correctly - the '*' cluster row with the
superuser its image actually creates - so the drill now has one working row
and one impossible one for the same instance.
Whether this role provides the engine is already written down: its service
template is reached through <role>/templates/compose.yml.j2 -> base.yml.j2, so
a role without that template never gets one. Across the 53 roles with a
database service exactly one lacks it, and none has it without including
base.yml.j2, which makes the condition exact rather than a stand-in for "is
this bigbluebutton". Reading it beats declaring it: a flag would be a second
copy of a fact the tree already states, free to drift from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Docker's built-in pools yield about thirty networks. The stack holds
seventy-two, so every further network creation failed with 'all predefined
address pools have been fully subnetted' — which is why web-app-nextcloud
never came up: it brings its own networks and was the first role to hit the
ceiling.
Declare the pools beside the nested daemon's in group_vars, so both docker
daemons read their allocation from one place, and render them from there.
The base moves to 10.208.0.0/12 because 172.16/12 is exhausted: a probe
subnet in 172.28 was refused as overlapping while one in 10.210 was
created.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
