sys-front-inj-javascript collapses a role's javascript.js(.j2) through
to_one_liner, body_filter.lua.j2:5 embeds the result as a Lua [=[ ]=] long
string, and add_csp_hash takes the hash over that same collapsed text.
Nothing checked that the text survives any of those three stages.
Both failure modes are silent. A script that parses only before the collapse
is served with a matching hash, so the browser rejects nothing and executes
nothing - the page loses its injection without an error anywhere in the
deploy. An inner ]=] closes the Lua long string early, after which the
OpenResty config stops loading for every site on the host.
The test walks all three source shapes (templates/javascript.js.j2,
templates/javascript.js, files/javascript.js), substitutes Jinja instead of
rendering it - a neutral literal for expressions, nothing for statements,
enough to expose an ASI break without standing up an inventory - and runs
node --check over the collapsed result. Eleven scripts are covered today and
all eleven pass.
Neither guard existed anywhere: git log --all finds no `node --check` and no
`]=]` assertion in tests on any ref.
node is gated with skipUnless, matching test_service_gating_helper.py and
test_timeouts_helper.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test set INFINITO_CI_DISTRO_BUDGET_SECONDS=1 and asserted the loop
returns 124. distros.sh:45 fixes the deadline at start+1s and only checks
it at :105, so whenever the shell needs longer than that one second to
reach the loop, `remaining <= 0` breaks out before anything runs, the
script falls through to render_summary and exits 0. The assertion then
reads 0 != 124.
That is what CI hit in run 32369164168 job 96426381092, under pytest-xdist
([gw2] in the failure header) where a second of wall clock is not the
script's to spend. A retry passed on the same code, which is the flake
stated plainly.
5s keeps every property the test is there for - sleep 30 still exceeds it,
timeout still kills, the assertion still demands 124 - while putting the
fixture's own startup well inside the budget. 67b6072bb made a budget kill
fail loudly on purpose rather than booking it as skipped; this only makes
the test reach that kill reliably.
The message argument moves from proc.stderr to proc.stdout because
stderr is empty here: the "[WARN] Global budget exhausted" line the script
prints goes to stdout, so the old failure said nothing about its cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workflow was created deactivated and never activated, so the declared
endpoint answered 404 and every client configured against it failed its
reachability check. The intent was that an operator opts in before an MCP
server answers, but mcp.enabled already is that opt-in: a second one hidden
in the provisioning script made the declared state unenforceable, and the
mismatch surfaced three layers away as a failing consumer.
Activate after create and after update. Keep the protection that matters:
an already-active workflow is left untouched rather than rewritten under
callers bound to its endpoint, and that case now returns instead of ending
the deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.
The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.
Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.
A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff PLW1510 wants the subprocess call to name its check mode, and the
cache-read lint flags the calls file. The stub rewrites that file on every
_run, so the shared LRU read would serve the first case's calls to all the
others -- keep the plain read and carry the marker on its own line, where the
formatter cannot move it away from the call it exempts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The id column printed the discovery id, so every mode/tor combination of a
role repeated the same number. Count the rows instead and remap 'covered by'
onto the row number of the covering row's first occurrence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Azure Ubuntu mirror stalled during runs 32265854076 and 32265543004, apt-get update burned its 10m timeout and exited 124, and 40 deploy jobs died in the install step. Every package the wrapper is ever asked for ships with the ubuntu-24.04 runner image, so the network round trip bought nothing and cost 60s per job.
dpkg-query decides per package now; apt only runs for what is genuinely missing, which keeps the act image (no envsubst, no skopeo, no gh) working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
89f53a942 landed the canonical mount surface without running autoformat,
so ruff reflowed six files the next time the quality gate ran. Pure
formatter output: expressions that fit on one line joined back up, one
import pair sorted, no executable change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3183814b9 put the marker at the end of the mock-IP line, which pushed
that line to 118 chars. `make quality` autoformats before it lints, the
formatter split the call across lines and left the comment sitting on
the closing paren -- and is_suppressed_at only honours a marker on the
construct's own line or the one above, never below. The suppression
silently detached and test-lint failed on a tree that looked annotated.
Bind the literal to a name so marker and IP share one short line that no
reflow can separate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 32118850138 left a dnsmasq that was alive with zero sockets,
nothing logged, and a 6m39s death window between two host DNS
consumers. Capture each dnsmasq's fd table and rlimits in the rescue
snapshot (closed listeners vs exhausted fd table vs lost netlink
watch), and sample the :53 listener set every 10s inside the DiD
container so the next occurrence is bounded to one bucket.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge resolver comes from ansible_facts docker0, a fact that does not exist on a first deploy: the same role installs docker, and the daemon only starts on this very template's notify. Run 1 therefore renders without the bridge and run 2 with it. Where bridge and clearnet are the same address that is one entry against two - different bytes, so the template reports changed, the handler fires, and dockerd restarts underneath a live stack. In CI run 32119014832 that restart killed postgres on the manager at 12:42:26 and the deploy unravelled from there.
Deduplicating makes both renders collide on identical bytes. Order is preserved: the bridge must stay first or dnsmasq never sees the onion names.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
engine_by_key merges the generation manifest's volume-to-engine entries and
its docstring promises they win, but replay() called it without the
generation it was already holding, so only repository-derived names resolved.
A dump whose names the application model does not claim aborted the drill:
matrix variant 2 seeds five mautrix bridge databases into the central
postgres, their dumps sit in postgres_data under names that are no entity,
and both Matrix#2 CI jobs died on
no database service in the repository claims volume 'postgres_data'
or database 'mautrix_meta_bridge', so its engine is unknown
tearing the stack down mid-cycle and cascading into four more CLI tests.
The drill's marker seeder has passed the generation since dd07275e8, so one
run was resolving engines from two different maps; replay now reads the same
one. A generation without a manifest contributes nothing, exactly as before,
and a corrupt manifest already fails earlier in dumps_of.
The new test pins the shape: a dump only the manifest can place is replayed.
Before the change it raised with the CI message verbatim.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test feeds daemon_json_dns() a literal {"dns": "1.1.1.1"} to prove a
string value is treated as empty; the IP is test data, not configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A Tor node rewrites resolv.conf to the local dnsmasq alone, so the
rescue bundle of run 32118850138 could not prove whether the pinned
upstream (172.30.0.53) was alive while that dnsmasq sat silent on a
dead :53. Probe the daemon.json dns servers too, deduped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.
The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four jinja lints carried a byte-identical `_is_scan_target`. They check
different rules - unguarded profiles, network_mode host, depends_on
conditions, structure - but share the question of which files they look at,
and each answered it for itself.
The predicate moves into the package they already import PROJECT_ROOT from,
named is_main_compose_template because a fifth lint in the same directory
matches a deliberately wider set: test_compose_template_no_inline_mount_section
accepts compose.override.yml.j2 and its siblings, since an inline mount section
is wrong in those too. Two different facts, now two distinguishable names
rather than one name and one near-copy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lookup read services.<type>.name only for the central provider and derived
the dedicated one from the entity, which holds as long as sys-svc-rdbms renders
the engine. bigbluebutton brings its own and declares bigbluebutton-postgres-1,
so the lookup answered bigbluebutton-database - a container that does not exist.
Nothing reads that field for bigbluebutton today, which is why the mismatch
survived: it was wrong and unused at the same time. The central branch is
untouched, where `container` is the compose service key rather than a container
name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Docker's built-in pools yield about thirty networks. The stack holds
seventy-two, so every further network creation failed with 'all predefined
address pools have been fully subnetted' — which is why web-app-nextcloud
never came up: it brings its own networks and was the first role to hit the
ceiling.
Declare the pools beside the nested daemon's in group_vars, so both docker
daemons read their allocation from one place, and render them from there.
The base moves to 10.208.0.0/12 because 172.16/12 is exhausted: a probe
subnet in 172.28 was refused as overlapping while one in 10.210 was
created.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-svc-mcp-reconcile sets application_id from a loop variable, so the
lookup received the literal {{ mcp_client.id }}. An unrendered id matches
no allowed_consumers, which rejected every provider as
consumer_not_allowed and emptied each client's registry without raising.
Pull the resolution into resolve_consumer_id() so the branch is reachable
from a test, and cover the three cases: a deferred id is rendered, a plain
id is kept, a missing id aborts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The id leads, the variant follows the name it belongs to, and covered_by,
weight and the new clone column close the row. ✂️ said only that a row was cut
while two different reasons hide behind it: 🔰 points at the earlier row that
already embeds this one, 🐑 says the row shares its service set with an earlier
role that stands in for it. 27 rows of a full plan carry 🐑 and were until now
indistinguishable from the rest.
_COLUMNS is the order and the headers again. tor and triggered used to sit
outside it as special cases, so a reordering had to be made in two places and
the tests indexed by hand; they now ask _COLUMNS for a column's position.
The chunk column loses its ⏭️ marker, which named the block the calling job was
running. Without it cells() has no use for `current` and --chunk no effect, so
both go, along with the workflow's argument.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
a2789f7bb was proven by a deploy run and nothing else. The pair here states
both halves of the rule: the engine holding the dump's volume does not block
its own replay, and a real consumer beside it still aborts - naming that
consumer and not the engine.
The second case is what keeps the first honest. On its own the first would
also pass if the guard were widened until it stopped refusing anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d130f5559 moved the swarm converge wait onto compose_mode_force-or-
DEPLOYMENT_MODE and left the compose health wait beside it reading
DEPLOYMENT_MODE raw. A role pinned with compose_mode_force: compose on a
swarm cluster then satisfied neither condition - the compose branch saw
'swarm' and the swarm branch saw 'compose' - so it got no post-deploy
readiness wait at all. web-app-matrix in its ansible flavour is the live
case.
The database lookup read DEPLOYMENT_MODE raw for the same reason, handing a
pinned role a swarm service name and a resolve-container-id address for a
stack that docker compose had created.
The override is scoped to the dedicated database only. A dedicated engine
runs inside the consumer's own stack, so the consumer's pin governs it; a
central engine is its own stack under the cluster mode, and letting a
consumer's pin reach it would rename a service swarm really did create.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
i8cifail died twice on the same run, each failure hiding the next.
The retrigger recovers a source run's dispatch inputs from a job log, because the title truncates at 512 bytes and REST answers inputs: null for a finished workflow_dispatch. That reader picked the first depth-1 job outright. A skipped job never uploads a log blob, so its log endpoint answers BlobNotFound as a bare HTTP 404 -- and a force-cancelled run can leave every depth-1 job skipped, which is why deeper jobs are now read too rather than assumed inputs-free. Candidates are taken shallowest first, so the orchestrator's own inputs still win over what a caller forwarded. gh 2.97 also refuses to emit a log holding terminal escape sequences and exits 1; the log downloader read that as no log and silently wrote none. A missed fetch is now tolerated instead of aborting, so the guard in dispatched_priority reports an expired log in words rather than the command dying on gh: HTTP 404.
Behind that sat the resolver: it decoded whole priority entries, but an entry may pin axes (web-app-zammad#1@compose+tor), which the display-name codec does not parse. Every pinned entry aborted as corrupt. split_axes -- already the seam decode_list uses -- separates name from suffix, so only the name is resolved and the truncation guard stays sharp. The pin rides along into the retrigger: the entry never ran, so the run holds no evidence its axes were wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test climbed to the repository root itself. Moving the file one directory would have left that index pointing somewhere else, and the test would then fail with FileNotFoundError instead of saying anything about compose.py. Its own package already defines PROJECT_ROOT and every sibling test module imports it from there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Output of 'make autoformat': ruff collapses expressions that were wrapped across lines without need, shfmt and the yaml formatters drop surplus blank lines. No semantic change - the whole diff is 10 insertions against 39 deletions, and every hunk joins lines that already said the same thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff B905 flagged zip(comments, spans) without strict=. spans is built by a comprehension over comments, so the two can only ever be the same length; strict=True can never fire and records that, while a future edit that breaks the pairing now fails loudly instead of silently truncating the shorter side.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A recent refactor imported baudolo into utils.recovery for three layout strings, and cli.administration.recover.__main__ imports full unconditionally - so the whole recovery CLI, --help included, died at import on every host without baudolo. That is most of them: recover device|nfs|secrets runs on swarm nodes, the NFS server and bare rescue hosts, while only the backup node carries svc-bkp-volume-2-local. utils/recovery had been deliberately stdlib-only for exactly that reason.
baudolo 5.0.0 writes a manifest into every generation stating the layout it used, so the names are now asked of the generation being read rather than of the installed package. A tree written by any version is walked with the names that version used, and utils.recovery.layout answers only for generations older than the manifest, which cannot describe themselves. That also retires two of core's guesses: the '<entity>_database' naming convention now yields to the engine the run actually recorded.
A repo whose manifest cannot be read is skipped rather than guessed at, but no longer takes the other repos with it - recovering two of three beats recovering none because one manifest is corrupt. The exit code reports the gap.
The pin lint compared two declared version strings and was therefore green while a different distribution was installed. It now checks the installed distribution and the layout literals against baudolo's own BackupPaths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
seed/token.py sat on sys.path beside the scripts that ran it, so any import chain reaching tokenize picked up the drill's seeder instead of the stdlib's token module - dataclasses -> inspect -> tokenize -> token is enough to trigger it. marker.py names what the file does and collides with nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two collectors run over a .j2 template, so the closing '#}' of a Jinja
block was also a standalone '#' comment to the line-based one. The
header rule protects the opening '{#' but not a closer 22 lines down,
so the fragment got reported, deleted, and the template ended up inside
an unterminated comment: it rendered to nothing while every parser
stayed happy.
_drop_nested drops any comment whose line span is a proper subset of
another's, so the enclosing comment always wins.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The linter only saw the working-tree diff, so 3529 invalid comments in
528 files never broke a build. It now enumerates git ls-files, and the
policy holds for the whole repository.
Two changes ride along, both required to make that scope survivable:
_css_comments never closed a /* */ block on the line that opened it, so
every single-line CSS comment swallowed the following code as its body.
The reported spans were unusable for anything but counting.
A single comment line directly above a key in defaults/*.yml,
vars/*.yml or group_vars/ now counts as that variable's doc, the way a
docstring counts above a function. Two lines are a block again and stay
flagged, and the carve-out holds nowhere else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule only fired one way: a stack role with named volumes had to carry
the flag, but a role without any volume could carry it unnoticed. That
entry pulls svc-bkp-volume-2-local into the role's dependency graph and
schedules a backup run over a role with nothing to back up.
_scan() now classifies every invokable role instead of skipping the ones
that need no flag, and returns both halves. The nocheck opt-out still
exempts a role from both directions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4.0.0 renamed --dump-only-sql and rejects the old spelling with exit 2, so the timer's assembled exec string would have failed on every run against the version the previous commit pins. The launcher fixtures and the role README carry the same rename; nothing else in the repo still says it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/recovery restated three things baudolo owns: the generation layout (suffixes, sql/ and files/ directories), the \connect parser for cluster dumps, and the databases.csv reader. Each was a second implementation of a contract whose author is one pip install away - the connect parser had even received the same quoted-identifier fix twice, once per repo.
The layout constants are now derived from BackupPaths, the inventory comes from dump_inventory, and the CSV goes through read_rows. dump_inventory is stricter than the parser it replaces: it also reads CREATE DATABASE lines, so a database a stream creates but never reconnects to is found - a new test pins that. recover/full.py and the drill's token.py take their globs from the same constants instead of spelling them out.
The dependency this creates is declared: backup-docker-to-local==4.0.0 in the base dependencies, because the deploy CLI must import from the base set alone. The role installs the same version, and a lint holds the two pins together - they drifted apart is exactly the failure this whole commit removes, so it must not come back one level up.
The generation-layout tests now write their fixtures through BackupPaths itself; before, the fixture hand-built the paths with the same assumption the test claimed to verify.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5bb803054 tied the dnsmasq listener to group membership instead of the deploy mode, but left these assertions behind, so the branch carries two red tests. A Tor node runs the same listener in compose and in swarm, so a swarm node now expects the bridge resolver first and the clearnet upstream behind it, and the same node without an upstream expects the bridge alone rather than nothing.
The empty list did not lose its case, only its reason: it belongs to a node that is neither on Tor nor given a clearnet resolver, which now has a test of its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four files outside the healthcheck work, all flagged by make quality-high.
The shopware marker sat on the first line of a value the YAML formatter wraps, where the lint never looked; it now sits on the last line, beside the trailing comment the formatter keeps there. The two ci meta tests get ClassVar for a mutable class attribute and unpacking instead of list concatenation.
The keycloak realm-import test asked jinja2 for an Environment without naming autoescape. It stays False - the template renders JSON, and HTML escaping corrupts the values - and the rule is silenced the way 9a9c2f239 already silences it for the dotenv filter test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A container without a healthcheck counts as healthy the moment its process starts. That is how a service which came back empty passes a restore drill: the drill asks the host's health service, and the host says yes.
The image side is read from the registry rather than by pulling: the manifest names the config blob, the blob carries the HEALTHCHECK. The answer is three-valued on purpose. A mapping means the image ships a probe, an empty mapping means it definitely ships none, and None means the question could not be answered. Only the empty mapping is evidence; reading None as absent turns one network hiccup into a hundred false findings.
Probes address the GHCR mirror first and fall back upstream only for images the mirror does not carry, and a second test requires mirror plus cache to answer more often than upstream, so the fallback cannot quietly become the rule and spend the anonymous rate limit again.
Detection spans the four places a probe can come from: the services.yml declaration, a literal block in a template, a partial pulled in by include or lookup('template'), and a HEALTHCHECK in a Dockerfile the role builds its own image from. Without the last three the sweep reported nextcloud, every redis sidecar and openldap as gaps they are not.
The opposite direction gets its own lint: a declared healthcheck that no template renders is dead configuration. Fixing it surfaced that the shared resolver silently dropped every service_name call site, because it accepted only quoted literals and a role var resolves to a bare key. And the literal-protocol lint could never be suppressed on a wrapped value, since autoformat moves the marker to the value's last line while the offence is reported on its first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A databases.csv row with database = '*' makes the backup dump a whole instance with pg_dumpall. The recover chain had no path for that file: it sorted cluster dumps out and printed SKIP, so a host whose engine is dumped that way could not be restored at all, and the drill refused to certify it.
Cluster dumps now go back through 'baudolo-restore cluster' with the instance's superuser. Three details decide the shape. A cluster row states '*' where a single-database row names a database, so credentials_of would file every instance under one key and keep the first, hence a lookup keyed by instance. Which applications must be down is stated by the dump's own connect lines and nowhere else, and reading them needs a real identifier parser: a quoted name may hold spaces and psql options precede the target. And the consumers of a cluster sit in the compose project of its engine, not in projects named after each database, so both are checked before anything is dropped.
The version gate moves out. baudolo compares the dump's header against the running server before its own pre-clean, so the copy here was a second implementation of the same rule, free to drift from the one that actually gates the replay. The test asserts the delegation negatively: the replay must not query the engine version itself. The pin follows, and baudolo-restore joins the command assertion.
The docker probing splits into utils/recovery/docker.py, which keeps the replay module under the size limit and gives the two halves one job each. Calls go through the module rather than through imported names, or a test patching the new module would not reach them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The drill removed each volume and re-created it. Docker refuses to remove a volume any container still references, and a container outside compose is only stopped here, not removed, so discourse's launcher container aborted the run at that point: after the host was already down and part of the volumes were gone. Re-creating a removed volume also drops its driver options, handing an NFS- or bind-backed volume back as a plain local one.
Volumes are emptied in place instead, from inside a throwaway container, and the emptiness is proven in the same call. Nothing has to be re-created, so a launcher container starts again exactly as it was built.
The engine that receives a replay is started by the service name on its own compose label. The previous fixed name existed for no engine on the host: the central Postgres serves project postgres from volume postgres_data, bigbluebutton serves project bigbluebutton from service postgres. Neither is called database, so the drill died at 'up -d database' after the teardown and before any restore.
A cluster dump yields one marker per database it recreates, or a replay that brought back only part of an instance would still pass. The drill runs once, in the sync pass; repeating it in the async pass would only tear the host down twice for the same answer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A backup generation stores databases as sql dumps, not as a file tree
(--dump-only-sql), and the recover chain only ever restored trees:
`git grep sql HEAD -- cli/administration/recover utils/recovery` was
empty. An operator running `recover full` after a real loss got their
files back, their databases untouched, and an exit code of 0.
Adds the missing leg as a fifth recoverer, planned right after the
volume steps so the dumps land once the volumes are back. Three
properties make it a restore rather than an attempt:
- It refuses while a consumer of that database runs. `--empty`
pre-cleans the schema in one psql session and replays in the next; a
booting consumer recreates it in between and the dump's own CREATE
TABLE fails. That is what turned CI red with `relation
"ar_internal_metadata" already exists`.
- The engine is read, not guessed: the service key in the consuming
role's meta/services.yml, the same value that fed the backup seed.
The two previous call sites each sniffed it from the container image
and could disagree.
- The dump's own header states which server it came out of, and that is
compared against the running engine before the pre-clean drops
anything. Forward across a major version is the upgrade path and
stays allowed; backward is refused. The version deliberately does not
travel in databases.csv: that file is written at seed time and would
state the version deployed back then, and a generation copied to
another host carries its header along while a csv row stays behind.
Mind the header formats, they are not symmetric - postgres puts
`Dumped from database version` on line 7, while mariadb's line 2 opens
with mariadb-dump's own version and only the tab-separated `Server
version` line names the server. Both are pinned as test fixtures,
captured from postgres:17-alpine and mariadb:11 themselves.
baudolo is untouched: nothing beyond the credentials it already carries
is read from databases.csv, and no upstream format changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The egress router scraped /etc/resolv.conf once and persisted whatever it found. Inside a container that is 127.0.0.11, docker's embedded resolver, which answered REFUSED to recursive queries in CI run 31927586060 and left the node with no clearnet DNS. The upstream now comes from networks.internet.dns, the same source lookup('container_dns') already gives the docker daemon, and an assert demands it before the role touches the host.
no-resolv, the clearnet server= lines and the /etc/resolv.conf pin now hang on TOR_DNSMASQ_OWNS_LISTENER. A dnsmasq the role does not own keeps its upstreams and the node keeps its resolver; only the onion routing is added there. Without that split a swarm node would end up with no-resolv, no upstream, and its resolver pinned at itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The realm carried exactly one client, the shared OIDC one, and no SAML client
existed anywhere in the repo - `flavor: saml` was in the resolver's enum with
zero consumers. Apps that cannot speak OIDC therefore had no way to
authenticate against Keycloak at all; they could only be fronted by the
oauth2-proxy, which authenticates at the edge and leaves the app itself without
a session.
KEYCLOAK_SAML_APPS collects every deployed app whose services.sso.flavor is
saml. It filters stepwise - value.services, then .sso, then .flavor, each
guarded by `defined` - because Ansible's undefined is a Marker(StrictUndefined,
Tripwire): resolving that chain in one selectattr raises for every app without
an sso block instead of yielding undefined, which would have broken the realm
render for every deployment in the repo.
The include is unconditional and the loop emits a leading comma per client, so
the realm needs no branch on the list being empty. That matters: an Ansible
`>-` value that failed to convert would be a non-empty string and a truthiness
guard would then splice a broken clients array.
The client's shape is not guesswork - every attribute comes from a failure
reproduced against a live Keycloak 26.7.1 until a SuiteCRM SAML login completed:
defaultClientScopes is emptied because the built-in role_list scope emits one
Role attribute per realm role and the SP rejects the assertion with "Found an
Attribute element with duplicated Name"; saml.client.signature is false because
the SP does not sign its AuthnRequests; the username mapper's attribute name
comes from OIDC.ATTRIBUTES.USERNAME rather than a hardcoded claim.
The new test renders the complete realm - every include, the role's own
ldap_groups_filter, StrictUndefined - and parses it as JSON for zero, one and
several SAML apps. The file had no test before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A clone and a row an earlier one already embeds prove nothing the sweep does
not already prove, but both still took a slot. Drop them when the regular line
is chunked, so the budget buys unique coverage only, and mark them in the plan
table with a status of their own instead of letting them read as 'beyond the
budget'.
A priority row is exempt: an operator who pins a clone asked for that deploy,
and dropping it would be the silent withdrawal 6605c7ef9 just removed.
The cut sits in chunks_of, not in entries_of, so the plan still lists every
candidate while no chunk can see one. 55 of 254 rows are cut on this branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The regular line was queried with selection.names(leading) as its blacklist,
and names() yields role ids only -- so pinning one variant in the priority
line withdrew every variant of that role from the sweep. Run 31927586060
deployed nextcloud only as #10 and #5; the other twelve variants existed
nowhere.
Blacklist only the tokens that name no variant, and drop the pinned
(role, variant) pairs from the regular rows afterwards. The variant-level cut
cannot go into the query filter: its tokenizer has no '#', so an axis token
aborts the parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
