The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
The lint shipped in the previous commit had three defects, all reproduced in a
throwaway copy of the tree rather than argued.
Its documented opt-out could not be used. The docstring said to put
`# nocheck: playwright-env-template` on the first line of the role's spec.
Every spec is a CommonJS .js file, where a leading `#` is a hard parse error --
`node --check` refuses it. Following the instruction broke the file the lint
was protecting. It is `//` now, and the marker may sit anywhere in the head
rather than on line 1, which the mandatory `require` already occupies.
Its suppression check was a bare substring match: `if _RULE in
read_text(...).split("\n", 1)[0]`. Any prose containing the rule name disarmed
it -- proven by deleting a template and writing
`// TODO: someone still owes this role a playwright-env-template`, which the
lint then passed. A typo in the rule name disarmed it just as well. It now uses
`is_suppressed_in_head` from utils/annotations/suppress.py, the shared grammar
every other lint here consumes; this was the only one with a hand-rolled parse.
Its scope followed test_has_spec's `web-app-` prefix, but the mechanism it
guards does not: discover_playwright_roles.py globs
`rglob("templates/playwright.env.j2")` with no prefix filter. Six web-svc-*
roles ship specs. All six carry a template today, so the hole was latent, but
consistency with a narrower sibling is not correctness.
Verified in the copy: missing template -> fails; bare mention of the rule name
-> still fails; a real `// nocheck` in the head -> passes, and node --check
accepts the file. In the tree: tests/lint + tests/integration 532 passed,
1548 subtests, and the lint itself green at 82 specs against 82 templates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_has_spec.py forces every roles/web-app-* to ship a Playwright spec and
calls a missing one a hard error, because the persona contract must never be
silently absent from the deploy capstone. Nothing forced the other half. The
runner discovers roles by rglob('templates/playwright.env.j2')
(discover_playwright_roles.py:53), so a spec without that file is staged
nowhere and executed never.
21 of 76 web-app roles were in exactly that state. Verified rather than
inferred: running the discovery filter directly returns 61 roles and none of
the 21 is among them, `make compose-playwright` refuses without a rendered
.env from a prior deploy, and `git log --all` on those paths shows the file
never existed on any branch -- the single hit is web-app-gitlab on a fork
branch. Their specs have never run once. 15 of the 21 deploy in CI today,
reporting green while their only functional verification never fires.
The new lint mirrors test_has_spec and carries a per-role opt-out that is
used nowhere: uniform beats exceptions.
Every one of the 21 now has a template written against its own
meta/services.yml rather than a shared skeleton -- espocrm splits
ADMIN_PASSWORD from ADMIN_NATIVE_PASSWORD because OIDC_ALLOW_ADMIN_USER
leaves both login paths open, the four permanently auth-less roles declare
their personas blocked instead of shipping a knowingly-red spec, and
web-app-mastodon gained a real test-seaweedfs.js because no honest
suppression existed for a role that does route Paperclip to S3.
web-app-magento moves pre-alpha -> alpha here: it now has a spec that runs,
which is what the lifecycle gate is about.
Six meta/services.yml files change only where adding a template falsified an
existing suppression -- gitlab and shopware both claimed the role 'ships no
templates/playwright.env.j2', which stopped being true in this commit.
Not resolved here, and the adversarial review says so plainly: the specs are
enabled, not proven. None has executed against a live stack. The review found
three classes of defect that land in follow-up commits rather than being
smuggled in silently -- persona blocks that are missing or should be tied to
sso.enabled, eleven suppressions naming a sys-svc-mail test suite that does
not exist, and eight claiming a CSP assertion that cannot fire because no
consumer role renders CSS_BASE_URL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defines what a services.mcp block may declare and enforces it: classification,
transport, auth, implementation and the provenance fields that make an upstream
claim checkable. The lint refuses an unknown key anywhere in the block and
refuses a provider whose allowed consumers cannot speak its transport or its
auth scheme.
Discovery resolves each provider's credential and endpoint per consumer and
per deployment, rejecting a server with a named reason. Transport, auth and
endpoint faults are fatal; a missing credential is not, because it also occurs
when the provider is simply not provisioned yet in the same play.
The filters render the Authorization header from the declared scheme, build
Open WebUI's tool-server connections with their group grants, and derive the
members of those groups from the RBAC assignments. Role scoping gains
application-scoped roles, so an mcp role only counts when it is granted on a
specific application.
roles_with_service gains a scope and a direction filter, and drops the canonical
domain requirement for direction-filtered entries: an MCP URL is built from the
service key and the internal port, so a provider without a public vhost stays
valid.
rsync -b turns a deletion into a rename, so a file the source dropped between
two writes of the same generation survives as name~ beside the real one and is
restored into live data. baudolo 3.4.1 fixes the copy that caused the
elasticsearch restore failure; this is the sweep over the sibling call sites
here.
svc-bkp-remote-2-local is the live one: the rsync command is assembled once and
re-executed up to twelve times against the same destination, which is created
with exist_ok before the loop starts. A retry after a partial transfer is
structurally the same second write as baudolo's hot/cold pair, only triggered by
--timeout instead of a container stop. That path pulls other hosts' backups, and
retries are its normal case rather than an exception.
svc-bkp-local-2-device cannot fire today - it writes one pass into a directory
named after the current second - but mkdir(exist_ok=True) makes a re-entry the
same two-write case, so the flag goes rather than the argument for keeping it.
The integration test mirrors the production command and follows.
The lint is what makes this complete: it scans every tracked .py/.sh/.j2/.yml
for a backup flag near an rsync token, and it found the integration test that
the manual sweep had missed. It excludes exactly one file, its own, because the
rule has to name the token it forbids; the sibling guard for || true avoids the
same problem only by scanning .sh alone.
Verified: make quality green across all four targets, the ssh-wrapper
integration test passes with real rsync, and test-lint goes 307 -> 308 with the
new guard collected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role shipped a complete probe script at files/test.sh and no
templates/test.env.j2. discover_cli_roles globs the env template alone,
so the role was never discovered and the script had never run — and the
script hard-requires TOR_SOCKS and NGINX_SERVERS_DIR via ${VAR:?}, so it
could not have run without one either. Its own usage hint pointed at a
path that does not exist.
Moving the script to the discovered path and adding the env template
makes both halves line up. Verified against the live stack: the onion
answers through the Tor SOCKS proxy, 1 probed, 0 failed.
The new lint keeps the pair together and rejects a script left at
files/test.sh. The two halves fail differently and only one is loud: an
env template without a script aborts the play, a script without an env
template is never discovered and its silence reads like a pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rsync -b turns a deletion into a rename, so a file the source dropped between
two writes of the same generation survives as name~ beside the real one and is
restored into live data. baudolo 3.4.1 fixes the copy that caused the
elasticsearch restore failure; this is the sweep over the sibling call sites
here.
svc-bkp-remote-2-local is the live one: the rsync command is assembled once and
re-executed up to twelve times against the same destination, which is created
with exist_ok before the loop starts. A retry after a partial transfer is
structurally the same second write as baudolo's hot/cold pair, only triggered by
--timeout instead of a container stop. That path pulls other hosts' backups, and
retries are its normal case rather than an exception.
svc-bkp-local-2-device cannot fire today - it writes one pass into a directory
named after the current second - but mkdir(exist_ok=True) makes a re-entry the
same two-write case, so the flag goes rather than the argument for keeping it.
The integration test mirrors the production command and follows.
The lint is what makes this complete: it scans every tracked .py/.sh/.j2/.yml
for a backup flag near an rsync token, and it found the integration test that
the manual sweep had missed. It excludes exactly one file, its own, because the
rule has to name the token it forbids; the sibling guard for || true avoids the
same problem only by scanning .sh alone.
Verified: make quality green across all four targets, the ssh-wrapper
integration test passes with real rsync, and test-lint goes 307 -> 308 with the
new guard collected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
autoformat joined the two operands of the message concatenation during
make quality; 1408c6770 had them on separate lines. Formatting only, same
expression and same output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule checked whether both keys contain the token `omit`. That catches the
Ollama shape, where async carried no omit at all, but it does not enforce what
the failure message claims: two keys can both carry omit and still hang on
different switches, and the old check passed that.
_gate() now reads the switch each key states inline - the test of an
`if ... else omit`, or the left side of a `| ternary(..., omit)` - and the two
must match. A key that names no gate matches another that names none, which is
the shape sys-service and the canonical ASYNC_TIME/ASYNC_POLL pair use; both
then inherit whatever their variables do. An unrecognised omit-bearing form
falls into a single bucket, so it only matches another unrecognised form.
Checked against all five shapes in the tree: the pre-fix Ollama pair is an
offender, the fixed pair and the ternary and bare-variable pairs are not, and
two different conditions are now an offender where the token check passed them.
The rule still does not catch two ungated keys whose variables diverge. That is
deliberate: group_vars makes ASYNC_TIME evaluate to omit while ASYNC_POLL stays
numeric, and a poll without async is ignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pre-pull task set `async` unconditionally while gating only `poll`. With
async disabled, `poll` became `omit` and ansible fell back to its built-in 15s
poll while `async` still routed the task through async_wrapper: ansible
collected the job itself and then ran `async_status mode=cleanup`, deleting the
job file that tasks/utils/async/reap.yml expects to read. The reap reported a
successful `ollama pull` as a failed async job, failing web-app-nextcloud 2 in
run 30781723102.
Both keys are now switched off together, matching all six peer call sites. On
the resulting synchronous path the reap's `async_reap_tolerate` list no longer
applies, so the tolerated "up to date" answer is carved out in `failed_when`
instead. It keys on `failed` rather than `rc`, so a module-level failure that
carries no rc is not swallowed.
async_failures._which() now honours loop_control.loop_var: the fired result
records the chosen name under `ansible_loop_var`, so a batch failure names
`smollm2:135m` instead of the bare job id.
A second lint case asserts that async and poll carry `omit` together - the rule
this call site was the only one of seven to break.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A service without a healthcheck.flavor or an explicit test only breaks
while its compose file renders, one role at a time during a deploy.
Replay the lookup's own validation against the static config so the
failure surfaces in CI, and reject declarations the lookup would refuse
anyway: an unknown flavor, or a flavor and a test fighting over the
same entry.
Both tests share _probe_defect and import PROBES from the same SPOT the
lookup uses, so the known flavors cannot drift apart. 49 of the 56 call
sites resolve statically; the rest set service_name in an included base
template and are skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications registry is built before any role-vars binding, so a
`{{ SOME_ROLE_VAR }}` in meta/*.yml stays an unrendered Jinja string and
silently resolves to nothing. The sibling rule already covers
`application_id`; this extends it to every key declared under
`roles/*/vars/`.
String literals, keyword arguments and filter or attribute names are
stripped before matching, so `lookup('domain', ...)` and
`lookup('rbac_group_path', application_id=...)` are not mistaken for
variable reads. Scope stays at meta/*.yml: files under meta/addons/ are
rendered inside the role's own play, where role vars are bound.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`shell` populates stdout on every path where changed_when is evaluated,
so the default filter guarded nothing: a non-zero exit and even a chdir
failure both carry stdout, a skipped task never evaluates the condition,
and on a module failure Ansible records the conditional error in
changed_when_result while still reporting the real cause.
The async lint walked the tree itself instead of going through the
shared cache. Both selections cover the same 1079 files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An async task is a deployment wide decision taken in one line. Whether it
holds depends on things the line does not show: in swarm a delegated
command crosses a node boundary and runs far longer than the same command
in compose, so a window that never opens on one node routinely opens
across three, and the playwright run starts the moment the play ends, so a
command still writing in the background surfaces as a flaky spec rather
than a failed task.
Require one sentence per async naming what does not consume the result.
"It is slow" is why async is tempting, never why it is safe, and the
docstring says so. There is no nocheck escape: a task that cannot be given
the sentence has not earned its async.
The comment lint learns `rationale:` as a marker so those lines are not
read as narration. It is deliberately narrower than the exception marker
next to it: a rationale is demanded by another lint that refuses the
construct without it, so it cannot be self granted, which is exactly the
failure mode of comments that explain themselves.
This lands after the change it describes, so every commit stays green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a blocker vocabulary so a role whose upstream MCP path this
deployment cannot use says why instead of reporting an absence. Without
it OpenProject reads as "upstream has no MCP", which is false: the server
ships, it is Enterprise-only. A blocker and a served surface are mutually
exclusive, so a role cannot claim both.
Add endpoint.key_credential and endpoint.suffix for servers that address
a session through the URL path. Baserow streams from
{base}/mcp/{key}/sse, and endpoint fields must stay literal, so the
composition belongs in the discovery layer. A server that declares a path
key without a resolvable value is dropped rather than handed to clients
half-built.
Carry the upstream provenance in the role block, so the audit derives it
instead of keeping a second copy.
Require an explicit mutating_tools_enabled: false for a privileged
auth_subject; the rule previously only fired when the key was present and
true, so omitting it slipped an administrator-scoped surface through.
Move the MCP filters into plugins/filter/mcp/ so they stop sharing a name
prefix with their siblings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint walked each role's tasks directory itself, once per role. The
project's own test_no_raw_project_walk rejects that: utils.cache.files
shares a single filesystem walk and one read per file across the whole
pytest session, so 251 separate directory traversals are 250 too many.
Roles are now identified from the relative path of each yielded file
instead of by iterating the roles directory. The scanned set is unchanged
at 806 files, with empty differences in both directions, and the
unresolvable-target injection still produces the expected finding.
main.yml now reads as the chronological sequence it describes: it includes
01_fullstack.yml, then the single IS_STACK_HOST-gated 02_manager_ops.yml,
whose substeps live in the matching 02_manager_ops/ directory numbered in
execution order. The plugin pipeline moves under its parent step so its
include resolves against its own directory instead of an inherited search
root.
The flush that deploys the stack moves out of the config step to the top of
02_manager_ops.yml. It previously ran after the swarm resolver, which then
waited 600 times for a service that did not exist yet and killed two CI jobs
with "no such service: nextcloud_nextcloud". The second flush and the
re-resolve behind it are removed: neither included file notifies a handler,
so they could never fire.
A new lint pins what made this refactor risky. A relative include is resolved
at runtime, so a typo or a moved file survives ansible-lint and
--syntax-check and only fails mid-deploy; worse, a bare name that matches
under two search roots silently shadows one file with the other.
Comments that only narrated code are dropped from the addon hooks and the
plugin install step.
apply is a parameter of the include_tasks action, not a task keyword. Written as a sibling of the module Ansible sees two action statements and aborts: 'conflicting action statements: include_tasks, apply' at 08_manager_ops.yml:9. All five delegated includes from a963280c1 carried it that way.
Nothing static caught it. ansible-lint passes the file, and ansible-playbook --syntax-check - which scripts/lint/ansible.sh runs and make quality-high covers - reports 0 failures against the broken tree, because a task file reached through a dynamic include_tasks is first parsed at runtime. The play therefore died 26 minutes into the deploy, after everything before it had run.
The new lint reads every task file on its own and rejects apply beside a module, which is never valid. Verified both ways: the sibling form is reported, the nested form is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_any_bridge_partner_deployed matched a bridge against deployed role ids by equality or '-<bridge>' suffix. That reaches a partner whose entity carries the name (matrix -> web-app-matrix) but never one registered under a service key: 'sso' is provided by web-app-keycloak and 'ldap' by svc-db-openldap, neither of which ends in the bridge name. Those five declarations therefore evaluated to false in every run, so their addon flags were false everywhere and skipUnlessAddonEnabled skipped the specs in every variant - silently, because a gate that never opens looks exactly like a variant that does not apply.
The lookup now resolves a bridge through build_service_registry_from_applications first and keeps the entity-suffix rule as the fallback, so both spellings work. Existing bridge tests are unaffected: their applications stub yields an empty registry and takes the old path.
The new lint asserts that every bridges: value names a service some meta/services.yml declares. It fails on a typo or an invented name while staying agnostic about which run deploys the service, which is the lookup's question at runtime. Verified both ways: no findings against the real service set, thirty against an empty one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Exclusion was a property of the image: --images-no-backup-required resolves through volume_is_fully_ignored, which skips a volume only when every container using it is ignored. A container holding a derived tree beside state that must be kept could not express the exclusion at all.
matrix_mdad_docker is that case - the data root of a nested docker daemon, sharing one container with matrix_mdad_matrix and matrix_mdad_state, which carry the actual Matrix data. Copying it is not merely wasteful: overlay2 encodes a deletion both as a character-device whiteout and as a trusted.overlay.* xattr, and no rsync in the chain carries -X, so the copy cannot be restored faithfully. The whiteouts also cannot be written onto an overlayfs destination at all, which is what kept the web-app-matrix DR drill red for 58 minutes a round.
meta/volumes.yml gains a 'backup:' key, validated like 'nfs:'. The backup_volume lookup resolves every 'backup: false' entry to its pinned docker name and feeds baudolo's --volumes-no-backup-required. A lint requires the decision to be written down for any volume mounted at /var/lib/docker or /var/lib/containers, so the next docker-in-docker role cannot repeat this silently.
Requires backup-docker-to-local 3.3.0, which carries the flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The deploy CLI shipped every mode as -e KEY=value, which ansible-playbook parses as a string at the highest precedence. MODE_DEBUG=false therefore arrived as the non-empty, and so truthy, string false. Every when: MODE_X | bool was unaffected because the filter maps it back, but four sites read a flag bare and silently inverted: openldap wrote LDAP attributes with state exact on every deploy instead of only under MODE_CLEANUP, SYS_SERVICE_DEFAULT_STATE was restarted where it should have been omit, SYS_SERVICE_RUNNER_STATE was never started, and mailu always logged at DEBUG.
Send the modes as one JSON extra-var, which keeps the declared type, and give those four sites the | bool they were missing so the meaning of a flag no longer depends on how its value reached Ansible. A lint pins the rule: a MODE_* read in a truthiness position carries the filter or a nocheck naming a genuine string comparison.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A module that dies on an unhandled exception carries no msg: the AnsiballZ wrapper returns only failed and exception, whose event holds the sole account of what went wrong. The aggregation defaulted to the literal string package install failed and discarded it, so an AUR fetch that threw in run 30713666732 reported nothing at all and took a full investigation to place. Walk msg, exception.event.msg, exception, module_stderr, stderr instead, and tag every result with the module that produced it. This applies to apt, dnf, pacman and every other module the plugin drives, not only to the AUR.
The aur module fetches the RPC record and the snapshot tarball through bare open_url calls with no error handling, so one blip kills a deploy; distribution repositories need none of this because apt, dnf and pacman retry inside their own transport. Give the calls that reach an endpoint of their own a single shared policy. Only the AUR path is declared today; copr, ppa and build carry it against the day they are.
Split the result along its responsibilities so no file passes 250 lines: calls.py holds the vocabulary, sources.py one builder per acquisition path, plan.py only the dispatcher, outcome.py the reporting and facts.py the fact shapes. The action plugin is left with executing modules. Tests follow the code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve conflicts, resolved by intent rather than by side.
Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.
web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').
handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.
svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.
Six follow-on failures the merge produced, fixed at the root:
- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
the prefix lint forbids; the pair now lives in utils/github/variant/ as
bundles.py and bundle_limits.py, with the -m entry point, the ruff
per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
three bare timeout literals; both route through gotoOnion and resolveTimeout
now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
distros that ship docker without it, the loop device is already detached when
the probe failed before attaching, and modprobe zfs may fail because the
/dev/zfs check decides.
make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three violations in test files written earlier today, all surfaced by the first
full quality run rather than by the targeted checks I had been running.
test_addons_vendored_files built its finding list with append in a loop.
test_env_handlers_registered walked the tree with rglob, bypassing the shared
project cache the lint exists to protect on a 250+ role tree.
test_resolve read two keys that resolve.sh writes per matrix entry into
GITHUB_ENV, which can never be .env entries; the second key now derives from the
first so a single line carries the marker - and stays short enough that ruff
format leaves it on the line the lint inspects.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
build_env() iterates an explicit registry, so a handler module that nobody
imports never runs and its key is silently absent from the generated .env -
the consumer reads an empty value and behaves as if the feature were off.
Nothing errors, which is what makes the omission survive review.
The check parses the registry with ast rather than importing it, because one
handler pulls a third-party dependency at import time and a registry check
must not depend on the environment it describes.
It immediately found variant_bundle/max_storage.py unregistered, so
INFINITO_VARIANT_BUNDLE_MAX_STORAGE never reached any .env although
utils/github/variant_bundles.py reads it; registering it lifts the generated
file from 129 to 130 keys.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The deploy died in the drill's second pass on `wp plugin install activitypub
--activate`: the first pass had installed it, the DR restore put the tree back,
and wp-cli refuses to unpack onto an occupied directory. `--force` would have
made that green by overwriting whatever the restore produced - on a branch about
NFS-backed volumes, the worst possible place to paper over a restore. The probe
keeps it red instead: `wp plugin is-installed` answers from `get_plugins()`
alone (verified in the pinned wp-cli phar: `Fetchers\Plugin::get()` iterates
`get_plugins()` and nothing else), so a tree WordPress cannot parse still
reaches the install and still fails on folder_exists.
Activation moves into its own step. Chained to `install --activate`, wp-cli sets
chained_command and skips the result check, so an activation failure returns 0
today.
The same install path now serves both call sites; the addon hook carried its own
copy, which is why one defect needed fixing twice.
mu-plugins follow the declaration the way discourse already does. The loop reads
`mechanism: mu_plugin` from meta/addons/ instead of spelling two filenames, which
is how `infinito-http-ca-trust` shipped undeclared while `infinito-oidc-rbac-mapper`
carried a `required: true` that nothing evaluated. Both install paths honour that
field now - a required addon aborts the play, an optional one warns and is
skipped - which is the first place in the repo to read it.
Three guards, because the loop replaced two guaranteed literals: an assert that
every required addon is claimed by some install path, a lint pairing each
vendored addon with its shipped file in both directions, and the spec the addon
contract demands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three of the failures triaged today were a single unretried external fetch:
deb.debian.org went dark for 76 seconds on one node while three other distro
rounds of the same job built the same image; packagist answered 502 for two
metadata files while the identical build passed thirteen minutes later; and
ghcr.io timed out on a manifest HEAD for six minutes.
Both build tasks were the only network-bound tasks in their file without a
retry, and their in-shell fallback made a transient fault worse: it re-ran
immediately, with no wait, and threw away the cache it had just filled. The
ladder now waits 30 seconds between three attempts, and the cache-discarding
rebuild moved out into its own task that runs only once those are exhausted -
at which point the transient explanation is spent and a corrupt cache is the
next candidate. Three attempts rather than ten because ten would run 36270s
against a 21000s step cap.
svc-registry-docker is excluded from the registry sync - it cannot mirror
itself into its own registry - which left it the one role fetching from a public
registry through the swarm agent, single-shot, where every other role gets ten
tries. It now pre-pulls through a retried task, gated on the sync task having
been skipped so the exclusion keeps one source.
The accompanying check computes the ladder's worst-case wall clock from the
handlers and measures it against default.env and the workflow step cap rather
than restating the numbers, and it fails if an ignore_errors ever loses its
paired fallback - which would turn a failed build into a silent pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The summary named the filesystem that was picked, never the one the run got.
A host that declines - no root, missing tooling, no /dev/zfs - still deploys,
and nothing recorded on what, so a red run could not be reproduced.
docker_dataroot.sh now ends every path with a verdict that measures the data
root with `stat -f -c %T` and reports applied, declined, already-prepared,
unchanged or required-but-unavailable. Compose and host write it straight into
the step summary; swarm runs the script inside each node, where that variable
does not exist, so the playbook lifts the verdict line into the deploy log and
report.sh aggregates it afterwards.
`hostname` is resolved with a fallback: it is not installed by default on the
RedHat family, and under `set -euo pipefail` a bare command substitution would
have turned the decline path - which exits 0 - into a hard failure.
test-deploy-host.yml gains the `filesystem` input. single-node has been passing
it since the per-matrix selection landed, and a reusable workflow that receives
an undeclared input fails the whole run with a startup_failure whose only trace
is a banner on the run page. The single-node line was skipped in the run that
would have shown it. A lint now pairs every local `uses:` with the callee's
declared inputs, and also demands that required inputs without a default are
supplied.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The compose and host deploy jobs tee the whole deploy stdout into the file
ANSIBLE_LOG_PATH names, and both downstream steps parse that stream. Only
stdout carries the host-side "matrix-deploy: round N/M ... PASS n" markers,
so it is the sole source the per-variant runtime tables can come from.
dedicated.sh's cleanup then copied the container-side ansible log onto that
same path. docker cp unlinks and recreates the destination, so tee kept
writing into an orphaned inode and the visible file became ansible's own
log_path output — whose "<ts> p= u= n= LEVEL| " line prefix defeats the
anchored recap matchers in logparse.py while the unanchored has_sections
substring test still fires. in_roles_recap never flips, every role-time line
is skipped, and the step dies with "no profile_roles entries found" although
the deploy succeeded. Run 30578244959: 14 of 14 compose+host jobs.
Why it looked like a flake: docker cp cannot read a /tmp that is a tmpfs
mounted inside the container, and the Debian image enables systemd's
tmp.mount while CentOS Stream 10 does not. The copy silently failed on
debian (host file survived, jobs green) and landed on centos (host file
destroyed). feature/swarm-nfs never had the block; it returned here through
merge drift from main, where the copy is still load-bearing because those
workflows do not tee.
The copy is dropped: stdout is a strict superset of what it provided, and
the in-container verifier reads the log inside the container. Alongside it:
the swarm workflow's file-exists guard is adopted so an aborted deploy does
not paint the summary step red on top; the log_path prefix moves to
utils/ansible_log.py and is applied in logparse.py, so an ansible run log
parses instead of yielding an empty summary; a lint pins that nothing writes
onto ANSIBLE_LOG_PATH again; and the parser regression test now covers the
recap headers, not just the role rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CI deploy bootstrap hand-listed two packages and let the ubuntu-latest runner image supply the rest, so a new host-side import surfaced as a red deploy job instead of a failed install. Size [project.dependencies] to what runs outside the project image and move the deployment stack into a deploy extra, then have every install site read that instead of naming packages itself: the image build and make install-python take the deploy extra, the CI host and the mirror and instructions workflows take the base.
Declare cryptography, which utils/manager/value_generator.py imports directly and which only reached the environment transitively through ansible. Drop numpy, which nothing in the repository imports. Guard the boundary with a lint test that blocks every deploy package and asserts the runner-host CLI still builds its parser.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, none resolved by picking a side wholesale:
- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
rc now counts as failure; the previous default(0) let an aborted Playwright
run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
which this branch's lint enforces, and adopted the incoming retry-inside-poll
for the OIDC button plus its networkidle wait.
The sweep's own governor was sized against the 6h platform kill (19800s) while the deploy step was capped at 300 minutes, so it could never fire: GitHub hard-killed the step mid-distro and reported a run that simply did not fit all distros as a job failure.
Size the ladder instead, 10 and 5 minutes apart: sweep budget 340min < deploy step 350 < job 355 < platform kill 360. The swarm per-leg ceiling moves to 350 so it stops matching the job cap, where it was dead code. The rescue-state dump drops from 10 to 5 minutes so it cannot alone consume the window between the job kill and the platform kill; measured teardown is 10-38 seconds.
A new lint asserts the budget stays under the governing step cap, the lower half of the invariant whose upper half the headroom lint already carried; that lint's ceiling moves from 345 to 355.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Open WebUI talked to Ollama directly. It now speaks the OpenAI-compatible surface of svc-ai-litellm with a per-consumer virtual key, so a second backend or an external provider becomes a gateway config change instead of a role change. A lint keeps future roles off the model servers.
The gateway advertised a hardcoded model list (ollama/llama3, ollama/nomic-embed-text) while the matrix variant preloads smollm2:135m, so it offered only models the backend did not have and hid the one it did. The list is derived from the ollama preload SPOT now.
The chat spec followed the rename: it gates on litellm instead of the removed ollama service flag, and picks the first non-embedding model from /api/models rather than pinning a variant-specific name, so it exercises the chain rather than a fixture.
Routing openwebui through the gateway pulls both model backends into the closure of every role that consumes it, which pushed nextcloud variant 2 over the memory budget. The budget stays at 32/64 GB; instead the matrix variants of svc-ai-ollama and svc-ai-lmstudio now declare the size their test model needs (1g/2g) while the base keeps the 6g/8g that llama3 and mistral require. The budget guard only saw half of that: it replaced the evaluated role with its variant but counted every dependency from meta/services.yml, a combination no matrix round deploys. It now evaluates every variant-carrying role at the same round index, falling back to variant 0, the way the deploy planner does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test-environment.yml and test-guide.yml become test-workspace.yml and
test-instructions.yml. Orchestrator job ids, the distros-SPOT matrix
lint, the act example, the diagnostics slug and the rescue artefact
names follow.
Replace the roles_only input with per-job workspace/instructions
tri-state inputs (auto|true|false). 'auto' derives the former
roles_only skip straight from whitelist: both jobs run on a global
scope (whitelist empty or __ALL__) and are skipped on a role-scoped
run, which keeps the PR behaviour identical because the diff resolver
never emits an empty whitelist. entry-manual exposes both as choice
dropdowns defaulting to false, so a manual run opts in explicitly.
The now redundant roles_only output is dropped from
pr_affected_roles.sh and subset_roles.py.
Branch-protection required-status-check names change along with the
job renames.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The five images in test-environment.yml were a verbatim copy of the
dev_runtime_image field of every distro in meta/distros.yml. A prepare
job now reads them through utils.distros and publishes a JSON array that
the matrix consumes with fromJson, so adding or dropping a distro no
longer needs a workflow edit. The resolver runs on the bare interpreter
because utils/distros.py is stdlib-only.
An empty list aborts the resolver: GitHub skips a job with an empty
matrix silently, which would leave the run green with nothing tested.
The lint could no longer compare literals, so it walks the wiring
instead: axis expression to needs edge to job output to step id, then
runs that step body the way the runner does and compares what it writes
to GITHUB_OUTPUT against the SPOT. The Act example in the iteration docs
was the last remaining literal and now reads the SPOT too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The env generator runs on the bare bootstrap python, before any
dependency is installed. utils/distros.py reached PyYAML through
utils.cache.yaml, so `python -m cli.meta.env` died with
ModuleNotFoundError on every dev-runtime image that ships python3
without PyYAML.
utils/yaml_bootstrap.load_block reads the block-YAML subset the SPOT
files use with stdlib only. It raises BootstrapYamlError on anything it
cannot parse identically to yaml.safe_load, so meta/distros.yml fails
loud instead of drifting.
The per-line regex lint meant to guard this is gone: one hop of
indirection defeated it and it stayed green while the invariant was
broken. It is replaced by a test that runs the real generator with
every non-stdlib import blocked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
