Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.
The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.
The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.
Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".
The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.
Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.
The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.
nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RFC 6761 reserves .test for testing and .example for documentation, and the
project's own domain was doing both jobs under the documentation name. It is
what default.env hands every local deploy and what the test fixtures assert
against, so it is a testing name.
Rename it wherever it stands for the running machine: the environment
defaults, the development inventory, the workspace and trust scripts, the test
suites, and the four documents that describe real behaviour - the DNS mapping
to loopback, the browser CA trust, WSL2 resolution. Those documents would have
started lying the moment default.env changed.
The sandbox allowlist moves with them, since egress is matched per pattern and
nothing resolves the old name any more, and the sandbox document that mirrors
that list follows it.
Two places keep .example on purpose: the link-style sample in the
documentation guide, which is documentation about documentation, and every
generic fixture name elsewhere in the suite - a.example, foo.example and their
kin. Those are a larger sweep of their own and are not touched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
32dd66099 gave web-app-jenkins the third consumer its neighbours already
had, and the report reads that list from the same mcp_consumer edges the
derivation does. One cell.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.
The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.
The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.
Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.
Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bb975843f1 raised web-app-gitea to 1.27.3 in meta/services.yml and left the audit
table at 1.27.2, so test_mcp_audit_report.py has been failing since:
AssertionError: audit.md differs from a fresh generation.
Regenerate and commit it by running audit.gen.py from the repository root
Regenerated, nothing else. audit.gen.py:54-62 reads the version out of each role's
own meta/services.yml and makes no network call, so the file is a pure projection
of the tree and drifts only when a version changes -- this is real drift, not a
flaky check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.
The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.
BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instructions test held a job of its own that picked one random role per
run, so a README's Production block was proven roughly once per 265 roles.
The replay now rides on a deploy row instead: no extra runner, and every role
the sweep touches gets its documented one-command deploy exercised.
The row is the role's smallest variant -- fewest services enabled -- because
that is the shape the README describes, and it is chosen among the variants
the sweep actually deploys, so a row cut as redundant coverage cannot silently
drop the role from the test. A role qualifies only when its README carries a
Production block and the guide's own mode is one the role runs and tests.
The marked entry carries the guide mode in `instructions` and a 📖 in its job
title, between the variant number and a priority ⭐. The guide mode is
independent of the mode the rotation gave the row: a swarm row still replays
in compose. The step runs last in the deploy job, after the swarm teardown,
because the replay deploys the role a second time and needs the runner's
ports.
call-test-instructions.yml, cli.meta.ci.guide_select and the `instructions`
dispatch input go with it. `redundant()` moves next to the marker so the
chunker, the plan table and the marker share one definition.
The plan table trades its 📊 Weight column for 📖 Instructions; nothing read
the weight any more, so it leaves the matrix entries too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container_address lookup does not return a container name in swarm. It
returns a shell fragment, "$(/usr/bin/resolve-container-id <stack> <service>)",
which is only a name once an interpreter evaluates the substitution. The shell
module provides one. The command module does not -- it execs the binary directly,
so the fragment arrives at container exec as literal text and the call dies with
"No such container: $(...)".
Compose hides the entire class: there the same lookup yields a bare name and
command: works. The failure exists only in swarm, and no swarm run had touched
these paths -- both were added by feature commits (b23653adf8, 0421ebeb16) that
wrote the same argv shape independently, which is the argument for a rule rather
than six edits.
Six call sites moved to the shell module: the moodle egress task, the three
moodle AI-gateway tasks, and the two wordpress AI-Engine tasks. The wordpress
pair keeps its stdin, so it uses the dict form with cmd:. Moving off argv exposes
the surrounding values to word splitting, so they now go through quote -- the
address itself must NOT, because single quotes would suppress exactly the
substitution this depends on.
test_argv_exec_needs_shell.py closes the gap that let this through.
container-exec-resolver checks that the resolver is used; it has nothing to say
about whether anything can evaluate the result. The new rule checks that other
half. It reuses the sibling's argv parser, which recognises only container/docker
in first position, so an argv list that does start an interpreter
(argv: [bash, -lc, ...], as joomla does) is correctly not matched.
Run against the tree the rule reported exactly the five sites found by hand plus
bigbluebutton, and passed over joomla and the discourse task, whose address is a
config lookup rather than the resolver. Bigbluebutton is pinned to compose via
compose_mode_force and can only ever see a bare name, so it takes a nocheck.
Five container-exec-resolver markers disappear with the argv form: under shell
the resolver is the ordinary path, not an exception.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1f870b1e5 left two marks the layout rules reject. friendica's utils/ held
patch_config.yml beside patch_db_credentials.yml, which is a shared
delimiter-bounded prefix, and its numbered files ran 00, 01, 02, 04, 05 with
nothing at 03 after the config patch moved out of the sequence.
The prefix becomes the folder it was standing in for, the way 8e8dbbd08c did it
for the AI files. All four roles move, not only the one the lint could see: the
path post_deploy.yml resolves a role's declaration through has to read the same
everywhere, so a folder in one role and a prefix in the other three would be no
domain at all.
utils/patch_db_credentials.yml -> utils/patch/db_credentials.yml (4 roles)
utils/patch_config.yml -> utils/patch/config.yml (friendica)
The numbers close up behind the file that left: 04_addons and 05_user_bootstrap
become 03 and 04.
Four references follow: the include in 01_manager_ops.yml, the lookup in
post_deploy.yml, the caller path in the unit test that reads each role's
declaration, and a requirement doc that linked the old addon file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
025 closes baserow, gitlab and the public-exposure limits. The gitlab
entry replaces the survey's Premium-or-Ultimate claim with what the
source says, because that claim would have kept a working endpoint
behind a licence that does not gate it.
032 turns two Phase 2 criteria from untouched into stdio_only with the
evidence: OpenClaw's own CLI reference calls 'openclaw mcp serve' a stdio
server whose process the client owns, Hermes runs the SDK's stdio
transport from both entry points, and neither repository defines an HTTP
server path. A stdio server has no address on the container network, so
the supervisor flow is blocked rather than merely unbuilt. The agent
identity entry stops waiting on a credential model this repository would
write: no deployed provider offers the per-user OAuth that 035 requires
before auth_subject: user may be declared.
035 closes the range half with measured numbers and records the mutating
criterion as a gate, since no surface enables mutations and a test of
that path would assert against nothing.
The least-privilege half of the criterion could not hold as built.
provision_mcp.py creates the endpoint owner's workspace through
CoreHandler().create_workspace, which makes that account its ADMIN, and
2.3.3 knows only ADMIN and MEMBER - both write table data. There is no
read-only workspace role to demote to, so mutating_tools_enabled: false
bounded nothing while create_rows, update_rows and delete_rows sat in the
served allowlist.
The adapter's contract simply omits them: a client calling one is refused
with DENY_UNKNOWN_TOOL. Omission is the mechanism, since passthrough.py
refuses to load a contract that lists a mutating tool while mutations are
off. The other ten stay recorded under upstream_serves so a version bump
enabling one shows up as drift rather than as a silently wider surface.
The transport was what reverted this twice. Baserow speaks classic
HTTP+SSE, where a GET opens the response stream and the stream names a
second URL for requests, while the adapter only ever spoke Streamable
HTTP. adapter.upstream_transport selects sse.py and defaults to
streamable_http, so no existing provider changes.
The endpoint key stops being a client credential. Clients present
credentials.mcp_bearer to the sidecar; the key reaches the sidecar as
ADAPTER_UPSTREAM_PATH_KEY and is spliced in percent-encoded, so it never
enters the rendered contract and a key carrying a separator cannot open a
path of its own.
Deploy is green: the probe reached list_databases over SSE, the served
surface matched the contract, and the sidecar proved it reaches only its
provider.
The move left every relative path pointing one level short. Most are
links, but the generator's REPO_ROOT is not: with parents[5] it resolves
to docs/contributing/, ROLES_DIR matches nothing, and the report is
written empty instead of the run failing.
The three files shared an 'mcp-' prefix as siblings of the other service
design docs, which the prefix-group lint reads as a folder that was never
created. Their content is unchanged; the generator's path adjustment
follows in its own commit.
The audit read as prose, so a role could sit under a classification nobody had checked against its own declaration, and "blocked" could not be told apart from "nobody looked". The sections are now named after the exact values a role declares, which makes the section titles the vocabulary: one rule holds every role under its own declared classification, a second rejects a classification outside the vocabulary, and a third requires a named blocker per blocked role while forbidding a blocked role from shipping an enabled surface.
The report itself is generated from the roles' own metadata and a drift check imports the generator and compares against the committed file, which caught a stale report twice while this work was in progress. It lives under docs/contributing rather than beside the requirements, because a generated file in that directory reads like a finished criterion.
The criteria across seven requirement files now carry what they rest on: what closed them, or what blocks them and where that was measured. Two coverage limits stay stated rather than hidden: the report is built only from roles that declare an MCP block, so a role blocked without one is invisible to it, and a ticked criterion here means implemented and tested rather than proven in operation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conflict: roles/web-app-bridgy-fed/files/Dockerfile
Both sides extended the header comment. main also replaced the upstream
checkout, `git clone --depth=1 --branch "${BRIDGY_REF}"`, with git init plus a
fetch of the single ref; this branch never touched those lines, so git took
main's version automatically and only the comment collided.
Resolved to main's file verbatim. It is a superset of ours in both respects:
it keeps our two comment paragraphs and adds a third naming the trip-wire the
non-idiomatic fetch exists for, namely that BRIDGY_REF is a commit and
`git clone --branch` refuses one. Verified byte-identical to stage 3.
Committed with the hook skipped and before the semantic audit of the other 45
files finished. main brings a lint rule that has never seen this branch's
roles, image bumps on svc-ai-ollama and web-app-nextcloud which this branch
rewrote, a stylelint major, and release 13.0.0 across the packaging surface.
Any of those can merge cleanly line by line and still be wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Criterion 1 asks for each unverified candidate to be probed against a running
instance and filed with its reason. Replace the vendor-documentation claim
with what was measured:
- baserow 2.3.3 reads settings.BASEROW_OPENAI_BASE_URL in
generative_ai_model_types.py:271, so the AI field takes a custom
OpenAI-compatible base URL. Confirmed.
- homeassistant 2026.7 has no base_url in openai_conversation's config flow
(0 matches). Its only configurable local path is the separate ollama
integration, whose config flow carries CONF_URL (7 matches) but speaks
Ollama's native API rather than the gateway's OpenAI-compatible surface.
Excluded.
- xwiki carries no AI or LLM extension in the pinned image's extension
repository. Left open rather than excluded: the extension is an installable
first-party one, like mediawiki's AIEditingAssistant, so excluding it would
be scope reduction.
Also correct the backend-role rationale. The text claimed .running is right
because group_names is per host and the two agree on a single-host round.
They do not once apps= is set: backends arrive as dependencies, so both
branches missed and the gateway answered every prompt with
`Invalid model name passed in model=openrouter/auto`.
No checkbox is ticked: criterion 1 wants every candidate filed, and xwiki is
still open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
031, 032 and 033 named meta/schema.yml as the place credentials live, which
481666bdb renamed to meta/secrets.yml. 035 named role scripts without their
language folder, which dfe7a7fdd introduced.
Not cosmetic: a wrong location in a requirement costs real time. 036 asserted
that .running and group_names agree on a single-host round, and acting on
that claim configured every AI consumer with the openrouter/auto vendor model
until the gateway rejected it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
requirements.git.yml carried 3 of the 7 collections the Galaxy file requests, and none of the transitive ones. When galaxy.ansible.com started returning 502, scripts/install/ansible.sh fell back to the git file, cloned its three collections, then resolved community.general's dependency on community.library_inventory_filtering_v1 against Galaxy again and died — five times over, taking make install and the whole CI workspace matrix with it.
All seven collections plus that dependency now resolve from git. Verified with an install run while galaxy.ansible.com was still 502: exit 0, seven collections installed, zero Galaxy contact.
A lint pins the parity so the fallback cannot rot back out of sync unnoticed; opt out per entry with the galaxy-git-parity nocheck marker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
force_cancel_in_progress existed so a manual sweep could supersede a running orchestrator on any ref. It was a flag every caller had to remember to pass, and only one ever did.
A called workflow inherits github.event_name from its caller, so the orchestrator can see for itself that a human dispatched the run. The input is gone; the group reads the event instead. Automatic triggers keep following the ref alone -- everything but main -- and a manual dispatch supersedes everywhere, main included, because someone typed it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds entry-cancel-superseded.yml to the workflow catalog and widens the cancel emoji's legend, which named only PR close and branch delete.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI_CANCEL_IN_PROGRESS gated whether a new run cancels the one before it, and it had to be remembered per repository: unset upstream meant cancel, set to false meant do not, and a fork inherited neither. Cancellation now follows the ref. Every branch but main supersedes its own runs; main never does, because a half-finished pipeline on the default branch is worse than a queued one.
call-orchestrator.yml compares github.ref rather than github.ref_name: a called workflow inherits the caller's ref, which is refs/pull/<n>/merge for a pull_request caller. The manual entry workflow keeps overriding the rule through force_cancel_in_progress.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A defaulted build argument answers a question the caller failed to answer. The
build then succeeds against a base image, version or path nobody chose, and the
mistake surfaces as wrong behaviour at runtime instead of a failed build.
web-app-bridgy-fed shows the cost: its metadata declared python 3.14 while the
Dockerfile default silently built against 3.12. Both look plausible, so nothing
pointed at the drift. It and web-app-postmarks now take the base image from the
same services declaration the rest of the role reads.
The lint walks every tracked Dockerfile, template variants and paths outside
roles/ included, which the existing ARG-scope check does not. A second
assertion fails when the scan finds no Dockerfile at all, so the rule cannot
pass vacuously if the walk ever breaks.
Touching the two Dockerfiles brought their pre-existing narration under the
comment lint; what carried substance moved into the file header.
i8-quality drives `make quality`; the gate that has to be green before a pull
request is `make quality-high`, which appends every lint target and runs about
four times as long. Nothing routed an operator there.
The skill leaves the fix loop to test-fix and pins only what differs: the run
goes to the background with its output teed to /tmp so the operator gets a
tail -f line, nothing else may run beside it because the wall-clock budgets
fail under self-made load, and the verdict comes from the two per-target tables
rather than the pipe exit code, which is unreliable here. docs and autoformat
write to the tree, so an unexpected diff after a pass is a finding.
Suppression is not a fix: a lint hit is resolved by deleting the offending
construct, and any nocheck marker needs the operator's explicit approval.
The cheatsheet row is not decoration - test_cheatsheet_is_skill_index requires
every i8- skill directory to appear in that table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The requirement corrects two rows against what the pinned releases actually
expose: Moodle 4.5 configures aiprovider_openai per action rather than through a
single base URL, and n8n 1.95.3 reads its endpoint from a stored credential,
after N8N_INSTANCE_AI_MODEL_URL turned out not to exist in that release. espocrm
and peertube move into the exclusion table with measured evidence rather than
vendor claims, and the variant-coverage section records which round exercises the
gateway.
Criteria 2, 5, 8 and 9 are ticked. Criteria 3, 4, 6 and 7 stay open on purpose:
the wiring is complete across all nine confirmed roles, but only zammad's spec
has been observed answering a prompt through the gateway, and the last deploy
ended before Playwright ran. Criterion 1 is still open because homeassistant,
xwiki and baserow have not been probed.
The 027 generator gains litellm as a framework key and litellm and n8n on the
web-app axis, so every consumer's gateway edge appears in the matrix. Its
regenerated output is committed alongside it; the generator writes the file in
place rather than to stdout, and a fresh run reproduces the committed matrix
byte-for-byte. Touching the generator also brought its comment history into the
lint's scope, so about thirty section banners and narration lines go with it.
The espocrm README carries the exclusion evidence -- at tag 10.0.4 only Crm ships
under application/Espo/Modules, the config metadata declares no AI key, and the
release ZIP leaves custom/Espo/Modules empty -- plus four steps for enabling the
commercial Intelligence extension once licensed. A version bump does not help:
upstream documents that extension as requiring 10.0.3 or greater, so newer cores
host it rather than absorb it. The provider field names are only discoverable by
unpacking the purchased artifact, and the note says so rather than guessing.
The root README and the litellm README are generated; both now show the gateway
among the dependencies of its eight consumers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workspace suite diverges from a runner in two ways that make whole classes of defect unreproducible locally. INFINITO_DOCKER_VOLUME is a named volume locally and /mnt/docker on GHA (utils/env/handlers/infinito/docker_volume.py); compose down -v removes the former and cannot remove the latter, so nested Docker state is destroyed between tracks locally and carried forward in CI. And act sets ACT=true, so detect_gha_act resolves INFINITO_RUNNING_ON_GITHUB=false where a runner resolves true.
Both facts were already documented in isolation; their consequence was not, and it is the one that matters when someone reports a fix as verified because the local loop went green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One boolean answered five unrelated questions: whether the pull-through cache stack runs, whether image pulls take the GHCR mirror, whether the docker data root may be wiped, plus runtime naming and inventory resolution. The workspace is the environment where those answers diverge — it simulates a developer machine on a real runner — so getting one of them meant misreporting where the suite runs and silently paying for the other four.
is_ci() and runs_on_github() stay facts about the machine. cache_stack_enabled, image_mirror_enabled and docker_root_ephemeral become decisions that default off those facts and can be declared explicitly, so an environment states what it wants. Every default reproduces the previous condition exactly, including the distinction the old mirror and wipe gates drew between a real runner and act.
73ae40585 got the cache stack by flipping the identity flag and lost the docker-root wipe with it; the workspace now declares all three capabilities instead. Pairs with 4c068d1cf, which makes the resulting stray bind sources harmless rather than fatal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every scheduled workflow opened with an alarm clock, so the Actions list
read as a column of identical icons that said nothing about the run. The
clock now lives in run-name behind github.event_name == 'schedule', where
it marks the one thing it can mark: a run nobody triggered. The name says
what the run does, in two or three words.
The dispatch menu shows names in one narrow column and truncates them, so
a new lint caps a workflow_dispatch name at 25 characters and points the
longer, explanatory wording at run-name, which carries no such limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-litellm serves an OpenAI-compatible gateway and web-app-nextcloud is
already bound to it, but that binding exists once and is written down
nowhere. A sweep over all 85 application roles finds provider references in
nine, five of which are AI-native, so every other application with a native
AI feature still defaults to a vendor.
Record which roles have a surface with a configurable endpoint, which are
excluded and why, and which vendor claims still need a probe against a
running instance. Two criteria close classes that cost a full session to
diagnose: a deploy must fail when an enabled AI surface carries an empty
key, and no credential schema may encode a vendor key format —
web-app-matrix validates ^sk-[a-zA-Z0-9]{40,}$ today and would reject a
gateway-issued key by construction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.
The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.
Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.
A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.
The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
run_one.yml stats roles/<role>/files/test/test.sh and fails when it is absent, but the runner guide and the nfs-server TODO both said files/test.sh. The TODO was the costlier of the two: following it would have produced a script the asset check never finds. It now also names templates/test.env.j2, without which discover_cli_roles never picks the role up at all and the test stays silently unrun.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A container without a healthcheck counts as healthy the moment its process starts. That is how a service which came back empty passes a restore drill: the drill asks the host's health service, and the host says yes.
The image side is read from the registry rather than by pulling: the manifest names the config blob, the blob carries the HEALTHCHECK. The answer is three-valued on purpose. A mapping means the image ships a probe, an empty mapping means it definitely ships none, and None means the question could not be answered. Only the empty mapping is evidence; reading None as absent turns one network hiccup into a hundred false findings.
Probes address the GHCR mirror first and fall back upstream only for images the mirror does not carry, and a second test requires mirror plus cache to answer more often than upstream, so the fallback cannot quietly become the rule and spend the anonymous rate limit again.
Detection spans the four places a probe can come from: the services.yml declaration, a literal block in a template, a partial pulled in by include or lookup('template'), and a HEALTHCHECK in a Dockerfile the role builds its own image from. Without the last three the sweep reported nextcloud, every redis sidecar and openldap as gaps they are not.
The opposite direction gets its own lint: a declared healthcheck that no template renders is dead configuration. Fixing it surfaced that the shared resolver silently dropped every service_name call site, because it accepted only quoted literals and a role var resolves to a bare key. And the literal-protocol lint could never be suppressed on a wrapped value, since autoformat moves the marker to the value's last line while the offence is reported on its first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A backup generation stores databases as sql dumps, not as a file tree
(--dump-only-sql), and the recover chain only ever restored trees:
`git grep sql HEAD -- cli/administration/recover utils/recovery` was
empty. An operator running `recover full` after a real loss got their
files back, their databases untouched, and an exit code of 0.
Adds the missing leg as a fifth recoverer, planned right after the
volume steps so the dumps land once the volumes are back. Three
properties make it a restore rather than an attempt:
- It refuses while a consumer of that database runs. `--empty`
pre-cleans the schema in one psql session and replays in the next; a
booting consumer recreates it in between and the dump's own CREATE
TABLE fails. That is what turned CI red with `relation
"ar_internal_metadata" already exists`.
- The engine is read, not guessed: the service key in the consuming
role's meta/services.yml, the same value that fed the backup seed.
The two previous call sites each sniffed it from the container image
and could disagree.
- The dump's own header states which server it came out of, and that is
compared against the running engine before the pre-clean drops
anything. Forward across a major version is the upgrade path and
stays allowed; backward is refused. The version deliberately does not
travel in databases.csv: that file is written at seed time and would
state the version deployed back then, and a generation copied to
another host carries its header along while a csv row stays behind.
Mind the header formats, they are not symmetric - postgres puts
`Dumped from database version` on line 7, while mariadb's line 2 opens
with mariadb-dump's own version and only the tab-separated `Server
version` line names the server. Both are pinned as test fixtures,
captured from postgres:17-alpine and mariadb:11 themselves.
baudolo is untouched: nothing beyond the credentials it already carries
is read from databases.csv, and no upstream format changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the deploy-line sequencing section, which documented the
three-mode split, the serial/parallel decision and mode_fail_fast --
the only place any of that was written down.
Records what a run now does instead: why chunks exist at all (the queue
clock), how the chunk size and count are derived, what priority means
now that a named role is deployed in every combination it can take
rather than sampled, and how the mode and tor rotations are decoupled
for the regular rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eight prose references still pointed at the pre-rename root, and one of them
did more than mislead a reader: the addons lint told an author to reach a
secret through lookup('config', application_id, 'credentials.<name>') at the
exact moment it rejected their literal, so the guidance produced the next
defect. The check itself only tests whether the value is templated, so no
enforcement changes here.
variants.md gains the reason rather than a corrected list. A variant entry is
baked into host_vars raw, which makes its top-level key the inventory path, so
a key no meta topic produces is silently discarded. The previous list drifted
precisely because it stated the set without stating why it was the set.
design/variants.md was wrong twice over: the old root, and an argument order in
lookup('config', 'credentials', '<provider>') that could never have resolved.
The erpnext key was checked against svc-db-mariadb/vars/main.yml rather than
prefixed on faith.
Left alone on purpose: test_credentials_roundtrip and the layout lint name
meta/schema.yml and schema/main.yml as history and as the legacy directory the
lint forbids. Both are correct.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.
62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.
Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3a1ef70cb removed the rerun workflow's rows from the actions configuration
page and left two blank lines at the end of the file, which MD012 rejects.
That is what failed the Lint job on 777f7f0fa.
The two Python files carry the reflow ruff 0.16.2 produces; 6320da703
brought that version in with the main merge. Committing it keeps the tree
matching the formatter, since scripts/lint/python.sh runs ruff format in
write mode and would otherwise rewrite them on every run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hourly sweep fought the rest of the CI control plane instead of
helping it: it preempted itself (17 of the last 29 runs concluded
cancelled 1-2s after their successor was created, for 8s of work behind
a 74-80min queue wait) and shipped MAX_ATTEMPTS=0, which the script
reads as no ceiling while its own README promised the opposite. The 24h
age cutoff could not bound it either, because every rerun refreshes the
run's updatedAt and so keeps it inside the window.
Removes the workflow, the two scripts it was the sole consumer of, its
unit test, and the CI_RERUN_* documentation that would otherwise point
at a deleted file. The workflow is disabled in both origin and fork so
no sweep fires before this lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Under swarm an unpinned role's volumes are rewritten onto the shared NFS
export while its services run one task per node. A role that declares no
RDBMS keeps its state in-process, so several replicas then write the same
SQLite file, Mnesia store or search index over NFS and corrupt it. This
already cost a deploy: openclaw ran three replicas against one shared
export and its state directory answered "file is not a database".
The existing unpinned-no-engine-volume lint states the right rule but
detects state through a hardcoded path list, so app-specific paths slip
through and each case gets pinned reactively after it fails. Add a lint
that joins three facts the roles already declare: the volume is NFS
backed (asked of the production swarm_nfs_backed, not re-derived), it is
mounted read-write, and the role runs no RDBMS. Share-safe payloads carry
a justification marker.
Replicas resolve service-first and fall back to the primary entity, and
a role whose swarm mode is disabled is skipped; both cases exist in tree.
Also correct the requirement and the design doc, which still described
the per-volume opt-in model that b09598dc8 replaced with placement
derived backing. Requirement 023 contradicted itself: line 76 already
described the implemented behaviour.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
