The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.
All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.
type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.
The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.
The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.
For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.
An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
At -vvv Ansible prints each module argument and result, so a password in
a command line, a request body, a header or a fact lands in the deploy
log. Nothing enforced masking outside the MCP task files, and no check
looked at parents at all.
A new lint, test_credential_tasks_are_masked, finds tasks whose printed
parts (module arguments, environment, loop) reference an identifier ending
in password, secret, token, api_key or private_key, or
secrets.credentials. A task passes when it or an enclosing block sets
no_log, when every file that pulls it in does so through include_tasks
with apply.no_log or import_tasks with no_log (transitively), when the
secret only reaches a parameter the module masks itself, or when the
module masks its values. no_log on an include_tasks task itself does not
count, because it does not reach the included tasks. The rule can be
suppressed per task with `# nocheck: credential-masking`, now listed in
the suppression catalog, and the lint tests its parent and suppression
logic on synthetic cases.
It found 39 unmasked tasks in 19 roles, all real: join tokens, client
secrets, API tokens in Cloudflare headers, database passwords in
command lines, the ChatGPT bot login body and more. Each now sets
no_log from MASK_CREDENTIALS_IN_LOGS. The sys-dns-hetzner-rdns alias
hetzner_no_log, which only restated that switch, is gone.
Not covered yet: template and copy tasks whose rendered file carries a
secret print it as a diff under --diff.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role merges the gateway service and bot into the stored Agents config and
then cycles the plugin so it adopts them, but agents/reload.yml and
agents/enable.yml both run mmctl in one container (run_once, a single cid).
mattermost-team-edition has no cluster bus, so no other replica hears of it.
Job 102705652088 of run 34421944328 (swarm, three replicas) shows exactly
that in the service log:
02:43:51-52 "Plugin activated" mattermost-ai on .1, .2 and .3
02:44:45 Point the Agents bot at the platform gateway (the merge)
02:44:48 Deactivate the Agents plugin so it drops its cached config
02:44:51 "Plugin activated" mattermost-ai on .1 only
Every post the plugin handled in the test's direct-message channel then
landed on .2 or .3, which still ran the config from 02:43 ("Notification
modified by plugin." at 03:10:30 and 03:25:32 on .2, at 03:35:42, 03:50:42
and 03:52:55 on .3), and the bot never answered:
Timeout 900000ms exceeded while waiting on the predicate ... Received: 0
After the reload the role now forces a rolling update of the service, so
every replica restarts and activates the plugin against the merged config,
and wait_converged.yml blocks until the update has completed and every task
is running again, before the stored config is re-read and asserted. Compose
runs one container, which the reload already covers, so both steps are swarm
only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-litellm has two variants: 0 with both backends on, 1 with both off. The planner assigns a dependency round_index when it has that many variants and falls back to 0 otherwise (inventory/planner.py:111), so round 1 is the one round in which the gateway deliberately serves nothing - a topology that is green in its own job.
Commit e0048f157b gated 14 consumers on LITELLM_CHAT_MODEL_SERVED so a consumer switches itself off when no model is served. Exactly two of them pinned litellm on in round 1, and a variant's literal beats the base Jinja gate, so mattermost rendered LITELLM_SERVICE_ENABLED=true against an empty model_list. The Agents plugin then failed client-side with 'bifrost error: model is required' 47ms after the prompt, never reaching the gateway.
Moving both pins to variant 2 lands them where the fallback supplies svc-ai-litellm=v0 with svc-ai-ollama=v0. Measured with the planner: round 2 pulls both. Both polarities stay pinned, so the variant-coverage lint still holds. The contract assert now also refuses an empty model, which the stored-vs-intended comparison read as converged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
supported_version and minimum_version sat in 24 roles and were read by no template, no task and no lookup. Their only consumer was the audit report version column, which turned false the moment an automated image bump ran: qdrant reported a verified version nobody had verified, two releases long. Both leave MCP_KEYS, so a role that declares them again fails the schema lint instead of carrying a field with no effect.
Requirement 025 mandated minimum_version as a MUST and 035 carried it in an example; both are corrected here, otherwise the repository would contradict its own code.
The gate then surfaced five dead docs.mattermost.com links. Mattermost moved to /administration-guide/ and /deployment-guide/, and every replacement was probed for 200 rather than guessed. The MCP page is gone with no successor, so mattermost now points at mcpserver/README.md in the Agents repository, the source rather than a rendered mirror.
Both concerns share three files - mattermost's meta/mcp.yml, requirement 025, and the generated audit.md, which derives from a source of each - so splitting them would commit an artifact that does not match its sources.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OIDC admin wiring is gated on SSO alone, and SSO is 'web-app-keycloak' in group_names. Round 1 takes keycloak variant 1, which pins ldap off, so svc-db-openldap is not in the closure; the block still resolved its swarm service and the waiter spent 600 attempts on a service that does not exist (run 33698844864, Mattermost#1, 25 minutes). In compose the same block would have died on a missing container, so the defect is mode-independent.
The number it fetched is already in the registry: svc-db-openldap/tasks/04_users.yml:118 writes uidNumber from the same lookup('users'), and an earlier run shows inventory uid 1001 against ldapsearch 1001. Reading it directly removes the cross-stack exec entirely.
Keycloak's mattermost-id mapper reads the uidNumber attribute, which until now only the LDAP federation wrote, so the same round would have produced no id claim to match. ensure_realm_user.sh now mirrors the uid into that attribute whenever it is passed one, leaving callers without a uid unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four SQL bodies behind the Agents integration were Jinja templates read
through lookup('template'), which meant the raw query: parameter and a
"# nocheck: database-query-raw" on all five call sites in
tasks/utils/ai_gateway.yml.
They now use the mechanism the database_query module already offers: static
.sql files under files/sql/, reached by query_file:, with the values supplied
as named_args and %(name)s placeholders. All five suppressions are gone.
Same move as 4e9d4b53db made for odoo, for the reason that commit gives: an
interpolation straight into SQL carries an injection surface that a bound
parameter does not. Postgres dollar-quoting was doing that job here; the
module's own escaping does it now, doubling single quotes per the _ENGINES
postgres entry.
The | to_json filters did not disappear, they moved to the call site.
MATTERMOST_LITELLM_SERVICE and MATTERMOST_LITELLM_BOT are dicts, and the
module raises TypeError on those, so pre-serialising is what it asks for.
set_ai/ became update/ai/ to match the select/ and update/ verbs the role's
existing files/sql already uses.
The api key inside MATTERMOST_LITELLM_SERVICE is now a named_args value, which
the module adds to no_log_values; as part of the query string it was not.
Verified statically only: the database-query-raw lint passes, the task file
still parses to the same 11 tasks in the same order, every placeholder has a
matching named_args entry, and no reference to the old paths survives. Nothing
ran against a database, and the adversarial review of this change had not
finished when it was committed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
set-ai-agent.sql.j2 and set-ai-service.sql.j2 shared the delimiter-bounded
prefix `set-ai`, which the repository lint treats as a folder in disguise.
The same rule was already applied in 8e8dbbd08c.
Move both into templates/sql/set_ai/ and drop the prefix, updating the two
lookups in ai_gateway.yml.
The violation was introduced by 43f6ff62e, which added the second file: the
gate ran before that commit series but not between its commits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Agents 2.5.1 migrates config->'bots' into its own agents_useragents table on
startup and logs "Migrated legacy config bots to self-service agents table".
The wiring still treated the legacy location as the source of truth, which
broke in three ways.
set-ai-service.sql.j2 re-inserted the bot into config->'bots' unconditionally,
so both stores held a bot named `agent`. EnsureBots aborts on that with
`duplicate bot name: agent`, leaves its in-memory bot list empty, and
handleMessages then returns without logging: no reply, no gateway request,
no error. Only write the legacy entry while no live agent row owns the name.
set-ai-agent.sql.j2 retargets the migrated row, because once
legacy_config_bots_migrated is set, writing config->'bots' is a permanent
no-op and the role could never change its model or service again.
select/ai/service.sql.j2 asserts against agents_useragents and additionally
reports stale when a legacy entry collides with a live agent row; the old
check reported converged over a plugin whose bot list had failed to load.
ai_gateway.yml fixes two orderings: the retarget now runs before the plugin
cycle, because the plugin reads its agents into memory at startup, and the
cycle is unconditional, because a converged-looking database was preventing
the very reload that repairs a drifted plugin.
Verified live: legacy bots 0, no duplicate bot name, the bot posting real
replies, and test-litellm-agent.js passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_prefix_groups_belong_in_subfolders flagged three groups added by this change
set: a shared delimiter-bounded prefix is a folder in disguise, so the group moves
into a folder named after the prefix and the prefix leaves the file names.
sql/select/ai-{bot-user,service}.sql.j2 -> sql/select/ai/{bot-user,service}.sql.j2
files/provision_{ai,mcp}.py -> files/provision/{ai,mcp}.py
files/ai_engine_{configure,endpoint}.php -> files/ai_engine/{configure,endpoint}.php
All six are recorded as renames, so history follows them. Every reference moves
with them: four templated query paths in mattermost's ai_gateway.yml, the two
lookup('file', ...) calls in n8n, the two in wordpress, and SCRIPT_PATH plus the
module docstring in the n8n unit test. lookup('file', ...) resolves relative to
the role's files/ directory, so the new argument is provision/mcp.py rather than
a repo-relative path.
Worth recording why this survived every gate: make quality was green on each run
this session, including immediately before the commits that introduced these
files. The lint enumerates through git ls-files, and the files were untracked, so
it could not see them. Committing them is what made them visible. A green gate
taken before new files are tracked says nothing about the gate after.
Verified: repository lints go from 88 passed with 1 failure to 89 passed, the
unit suite passes at 4448, and a repo-wide grep for every old path returns
nothing.
Not verified: no deploy has run. The lookup('file', ...) arguments and the
templated SQL paths resolve at play time, so a wrong path would surface as a task
failure rather than a test failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deletions only, no line added. The rationale each of these carried is already in
the commit message of the change that introduced it, which is where it belongs.
Three of the twenty Exception blocks this change set accumulated had already been
committed when the audit ran, so they were invisible to it: an audit that
enumerates git status stops seeing a file the moment it is committed. Two further
details are worth recording because they are what let these survive.
The purge pattern (^|\{)#\s*Exception: does not match an indented comment, so a
block nested inside YAML never matched while the report claimed none remained.
Anchoring it as ^\s*(#|//|--|\{#)\s*Exception: raised the count from 12 to 17.
One of the three was written as a paragraph inside a Python docstring rather than
as a comment. Docstrings are an allowed kind, so no comment grep matches them and
the repository's own comment lint passes over them; using the allowed kind to
carry the banned label is the same rule-gaming in a syntax the checks do not
cover.
Verified the deletion removed nothing else: the diff contains zero insertions,
every touched YAML file still parses, and the unit suite and comment lint are
green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous contract could not fail. Its guard and its closing assertion both
keyed on apiURL -- the one field the merge itself writes -- so once any run had
set the URL, every later run skipped the merge on a false_condition and still
reported all assertions passed. The live row shows what that hides: an apiKey of
`sk-` followed by 48 literal `a` characters, which the gateway answers 401 to,
and defaultModel `llama3:latest` while the round's gateway serves only
`smollm2:135m`. The merge task is `skipping` in every deploy log on this host.
The read now compares the whole desired service and bot with jsonb equality and
answers `converged` or `stale`, so a drifted key or model is drift the guard can
see. jsonb equality is key-order independent, which keeps the check from
depending on how Jinja happened to serialise the dict. Both SQL templates render
from the same two role vars, so read and write cannot disagree about what
correct means. The `jsonb_typeof(...) = 'array'` guards stay: before the first
write those keys exist as JSON null, coalesce does not fire on JSON null, and
jsonb_array_elements then raises "cannot extract elements from a scalar".
A row edit cannot create the bot account, which is why the Playwright spec 404s
on user `agent`. In mattermost-plugin-agents v2.5.1 the config lives in an
in-memory atomic pointer; bots.EnsureBots() is the only path that calls
pluginAPI.Bot.Create, and it runs from OnActivate and from the config update
listener. Writing straight into agents_confighistory reaches neither. The role's
only reload lever was `notify: container recreate`, which never fired -- the
container is unchanged with RestartCount 0 while the config row was written
inside it. The plugin is now cycled explicitly through mmctl, and the enable step
moves into tasks/utils/agents/enable.yml so the MCP path and the AI path share
one definition.
The closing gate is deliberately not another read of what we wrote: it polls the
bots-to-users join until the agent account exists. That table is written by the
plugin and never by this role, so it cannot pass by echoing the deploy's own
work. The reload fires on stale config OR a missing bot, which covers the state
this stack is actually in, where the config matched but no account existed.
Verified: the stale row, the absent bot account and the always-skipping merge are
all measured on the live stack; the EnsureBots path was read from upstream v2.5.1
source rather than inferred; every changed task file parses.
Not verified: none of this has executed yet. Specifically open are whether
`MATTERMOST_LITELLM_SERVICE | to_json` renders nested Jinja inside a dict var,
whether the converged/stale SQL is valid against live Postgres, and whether a
disable/enable cycle triggers EnsureBots on this version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plugin caches mcp.enablePluginServer in process and refreshes it only
on activation or on a plugin cluster event, which Team Edition drops
silently. Toggling the plugin over mmctl reached one container, so the
other replicas kept serving 404 on the MCP route behind the round-robin
VIP. Patching the config row now notifies container recreate, which the
repository already defines for in-database state a process must reload.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reconciler addresses a client with tasks_from: utils/mcp.yml, which
only works when every role offers its MCP work at the same path. The
files were named per role, and on the two agents the work sat inline in
the entry point, where nothing but a full role run could reach it.
The filter docstring pointed at the pre-rename path and now names the
file that exists.
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.
Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.
The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.
The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.
The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The eight roles whose MCP server is built into the application now end their MCP
step with the shared contract probe. It builds URL and Authorization header from
the role's own declaration, calls initialize and tools/list, invokes one
deterministic read tool per role, and compares the served surface against the
declared allowlist. It cannot be a Playwright test: the check needs the real
provider credential and a Playwright run would capture it in its trace.
Eleven sites wrote the MCP token into the store under the administrator's
identity while their metadata declared an owner of their own. The schema lint
forbids administrator as a credential owner, so declaration and storage
disagreed, and the probe would have looked for the bearer under the declared
owner and found nothing. They now read the declared owner.
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
The merged branch renumbered the Nextcloud tasks to 01/02, leaving the
MCP tasks stranded at 09 with a gap the numbering lint rejects; rename
them to 03 and follow every reference.
The Mattermost select statements only formed a prefix group once the
incoming branch's own select landed beside them, so they move into
sql/select/, and the two user-scoped ones again into sql/select/user/.
Pin the object kind on the kata isolation inspect: this repository names
a role's overlay network after the same entity as its main container, so
an untyped inspect can resolve the network and report a missing State key
instead of the missing container. Suppress the address-lookup rule there
because both callers gate the assertion on compose, where the container
name is the address.
Give the sandbox-runtime test the data-root variable the incoming
daemon.json template reads. Both sides were green alone: their template
gained the variable, this test renders it under StrictUndefined.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mattermost keeps the plugin's MCP server switch in the database rather
than in a config file, so the role reads the plugin state, flips the
server on, and reads back the personal access token through SQL, with
reading and writing statements kept in separate directories.
Take over the token the server issued instead of generating one: the
value only exists once Mattermost has minted it, so it is persisted
through sys-token-store rather than declared as a vault credential.
Calls execute as the account the token belongs to, so the surface is
bounded by that account's permissions rather than by an administrator
role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Variant 1 (OIDC off) died on the LDAP uidNumber task: ansible-core 2.19 templates delegate_to before evaluating when, so the undefined MATTERMOST_LDAP_HOST_NODE exploded on a fully skipped path. The OIDC admin wiring now lives in 03_oidc_admin.yml behind a single OIDC-gated include (the file is never loaded when OIDC is off) with the authservice!=gitlab branch as one inner block, and the delegate reads the resolved node via default(omit) for the remaining skip path. Each condition is declared exactly once.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 60x5s override lost against mattermost's plugin-prepackaging first boot and keycloak's healthy-wait under CI load; 120x5s stays well under the resolve util's own 600x2s default and the probe still fails hard after the budget. The LDAP host-node delegate reads the resolved var directly since resolve and consumer share the same when-gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mmctl readiness/setup execs and the kcadm OIDC-config execs ran container exec against a manager-resolved container id. When Swarm scheduled mattermost (or keycloak) on a worker, that id does not exist on the manager, so the exec failed with 'No such container' and the deploy aborted (nextcloud-3 swarm: mattermost readiness wait exhausted its retries).
Adopt the established resolve_host_cid + delegate_to pattern (matomo, keycloak, opentalk): resolve the hosting node and the node-local container id, then delegate every container exec to that node. Applied to the mattermost mmctl execs, the openldap ldapsearch, and the 10 keycloak kcadm execs; the keycloak chdir is now compose-only. Node-agnostic, no manager pin, compose behaviour unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_service_core_first_task_run_once requires tasks/01_core.yml to set the
run_once flag (include_tasks: utils/once/flag.yml) as its very first task so the
guard is in place before the role body runs and the variant-aware loader cannot
recurse. Move the flag include from the end to the top in the shared-provider
roles that had it last (bigbluebutton, flowise, gitlab, matrix, mattermost,
moodle, openproject, openwebui, peertube, suitecrm, xwiki).
Also give the template-only providers svc-db-redis and svc-db-memcached the
run-once guard (main.yml includes 01_core.yml when run_once_<role> is undefined;
01_core.yml sets the flag) so they satisfy the shared-provider run_once guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Declare the 14 services that consuming roles reference but no role provided
(bigbluebutton, xwiki, openproject, gitlab, mattermost, matrix, zammad,
openwebui, flowise, peertube, moodle, suitecrm, redis, memcached) as shared
providers: set enabled/shared/bond on each entity so the service registry
registers them and the dangling references resolve.
Add test_shared_service_provider.py (lint): fail when a referenced shared
service has no provider and its name matches a role's entity/provider that does
not register it (suppressable via '# nocheck: shared-provider').
Guard the 9 web-app shared services with the run-once 01_core.yml pattern
(main.yml includes 01_core.yml only when run_once_<role> is undefined; the core
flags itself via utils/once/flag.yml) so they deploy once when pulled in by
multiple consumers. svc-db-redis/memcached are template-only (no runtime tasks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Across 12 task files (gitea x5, shopware x5, wordpress install, mattermost
setup), strip the leftover `chdir: "{{ lookup('container', application_id,
'directories.instance') }}"` arg from shell tasks that already drive the
container via `container exec` through the wrapper. The chdir points at
the compose project root, which is only useful when calling docker-compose
CLI verbs. With `container exec` the cwd is the container's own working
directory; the host-side chdir does nothing. Empty `args:` blocks left
behind by the removal are collapsed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.
Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.
Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
01_setup.yml and 02_keycloak.yml only contain shell tasks that exec
into MATTERMOST_CONTAINER / MATTERMOST_LDAP_CONTAINER /
MATTERMOST_KEYCLOAK_CONTAINER, which exist solely on STACK_HOST. On
swarm workers the chdir to the mattermost instance directory fails
during preprocessing, leaving mm_ready_wait as a module-failure dict
without .rc; the next "until: mm_ready_wait.rc == 0" then aborts with
"object of type 'dict' has no attribute 'rc'" at 0.4s, breaking the
swarm deploy for both workers.
Wrap both include_tasks in a single block gated on
"when: IS_STACK_HOST | bool" so workers skip the entire post-deploy
chain cleanly. The OIDC condition stays task-scoped on the keycloak
include and is ANDed with the inherited block gate at runtime.
The block-with-IS_STACK_HOST pattern is already used by
svc-bkp-container-2-local and web-app-pixelfed. Compose mode is
unaffected because IS_STACK_HOST evaluates true for non-swarm
deployments.
- Rename credential basic_auth_password → api_bot_password and var
ZAMMAD_ADMINISTRATOR_PASSWORD → ZAMMAD_API_BOT_PASSWORD to reflect
that the value is the role-internal bot's local password, not the
administrator's (reused for the wizard-bypass admin only because both
need the same shell-safe alphanumeric form).
- Introduce ORGANIZATION constant (defaults to SOFTWARE_NAME) and migrate
organization-string consumers (mattermost team, shopware shop title,
dashboard login/logout descriptions, simpleicons "provided by",
user-administrator description/lastname).
- Add HOST_LL_CC_TAG (BCP-47 form, e.g. en-gb) and use in auto_wizard.
- Add NETWORK_TRUSTED_PROXY_CIDRS SPOT and consume from suitecrm,
shopware and zammad reverse-proxy configs.
- Extract ZAMMAD_WEB_PROTOCOL via lookup('tls', …, 'protocols.web')
instead of inline 'https' if … else 'http'.
- Use database_query for CREATEDB grant so it works on shared+non-shared
databases; back the SQL with files/sql/grant_createdb.sql SPOT.
- Read administrator firstname/lastname from users lookup in
auto_wizard.json.j2 instead of hardcoding.
- Drop redundant elasticsearch: block from web-app-zammad services.yml
(no consumer reads it).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Replace 13 raw `container exec mysql/psql` invocations across
web-app-pixelfed, web-app-fider, and web-app-mattermost with the
canonical `database_query` module + `query_file:` + `named_args:`
pattern. The module already speaks both shared (central `svc-db-*`)
and non-shared (per-role `<entity>-database`) modes via the
`database` lookup, so the same task shape works regardless of DB
topology — the per-role hardcoded container-exec argv was the only
blocker.
- Extract every inline SQL body into per-role `files/sql/` trees.
Pixelfed splits further into `files/sql/oidc-mapping/` because the
OIDC-mapping cluster is a distinct concern from the bootstrap /
administrator queries. Each SQL file is single-purpose and named by
verb-noun (count-users, select-admin-id, upsert, etc.) to match the
precedent set by web-app-odoo.
- Fider's existing `files/{create_admin_user,configure_oidc_provider}.sql`
move to `files/sql/` and switch from psql-specific `:'var'`
variable substitution (which forced `stdin:` piping with `-v`
flags) to engine-neutral `%(name)s` placeholders the module
escapes per backend.
- Pixelfed loses `PIXELFED_MARIADB_ARGS` + 3 derived DB vars
(`*_CONTAINER`, `*_USERNAME`, `*_PASSWORD`) from `vars/main.yml` —
they were the manual argv bundle for the old container-exec
pattern and become orphan once `lookup('database', application_id)`
takes over.
- Every refactored task carries `mask_values:
MASK_CREDENTIALS_IN_LOGS` (module-level credential redaction);
tasks that previously used `no_log: MASK_CREDENTIALS_IN_LOGS`
keep both directives so the task-level masking stays alongside
the module-level one.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive
and warns on every non-flake8 rule key. Every catalog rule in
`docs/contributing/actions/testing/suppression.md` is project-specific
(kebab-case), so `# noqa:` markers fired the warning on every parse.
`# nocheck:` is the project synonym ruff ignores.
Scope:
- 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure
noqa→nocheck swap for catalog rules. Mixed flake8 markers
(`# noqa: E402, X`) preserved verbatim.
- 5 production .py + 2 production scripts: same swap.
- All catalog-rule lints' docstrings updated; redundant "or # nocheck:"
parentheticals collapsed to a single nocheck form.
- `utils/annotations/suppress.py`: docstring rewritten to spell out
the new convention. Parser still accepts both keywords as synonyms.
- `docs/contributing/actions/testing/suppression.md`: convention
rewritten to mandate `nocheck:`; missing `domain-spot` and
`literal-protocol-lookup` rows added; "enforced by" pointer added.
New guard:
- `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint
failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+`
(real ruff/flake8 shape). Allowlists the impl, its unit test, the
catalog page, the direct-yaml lint, and itself.
- `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to
reject the `noqa:` form via inlined regex (no longer routed through
`utils.annotations.suppress`).
Tangential cleanups carried in this same commit:
- `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml`
markers swapped.
- `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved
up to the import block (E402).
- `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined
before `__main__` is imported (also fixed a latent circular import).
- PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test
files swapped from `def repo_root()` to `from . import PROJECT_ROOT`.
- ~20 ruff auto-fix removals of unused `from pathlib import Path`.
Verification:
- `ruff check .` clean.
- 23 suppression-related tests pass.
- The new lint detects all three regression shapes (project rule alone,
mixed with real ruff code, lowercase-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/repository/test_domain_primary_spot.py which fails when a
file constructs a hostname via a literal `<sub>.{{ DOMAIN_PRIMARY }}` outside
the role's `meta/server.yml` (or `meta/variants.yml`) `domains.canonical` /
`domains.aliases` blocks. Every other reference MUST flow through the
project's lookup plugins so the canonical/aliases entries stay the SPOT.
Mechanics
- Regex matches `<word>.{{ DOMAIN_PRIMARY }}` patterns; wildcard fragments
(`.{{ DOMAIN_PRIMARY }}`, `*.{{ DOMAIN_PRIMARY }}`) and bare
`{{ DOMAIN_PRIMARY }}` references are intentionally NOT flagged
- meta/server.yml and meta/variants.yml: a state-machine line scanner
exempts list items and dict entries inside `canonical:` / `aliases:`
blocks (handles compact, expanded, list, and dict YAML styles)
- README.md is exempt; docs/ and files/ subtrees inside roles are skipped
- Per-line opt-out via `# noqa: domain-spot` (or `# nocheck: domain-spot`)
- File reads route through utils.cache.files.read_text and the helpers
themselves are wrapped in lru_cache so repeated path lookups during
the traversal pay the parse cost once
Repo cleanup
- 11 role meta/server.yml files now reference Keycloak, Collabora,
OnlyOffice, Nextcloud, and their own canonical hostnames through
`lookup('domain', '<role>')` (and `lookup('config', '<role>',
'server.domains.canonical.<key>')` for the dict-canonical roles
Matrix and Bluesky) instead of duplicating the SPOT
- `roles/web-app-mattermost/tasks/main.yml` keeps a single
`# noqa: domain-spot` on the legacy domain cleanup task; the
hardcoded historical hostname must persist there to remove stale
on-disk nginx configs from the rename
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
