TestDotenvSmoke ran `python -m cli.meta.env` with cwd=PROJECT_ROOT and no
--output, so every make test rewrote the real .env with defaults: as root
inside the container, with a fresh INFINITO_CACHE_PACKAGE_ADMIN_PASSWORD
and possibly another INFINITO_DOMAIN.
The test now writes to --output <tempdir>/.env and asserts that the
repository .env keeps its inode and mtime. A marker line in a worktree .env
vanished on every run of the old test and survives the new one; without
--output the new assertion fails with "The smoke run replaced the
repository .env".
Fixes#1188
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
manage.py create-profile and update-profile parse their options with
argparse, and `-p <value>` reads a value that starts with `-` as the next
option. A generated password with a leading dash therefore fails both
commands with exit 2 ("argument -p/--password: expected one argument")
before they reach the backend, and the bootstrap task retries until it
gives up. On main this broke the guide run of Penpot#2 in run
39668792538: the Docker events of the rescue snapshot show eight
create-profile execs ending with exit 2 while the backend was healthy
and its prepl received no call.
Without -p, manage.py reads the password with getpass, which falls back
to stdin when the exec has no TTY. Both tasks now feed the password to
`container exec -i` on stdin. This also keeps the password out of the
exec command line, which Docker writes into its event stream and
therefore into the rescue diagnostics artifacts.
The password-quote lint masked only block `stdin: |` / `stdin: >`
values and flagged the inline `stdin: "{{ PENPOT_ADMIN_PASSWORD }}"` as
an unquoted password in a shell command. stdin never reaches a shell, so
the lint now masks every `stdin:` form; an unquoted password in `cmd`
is still reported.
Fixes#1139
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scripts/meta/resolve/diff/affected_roles.sh emits __ALL__ as soon as
one changed path outside roles/<role>/ is not a .md or .rst file. A
role fix that also adds its unit test under tests/unit/python/roles/
therefore deployed the whole matrix: PR #982 changed
roles/web-app-zammad/ plus
tests/unit/python/roles/web-app-zammad/files/test_search_allocation_precheck.py,
and its "Detect affected roles" job logged "Resolver output: __ALL__".
Paths under tests/ now join the .md/.rst skip-list. No role deploy
reads the top-level tests/ tree, and those suites run in their own
jobs whatever the whitelist says. A diff of only skipped paths keeps
falling back to __ALL__, as before. The same resolver feeds
effective_whitelist.sh, so the narrowing applies to every diff-driven
run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ansible defines it on every play, like playbook_dir already on the list. The
wireguard CLI test reads it to run the mesh CLI with the controller's
interpreter.
Of the 76 open code scanning alerts, 39 point at code that can change.
svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
Percent-encoding keeps a separator inside its segment, but "." and ".."
need none, so an upstream that resolves dot segments could answer from a
path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
a long run of digits from every start position.
Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
uses select_autoescape now, as six test modules and
utils/templating/jinja.py already did. It escapes templates named
*.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
_SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
lookup test imports its module once.
Closes#807
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts, and no file touched by both sides. Incoming is a run of
Docker image version bumps plus one CI change, which moves the diff
baseline from a force-pushed mirror of main to a fetch in each consumer --
entry-push-latest.yml and the resolve scripts, none of which this branch
touches.
Gates on the merged tree: unit 6199, lint 525, integration 330, external
19, ansible-lint clean, markdownlint clean, ruff clean.
The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.
Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.
baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.
The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.
Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.
test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.
Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the generated role table of README.md. Both sides changed
the same rows: upstream swapped WordPress's Mailu dependency for Stalwart,
while this branch replaced the three svc-net-wireguard-{core,plain,
firewalled} rows with the single svc-net-wireguard it deploys.
Resolved by regenerating rather than by hand, since make readme-index owns
that table and gates it with --check. The regenerated table carries both
sides and the check reports it up to date.
README.md was the only file both sides touched, and nothing in the tree
still names the three replaced roles.
Deprecated roles are no longer tested, but Stalwart variant 1 still switched its mailu service key on so CI co-deployed Mailu for the migration e2e. The override and the static mailu key, whose only purpose was that pull, are gone, and the generated cosmos drops the mailu node.
The migration gate keeps only the check that services.stalwart.migration.import_mailu stays declared and off by default. The operator-side import (12_import_mailu.yml, migrate_from_mailu.py) still runs when web-app-mailu is in the inventory; the e2e harness under files/test/ and templates/test.env.j2 now skips itself and is left for removal.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
entry-push-latest.yml lets a push to main cancel the running pipeline only when its head commit starts with the release subject pkgmgr writes. The new integration test reads that prefix from the workflow and the tag schema from scripts/github/resolve/release/tag.sh, then requires every release tag from v13.0.0 on to sit on "Release version X.Y.Z". Six tags between v10.1.0 and v11.6.0 carry "Release X.Y.Z", so the check starts at the first tag of the uniform run.
call-test.yml now checks out with full history and tags, the pair detect-release already relies on; a depth-1 checkout carries no tags and would skip the test. Forks and tagless clones skip it on purpose.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A push to main whose head commit starts with "Release version " (the subject pkgmgr writes for every release) now cancels the main run in progress, so the release starts at once instead of queueing behind it. Every other push to main still leaves the running pipeline alone.
When the pushed main commit carries a vX.Y.Z tag, the orchestrator runs with chunk_gate false: every chunk deploys and reports, and a failed chunk still turns the run red, so release-guard and release-version stay skipped.
entry-cancel-superseded.yml mirrors the new rule behind its push guard, parenthesised so && does not bind the release clause; the mirror test composes the expected rule the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.
Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
The fixture holds one value per dispatch input, distinct from its default, so
an input the retrigger drops shows up as a missing key. Adding the input
without adding it here made that check fail on its own absence.
The mirror loop added in eef3c3be71 iterates with
`LMSTUDIO_PRELOAD_MODELS | subelements('mirrors', skip_missing=True)`. That
filter ships with ansible-core rather than with this repository's
`filter_plugins/`, and the allowlist that separates "provided by the runtime"
from "must be defined here" did not carry it, so test-integration went red on a
filter that is perfectly valid:
These filters are used in templates/YAML but have no local definition
(and are not in BUILTIN_FILTERS):
- subelements
Measured against the interpreter the suite runs on, `FilterModule().filters()`
from `ansible.plugins.filter.core` contains `subelements`, so the entry states
a fact rather than a suppression. The set has exactly one reader, the
subtraction in the same file, and the entry removes that one name from the
candidates and nothing else.
This is the same one-line extension as 9118b47654 (regex_findall) and
fa59b29b2a (urlsplit).
Residual: the check can no longer report this name, so a future local filter
called `subelements` would shadow the builtin unnoticed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
chunk_size() multiplied the runner slots a chunk may assume by the waves the
queue window affords, and nothing could say otherwise. The derivation is the
right default -- too few rows leaves runners idle, too many push a chunk's tail
past the queue cancel -- but a run that wants smaller chunks had to edit
default.env to get them.
INFINITO_CI_CHUNK_SIZE now takes precedence when it is greater than zero, and
zero keeps the derivation. The input travels the same path offset already
travels: entry-manual-steer.yml offers the field and maps empty to 0,
call-orchestrator.yml hands it to all four chunk blocks, call-test-deploy.yml
exports it as the env var. It is a string rather than a number for the reason
resume_from_chunk already documents: a workflow_dispatch number input reaches
the caller as a string, and a number-typed workflow_call input then makes
GitHub refuse the job before it exists.
Two places in call-test-deploy.yml set it, not one. The plan summary resolves
the chunk size itself, so exporting it only for the app resolution would print
a plan that disagrees with the matrix the run then deploys.
Measured against the live helper: 0 gives 80 rows over 3 chunks, 25 gives 25
over 4, and returning to 0 restores 80 and 3.
The gate found three faults in the first attempt and all three are in this
commit: the key sorted after INFINITO_CI_CONCURRENCY rather than before it, the
dispatch description ran to 81 characters against a limit of 80, and the
retrigger fixture did not carry the new input. That last one matters beyond the
test: it is the check that would otherwise let a retrigger drop the chunk size
silently. carried_inputs() itself needed nothing, since it derives from the
declared inputs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two guards over four primaries. svc-ai-litellm, web-app-bigbluebutton and web-app-peertube are the three whose round-1 include collapses to the primary alone, which is the only shape under which a deployed dependency can lose its payload; web-app-nextcloud is the ordinary control. One planner call costs 1.4s, so sweeping all 117 variant-carrying roles would cost 160s and buy nothing the three collapsing rounds do not already cover.
The first attempt asserted round_include as a subset of the payload scope. That invariant only holds without disable: a clearnet row prunes the tor provider and everything reachable only through it, so the scope legitimately shrinks below the planner's include. The assertion now runs the other way, that every deployed app with a non-empty override receives it, which holds in both network modes.
Both guards were watched failing before being kept: cutting web-app-keycloak and svc-net-tor out of variant_scope turns them red, and the suite passes with disable set as well as unset.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The systemd drop-in is written inside a block gated on TOR_DNSMASQ_OWNS_LISTENER, which turns false as soon as a foreign file sits in /etc/dnsmasq.d. The start task read that block's register unconditionally, so the expression's correctness depended on ansible defining a register for a task it skipped. Short-circuiting on the same condition means the register is never reached when the block did not run, and the question stops mattering.
The test asserted the literal declaration string, which exercises nothing. It now renders the expression read from the role with ansible's own bool filter and a changed test, in both states: owning the listener with a rewritten drop-in yields True, not owning it yields False without touching the register. The previous form raises UndefinedError under the second case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-net-tor rendered its own guard script, its own unit and its own handler, and hooked the rules into DOCKER-USER with an insert, because a rule appended to a chain docker owns lands behind the RETURN docker puts there. The script flushed its chain and refilled it rule by rule, so the boundary was gone for the duration of the refill.
A role that needs packet filter rules now writes templates/nftables.conf.j2 declaring its own table and includes svc-net-firewall, which renders it to /etc/nftables.d/<role>.conf, rejects it with nft -c, loads it, reads the table back and refuses one that carries no chain. Deployed on its own the role installs nftables and the ruleset that loads every fragment at boot. That ruleset is a distribution decision, so its path follows os_family: RedHat reads /etc/sysconfig/nftables.conf, the others /etc/nftables.conf. The unit is only enabled, never started, because on Debian its ExecStop flushes the whole ruleset, docker's tables included.
One table per role removes the shared path: a fragment replaces its own table and touches no other, and docker never touches either. nft -f on an existing table merges, so each fragment creates, deletes and redeclares its table in one transaction. The tests load every fragment twice into a throwaway network namespace and require the listing to match, because a merging fragment loads twice without error and shows up only as a duplicated rule.
The tor guard now applies in both deploy modes. It was pinned to swarm while torrc binds TransPort and DNSPort on the bridge address in compose too, so on a single-host install those ports answered every on-link neighbour. The guard drops rather than admits, so nothing short-circuits another table, and ip saddr carries an implicit ipv4 dependency, which leaves the ipv6 exposure exactly where the iptables rules left it. svc-net-tor reads its own chain back and refuses a guard that drops nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unset secret or variable resolves to the empty string, not to an error, so
an undocumented one is invisible twice: the administrator cannot know to set it,
and the run that needed it degrades quietly instead of failing. The guard reads
every ${{ secrets.X }} and ${{ vars.X }} out of .github/workflows and demands
the name in the page that owns it, secrets.md or configuration.md.
It compares the reference against the page rather than the page against itself,
so it also catches a doc naming something the workflows no longer read.
Three gaps were already there. DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
authenticate the image mirror against Docker Hub and lift the anonymous
pull-rate limit; both are declared required: false and the login step is gated
on them being non-empty, so their absence does not fail a run, it walks the
mirror into the rate limit. Neither was documented anywhere.
PLAYWRIGHT_KEEP_ALL is the sharper one: configuration.md listed
INFINITO_PLAYWRIGHT_KEEP, which is the name the workflow forwards it under, not
the name an administrator sets. Following the documentation produced a variable
nothing reads.
Commented-out lines are skipped. cron-security-scorecard.yml carries
`# repo_token: ${{ secrets.SCORECARD_TOKEN }}` as an option nobody has taken,
and demanding a doc row for it would describe an input no run asks for.
GITHUB_TOKEN is exempt in _PROVIDED_SECRETS because GitHub mints it per run; it
is not administered and cannot be set. A second case asserts the scan finds
workflow files at all, so the guard cannot pass vacuously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the
once-flag and the container network, nothing else. The gateway published
lmstudio/default anyway, so the route existed and nothing could answer it.
wp-compose-tor.log:48994 shows the route written into the config, :51770 shows
the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine
spec ran against that. LM Studio was deployed, published and never prompted.
The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the
hosting node and container id, loops the declared models through
`lms get --yes --gguf`, overlaps them with async/poll and reaps before
playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not
assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start,
and `lms get --help` documents -y for scripting and --gguf for the format.
Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM
Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry:
`alias` is what the gateway publishes and what Ollama pulls, `name` is what
`lms get` resolves. Both roles now reference that one list, and config.yaml.j2
publishes each backend under the alias, routing Ollama to ollama/<alias> and LM
Studio to openai/<name>. The lmstudio loop skips an alias Ollama already
publishes, so two backends never produce a duplicate model_name and Ollama
keeps precedence. lmstudio/default is gone; the model name is now the same
whichever backend answers it.
Ollama's preload_models changes shape from a list of strings to the {alias,
name} entries. Every consumer moves with it: the pull command, the template
loop, and both expressions in 16_ai.yml. An inventory that overrides
`applications.svc-ai-ollama.services.ollama.preload_models` with the old string
list will break at map(attribute='alias'); this is a deliberate shape change,
in the spirit of 7b15c61dfc.
The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI
lives in inventories/development/default.yml, whose own header scopes that file
to the cross-cutting environment knobs. test_variable_definitions only knew
group_vars, so it reported the constant as undefined; the shipped inventories
are a definition source too, because the deploy CLI bakes them into host_vars,
and the guard's own docstring already claimed "defined somewhere in the
repository".
A latent crash came out of this: `| list | first` raises on an empty sequence,
so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama
with only embedding models could already hit it; the existing test dodged it by
rendering only _SERVED. `| default('')` now yields the empty string that
_SERVED already reports as false.
Tests: config.yaml.j2 is rendered rather than read, over ollama-only,
lmstudio-only, both and neither, asserting the published model_name set, that
both backends name the same alias, that a shared alias is published once and
routed to Ollama, and that the render is valid YAML. test_litellm_chat_model
gains the lmstudio branch, the no-preload case and the same-model assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard recognised meta/services.yml as the only place a git URL may be
declared. 84a4b04f0 made meta/addons/<id>.yml the one place a Discourse plugin
pins itself, so the addon files now carry the clone URL literally, and
_iter_role_files picks up every .yml under the role: each declaration reported
itself as an undeclared reference.
Two halves. The declaration set is now the union over meta/services.yml and
every meta/addons/*.yml, so an addon pin legitimises that URL for the whole
role, not just for the file holding it. Path.glob on a missing meta/addons
yields nothing, so roles without addons need no special case. The self-skip
that kept services.yml out of the scan is now a membership test against exactly
those declaration paths.
Scope stays per role: declared is rebuilt inside the role loop, so a
declaration in one role legitimises nothing in another. The failure message and
the docstring name both sites, otherwise they send the next reader to the wrong
file.
This is the test-side counterpart of e4141cb826, which taught collect_entries
to see the same addon refs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A workflow the entries call declares its own concurrency group, and its jobs then leave the entry's group. The entry's cancel-in-progress no longer reaches them, so a superseded run keeps the runners its remaining jobs hold. Run 34614940508 held its runners after its successor arrived and only a manual force cancel ended it; its check runs carry no 'Canceling since a higher priority waiting request' annotation, only the forced cancel.
call-orchestrator.yml and call-test-runner-smoke.yml drop their groups. The smoke group was keyed on the caller's ref as well, so a manually dispatched smoke run cancelled the smoke job of an orchestrator run on the same ref. call-images-build-ci.yml and call-images-mirror-missing.yml keep theirs: their group deduplicates image builds across runs rather than scoping a single run, and it does not cancel.
The test asserts both halves, that the orchestrator declares no group and that every entry calling it carries group plus cancel-in-progress, and adds the chain below it. It finds the callers by their uses: line rather than by an entry-* filename, so a caller named otherwise cannot skip the requirement. It fails against HEAD, where call-test-runner-smoke.yml declares a group.
The cause is established negatively: no concurrency annotation exists on the superseded run. What would settle it is a run that supersedes another and cancels it, and that needs a push.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A sweep may deploy 210 rows (available) in chunks of 80. Three declared
blocks hold exactly that when there is no priority line, but priority
rows get a chunk of their own that is never topped up with regular rows.
Nine priority rows therefore occupied a whole block, left the regular
rows 2 x 80 = 160, and 42 rows of budget went unused.
Adding a block to the YAML alone would not have changed that:
matrix.chunks_of handed the split slots.chunk_count(), which is
min(ceil(available / chunk size), INFINITO_CI_MAX_CHUNKS) and ignores
the priority seam, so the plan still stopped at three blocks.
call-orchestrator.yml now declares test-deploy-chunk-3, gated on chunks
0 to 2 like every chunk gates on all before it, and report-main-failures
and done need it. INFINITO_CI_MAX_CHUNKS is 4, one above what the budget
fills, and slots.chunk_blocks() returns it. chunks_of hands the split
every declared block; available still caps the rows, and a chunk whose
slice comes back empty skips its deploy. chunk_count() stays the "chunks
filled" figure of the budget report.
Tests: the split spends 9 + 80 + 80 + 41 = 210 rows with a spare block;
chunks_of plans three chunks from chunk_blocks=3 while chunk_count=2
(the previous wiring yields [1, 10]); an integration guard requires the
declared chunk jobs to equal INFINITO_CI_MAX_CHUNKS and to exceed
ceil(available / chunk size) by one, and fails on the stale value 3.
A shell that inherited INFINITO_CI_MAX_CHUNKS=3 keeps failing that guard
until it is restarted; make dotenv-force rewrites .env.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Snipe-IT#1 (run 34615516009, job 103327034508) failed every login in the
async pass with an oauth2-proxy 500. The sidecar log shows 24x "error
saving redis session: WRONGPASS invalid username-password pair". The swarm
matrix rotated 54 credentials before that pass, svc-db-redis re-rendered
its requirepass and redis_redis was recreated ("about an hour ago"), while
snipe-it_sso-proxy kept running from the first pass ("3 hours ago").
oauth2-proxy reads its config, redis password included, from a bind mount
at start. Its render task notifies compose-up, which in compose is
"compose up -d --force-recreate" and so recreates the sidecar, but in swarm
is "docker stack deploy", which only restarts a service whose spec changed.
A new file behind an unchanged bind mount is no spec change, so the sidecar
never read the rotated password. pihole#1 in the same run shows the compose
side: shared redis, 47 rotated values, the config re-rendered, compose up
recreated the sidecar, and the job passed.
The sidecar now carries a container label with the sha256 of its config,
rendered from the same template sso_proxy.yml writes, with
sso_proxy_application_id supplied through template_vars. A changed config
changes the label and with it the service spec, so stack deploy restarts
only the sidecar; an unchanged config keeps the hash and restarts nothing.
Compose behaviour is unchanged.
A guard test pins the label and requires it to hash the template
sso_proxy.yml renders; it fails against the previous template. The hash is
stable for equal input and changes with a rotated secret, checked with the
template lookup on ansible-core.
Not covered: other bind-mounted configs that embed rotated secrets have the
same swarm exposure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34421944328 Nextcloud#13 lost the dashboard's Matomo tracker. The
dns53 sampler on mgr-01 shows no :53 socket from 01:39:52 to 01:48:44
while the dnsmasq PID stayed the same, six seconds after a short-lived
container's veth on docker0. In that window the manager could not
resolve its own onion vhosts, so sys-front-inj-matomo's site checks for
web-svc-logout (01:45:35) and web-app-dashboard (01:47:18) never reached
OpenResty, the injection was disabled as "API not ready", and the
Playwright check found no matomo.js. The other four swarm managers of
the run that carried the same listener kept it throughout.
Under bind-dynamic, dnsmasq 2.91 re-enumerates interfaces on every
netlink address event and then garbage-collects every listener it did
not re-find (network.c enumerate_interfaces), also when the enumeration
failed; release_listener has no other caller. 5aa7506e36 reproduced this
on the lab nodes and 11c98825aa already moved the bridge-less case to a
static bind. The bridge case now binds statically as well.
This supersedes two earlier rationales, both measured on the local stack:
- bb0e098a9f kept bind-dynamic because dnsmasq starts at boot before
docker0 exists. A unit drop-in (files/dnsmasq-restart.conf,
Restart=on-failure, RestartSec=5) now retries the start: with a missing
listen-address the start exits 2, and dnsmasq came up 4s after the
address appeared.
- 4739b2de99 called bind-dynamic load-bearing because docker0 can
disappear across a docker restart. Only a start without the address is
fatal; a bound socket survived the address being removed and re-added
and the interface being recreated with a new ifindex, answering on the
same socket inode each time.
The docker0 fact is re-gathered before torrc and the dnsmasq drop-in are
rendered. The play-start fact predates default-address-pools moving the
bridge (285ba1e76c), and a static bind on a stale address would make
dnsmasq exit. container_dns reads only the pool, so daemon.json is not
affected by the refresh.
The host resolv.conf moves from an inline copy content to
templates/resolv.conf.j2; both it and the drop-in render byte-identical
to what the local stack already runs.
A veth churn of 13.8k cycles did not reproduce the loss locally, under
either bind mode.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PASSWORD_TOKEN_RE matches the token anywhere inside a Jinja expression, so a
path or a filename carrying it demands "| quote" as loudly as a credential
does. It fired on Moodle upstream CLI script:
roles/web-app-moodle/tasks/01_manager_ops.yml:55: In shell tasks, password
expressions must include quote: {{ [MOODLE_CLI_DIR, reset_password.php] | path_join }}
The credential on the next line already carries the filter. Listmonk hit the
same shape earlier and was resolved by renaming the file, which is not open
here: the name belongs to Moodle.
The rule had no suppression at all - it never called is_suppressed_at - so a
marker in the role would have been silently ignored. Teach it the marker, then
use it.
Suppression is honoured at the finding line OR at the shell: key that opens the
block, and the second placement is the one that matters. A marker beside the
finding necessarily sits inside the scalar, and this block is folded: lines
join with spaces, so a hash would comment out the rest of the command and
swallow --username, --password and --ignore-password-policy. On the shell: key
it is a YAML comment instead. The failure message now says so, so the next
false positive explains itself rather than repeating the discovery.
Verified the command is untouched by loading the task and reading the string
Ansible would hand the shell: no hash, no marker, byte-identical.
Note the cost: the marker grants the whole block, not one expression, because
a per-expression marker cannot be placed. This block loses the guard against a
future unquoted addition.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_every_role_renders_the_shared_template demanded that a provider render
exactly the shared key set. That equality held only because no MCP provider
had extended its own test.env yet; other roles do so freely (svc-runner adds
17 lines, svc-db-postgres 8), so it was an accident of the current set rather
than a rule anyone stated.
The guarantee the docstring names, that every key of the shared template
resolves, is a superset check. Assert that instead, and name the provider in
the failure.
Unblocks the four role-local keys the Zammad AI gateway probe (b42f217efa)
needs to reach the gateway from inside the rails container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Line-length reflow in three test modules and a stray blank line in zammad variants. In router.yml the formatter also moved the nsswitch lineinfile regex from single to double quotes with the backslashes doubled to match; single-quoted YAML already passes a backslash through literally, so the string reaching lineinfile is byte-identical and the rule still changes only NOTFOUND.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The resolv.conf assertion looked its task up with a flat scan over router.yml top level. Encapsulating the two TOR_DNSMASQ_OWNS_LISTENER tasks in a block moved that task one level down, so the generator ran dry and StopIteration failed the test. A recursive walk over block/rescue/always restores the lookup while leaving what the test asserts untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hermes_hermes.1 ran on a worker while run_one.yml ran its script on the loop
host. resolve_container_id.sh finds nothing node-locally on the manager, falls
through to the cluster-wide branch and returns the task's ContainerID, which
only the owning node's daemon knows, so `container exec` answered
`No such container: 65801ca259a1` twenty times over 116 seconds. Run
33785198875, job 100756596935.
Resolve the owning node and run there. meta/tests.yml gains
container_service_key for the two roles that exec into their own container; a
role without the key keeps the loop host, and role_tests returns the default
for a missing path, so absence is a polarity rather than an error.
The reset at run_one.yml:27-30 is unconditional and precedes every
delegate_to. Without it the first loop iteration dies with
'CLI_HOST_NODE' is undefined, because _cli_apps starts with a role that has no
key; the same task closes the fact leak that include_tasks leaves open between
iterations. The resolve include is gated on an explicit length check rather
than truthiness, or a keyless role would hang in resolve_host_cid's converge
wait, and it caps that wait at 30 x 2s instead of inheriting 600 x 2s: the
deploy is long finished by the time the CLI stage runs, so failing fast beats
waiting twenty minutes.
Staging, render and run all delegate. Delegating only the run would turn
`No such container` into `No such file or directory`, since the assets would
stay on the loop host. Both templates now carry CLI_LOCAL_CID: with the exec
on the owning node the id is node-local, so container_address's deferred
`$( ... )` is not needed, and it never survived `set -a && . test.env` anyway,
which evaluates it once at source time.
The stage directory is removed on the delegate too. tasks/main.yml clears it
without delegation, i.e. only on the loop host, and the task's own name calls
its contents decodable DB secrets.
Measured locally rather than argued: self-delegation leaves file, copy with
content, copy with src and shell byte-identical, including mode; the loop
reset gives keyless roles an empty container and keeps CLI_HOST_NODE on the
loop host across five iterations; and container_service_key resolves to
hermes, openclaw and empty strings through the same walk cli.timeout already
uses in production.
Not fixed, and named so it is not forgotten: web-app-joomla, web-app-yourls
and web-app-penpot exec from an IS_STACK_HOST gate without resolve_host_cid,
so a task of theirs landing on a worker hits the same error from the same
source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite covered only the roles that serve an MCP endpoint. The roles that
consume one render the same test.env interface through the same path and were
covered by nothing. `_render` also pinned compose_mode and DEPLOYMENT_MODE to
`compose`, so the swarm branch of lookup('container_address') was never
entered anywhere under tests/ -- and that branch is the one that emits the
deferred `$( ... )` form the CLI harness then freezes at source time.
`_providers` becomes a call to `_roles_with_direction` with the same direction
set, `_clients` its counterpart, filtered to roles that also ship
templates/test.env.j2 so a declared client without a CLI test is skipped
rather than crashing setUpClass. `_render` gains a mode parameter defaulting
to compose, so every existing call is unchanged.
Client templates need the variable layers run_one.yml:44-70 loads before it
renders: web-app-hermes resolves HERMES_MCP_SERVERS from its own vars, which
resolves MCP_DISCOVERED_SERVERS from group_vars/all. Both layers are mirrored
under the explicit fixture vars, with trust_as_template applied recursively
because ansible-core does not template a loaded value otherwise.
One interpreter is enough for both modes: plugins/lookup/container_address.py
:128 reads DEPLOYMENT_MODE straight from the templar variables, so the
per-process config cache that limits the provider fixture to one polarity of
MCP_TEST_ENABLED does not reach the mode split.
The four new assertions are mode-independent on purpose. Asserting that
MCP_CLIENT_CONTAINER carries no `$(` belongs with the change that delegates
the exec to the node holding the task; it fails today and would make this
commit red.
No coverage is lost. The provider renders were compared with and without the
new layers across all 20 of them and are byte-identical, and the suite goes
from nine test methods to thirteen.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hub serves 94 tools across 23 categories, write tools included, and enabling the ExApp switches every category on. That surface was published on the public vhost and every discovered client was handed an app password for it. svc-ai-mcp-adapter now sits in front in mcp_passthrough mode: clients present credentials.mcp_bearer to contextagentmcp and reach the nine read tools of calendar, contacts and files, which files/mcp/tools.json names and the sidecar enforces per call. Those three categories are the ones whose upstream is_available is unconditional, so the served set does not vary with which optional apps a deployment installs.
Enforcing it in the sidecar rather than in each client is what makes it hold: web-app-openwebui renders no per-tool filter at all, so a client-side include list would have bounded two consumers out of three.
06c13e7f3e withdrew the declared path from the edge for exactly this reason - "that surface is what the adapter exists to narrow". Nextcloud answers the same route under a second spelling, without the /index.php front controller, which try_files rewrites back onto the route after the edge has already declined to block it. The withdrawal becomes a regex covering both, and only where stripping the front controller changes the path, so the four adapter providers without one render unchanged.
The app password is minted after the stack renders, so it reaches the sidecar through upstream.env and rebuild.yml rather than the compose environment: compose up resolves env_file while creating the container, a restart does not. The sidecar is the last service in the template because a Jinja set has no block scope and its env_file would otherwise displace the one talk, whiteboard and talk_recording each carry.
Two suites cover what nothing measured: the vhost template rendered without a container, asserting both spellings are withdrawn, and every shipped tools.json loaded through the adapter own loader, so a bad contract fails in a second instead of as a probe timing out an hour into a deploy. The guest spec probed mcp.endpoint.path, which this change makes the proxy answer 404 for its own reasons; it now probes all three public paths and requires no MCP body.
Not yet deployed: the sidecar build, the upstream.env delivery and the hub answering exactly nine tools are unverified against a running stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
assertTrue(x > 0) fails as "False is not true" and hides the value that failed; assertGreater(x, 0) reports the actual count. Same predicate, so pass and fail are unchanged on every input. CodeQL py/imprecise-assert on both generator tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
9ae0c7eff6 replaced bind-interfaces with bind-dynamic when it taught the role to bind the node address for swarm, an address that appears after dnsmasq starts. Without a bridge address there is nothing to wait for, and the rebuild bind-dynamic performs on every netlink event can then only drop the listener: on the DiD host of run 33698844864 job 100479883964 the sampler lost dnsmasq's 127.0.0.1:53 for four consecutive samples one second after a veth unregistered.
interface=lo is deliberately not added. dnsmasq unions interface= and listen-address= rather than intersecting them, measured on 2.93: naming lo additionally seizes 127.0.0.53 and ::1, and against a distro stub dnsmasq exits 2, which any_errors_fatal turns into a failed deploy.
The resolv.conf keeps both nameservers. bb0e098a9f added the second one from a measured total outage where dnsmasq held no socket at all, and that failure mode is untouched by this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".
The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.
Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.
The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.
nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RFC 6761 reserves .test for testing and .example for documentation, and the
project's own domain was doing both jobs under the documentation name. It is
what default.env hands every local deploy and what the test fixtures assert
against, so it is a testing name.
Rename it wherever it stands for the running machine: the environment
defaults, the development inventory, the workspace and trust scripts, the test
suites, and the four documents that describe real behaviour - the DNS mapping
to loopback, the browser CA trust, WSL2 resolution. Those documents would have
started lying the moment default.env changed.
The sandbox allowlist moves with them, since egress is matched per pattern and
nothing resolves the old name any more, and the sandbox document that mirrors
that list follows it.
Two places keep .example on purpose: the link-style sample in the
documentation guide, which is documentation about documentation, and every
generic fixture name elsewhere in the suite - a.example, foo.example and their
kin. Those are a larger sweep of their own and are not touched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A lookup('config', application_id, ...) is validated against the defaults of
the role the call is attributed to, and when the app argument is a variable
that role is guessed from the file's own roles/<role>/ location. The guess
holds for a role configuring itself and breaks for one rendering a template
on another role's behalf.
test-e2e-cli is the second kind. Its mcp/test.env.j2 is the single source
twenty provider roles render through lookup('template', ...), and the role
ships no vars/main.yml at all, so application_id there is always the
provider under test. The wildcard check read it as test-e2e-cli's own and
demanded services.*.name and services.*.ports.internal.* from a role that
owns no services.
Skip the fallback when the file's role declares no application_id, which is
how the sibling role-local check already guards the same guess. A literal
app argument stays authoritative either way. Folding the None case into
that condition also drops a branch the guard had made unreachable.
Measured: 114 wildcard paths still checked, exactly the two test-e2e-cli
entries dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mcp-consumer-variant guard demanded three things of a provider's MCP
round: every admitted consumer enabled, each of them pinned `services: null`
so it dragged no dependencies along, and no other service left enabled. Two
of those were the rule legislating a round's contents rather than checking
that the pairing gets proven at all.
Keep the first and drop the rest. What else the round carries is the role's
own call, so the tor/litellm exemption, the datastore exemption and the
`allowed` set they fed have no reader left and go with them.
Strip the 63 `services: null` pins the rule forced across 20 variants, all
written bare as `services:`. The topic-emptying capability behind them
stays: apply_topic still normalises None to {} for the inventory payload,
and the unit tests still cover it. Only the uses and the mandate are gone.
That leaves overridden_providers scanning a repository with nothing left to
find, so its three assertions passed over an empty dict and would have
stayed green however the function broke. Rebuild them on a synthetic role
tree, which also covers the case the repository scan never held: a variant
naming a plain flag is not an override.
Removing the pins changes what deploys. Measured across all 20 variants: 18
identical, prometheus#1 grows 23 to 31 roles, moodle#2 grows 31 to 32 and
crosses the memory budget until the following commit shrinks it back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A round deploys a provider once, so the role that pulls it in is the only place that can say how lean it has to be. A variant entry may now carry any application config topic under a service it depends on: a mapping deep-merges, so naming one key changes that key alone, and null empties the topic. Empty was the case that needed a marker of its own, because merging an empty mapping is a no-op and would leave exactly the services the round was trying to shed.
The topic vocabulary and the merge semantics live in utils.roles.applications.topics, which is also where resources.py now gets them rather than spelling the same null-or-merge out a second time. Collecting who demands what stays in the inventory tree, refuses two roles demanding different values for one provider in the same round, and skips a role's own entity entry so the existing services.<own>.users declarations are not read as overrides.
The CI set-cover learns the same fact. Its greedy cover retires a role's variant-0 row when an earlier row of another role embeds it, on the stated premise that embedding pulls the provider's variant 0. A variant that dictates that provider's config breaks the premise from the other side: whatever ran, it was not the provider as the provider declares it. Such a pair now covers nothing, so the retired row keeps its job, its artefact and its verdict.
Measured on the current tree: 20 override pairs, 41 covered rows before and after, so nothing is revoked today. The rule bites the moment an override lands in a row that covers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.
The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.
The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.
Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.
Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles enabled an MCP surface without a browser-side spec, so nothing
ever asked what the endpoint answers a caller with no credential at all.
Lift gitea's assertion into two shared registrars and register them per
role, and give each role the MCP_SERVICE_ENABLED flag they gate on.
mcpEndpointUrl now takes the base URL rather than reading APP_BASE_URL
unconditionally: jellyfin and zammad export their own base variable, and
declaring APP_BASE_URL for them would un-skip the shared guest persona,
which collapses itself on that variable being absent.
The env-gate scanner follows require("./x") into the shared harness so a
gate declared there counts as consumed, and strips block comments first
so service-gating.js's own doc examples do not credit EMAIL, SSO and OICD
to every role that imports it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
