CLI tests are discovered by group membership alone, so a test that builds
its own fixtures from the controller's checkout ran on every host carrying
its role. The meshed swarm arm puts svc-net-wireguard on every node, and run
38864400393 failed Pretix#2, Magento and Matrix#2 with
"cd: /home/runner/work/infinito.nexus/infinito.nexus: No such file or
directory" on each node.
A role now declares cli.controller_only in meta/tests.yml, read through the
role_tests lookup like cli.timeout and cli.container_service_key, and its
test runs only on a host reached over a local connection. Every other role
runs exactly as before. Checked against a local-connection host and a
docker-connection host: svc-net-wireguard ran on the first and was skipped
on the second, and svc-net-tor ran on both.
MCP Adapter 0.7.0 for WordPress refuses a request that follows
initialize without the MCP-Protocol-Version header: 'Invalid Request:
MCP-Protocol-Version header is required for a 2025-06-18 session'. The
contract probe never sent it, so the CLI test of web-app-wordpress
failed at tools/list.
The probe now keeps the version the server answers initialize with and
sends it on every later request of the streamable HTTP transport, which
the specification asks of a client since 2025-06-18.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Of the 76 open code scanning alerts, 39 point at code that can change.
svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
Percent-encoding keeps a separator inside its segment, but "." and ".."
need none, so an upstream that resolves dot segments could answer from a
path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
a long run of digits from every start position.
Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
uses select_autoescape now, as six test modules and
utils/templating/jinja.py already did. It escapes templates named
*.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
_SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
lookup test imports its module once.
Closes#807
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ansible warns that the name is already in use and carries on. Carrying on is the problem: inside the loop the original value is gone, so every lookup, template and nested include reading that name sees the loop item instead, and the task still reports success. The damage lands in a log nobody greps.
The lint collects every key declared in group_vars and in each role's vars or defaults, then reports any loop_var matching one. A denylist of known-bad names would have found application_id and missed domain, which is the same class in four more places.
All seven existing sites are marked, each with the reason read at the source rather than inferred. The three application_id loops pass the role id as the item, and the include_vars that follows loads that role's own application_id with the same value, so both meanings coincide by construction. The four domain loops hand the vhost to sys-stk-front-proxy, whose tasks read front_proxy_domain from exactly that name, so the shadowing is the interface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An app whose inventory group spans several swarm nodes ran its CLI test once per
node, in parallel. All of them authenticate as the one MCP service account the
role owns, and the adapter keeps every session of a user in a single user-meta
value which it reads, edits and writes back whole, without a lock and without a
key per session. Two simultaneous initialize calls overwrite each other and the
loser's follow-up is answered "Session not found: Invalid or expired session".
Run 34699757369, job 103572236972: three CLI tests finished 6ms apart, the node
journal shows two 202 responses ahead of a 200 inside the same second, and
exactly one 404 among seven 200s across the three replicas. The adapter's
DEFAULT_INACTIVITY_TIMEOUT is DAY_IN_SECONDS, so expiry cannot explain a session
lost seconds after it was made. Compose has one node and never shows this.
throttle: 1 removes the concurrency and keeps every execution. run_once would
have removed executions instead: this task runs the role's own test.sh, not only
the MCP contract, so per-node coverage is worth keeping.
The cost is bounded. In that job one app of ten spanned more than one node, and
its test took 9.29s; the two that run for six minutes each occupy a single host
and pay nothing. Compose pays nothing at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A client reaches the MCP endpoint by the provider's service name on the
container network, so the hop is plain HTTP. WordPress hands out an application
password only over SSL and, on an onion deployment, home_url() is
http://<onion>, so the gate stays shut: core never even attempts to validate the
credential and answers rest_not_logged_in. The client learns nothing about why.
Measured on a live deployment, same credential, same endpoint, the only
difference being the declaration of the scheme: 401 without it, 200 with it.
Deliberately wrong passwords produced the same rest_not_logged_in as correct
ones, which is what "never validated" looks like from outside.
The accommodation belongs on the provider's side, as 5e810c4a5a already settled
for Moodle: a probe that spoofs Host and X-Forwarded-Proto asserts the endpoint
works when spoofed and says nothing about how a real client reaches it. That is
exactly what happened here - the contract test sent the header and passed while
hermes and openclaw, which send only their Authorization, were refused in run
34699757369 on both compose and swarm.
WORDPRESS_INTERNAL_HOST is rendered from the same mcp.endpoint.service_key the
clients build their URL from, so both sides name one thing once. The filter
opens the gate for a request whose HTTP_HOST matches it, splits on ':' so a port
cannot defeat the match, and changes nothing when the variable is unset. Its
existing branches are untouched.
contract.py loses the X-Forwarded-Proto it used to send. The test now takes the
route a client takes, and can no longer be green while every client fails.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Over streamable_http the contract probe judged initialize, tools/list and the
read call by HTTP status alone. A server that answers 200 with a JSON-RPC
`error` object therefore passed initialize and the read call, and failed
tools/list only as "server serves []", which names the wrong cause. The SSE
path checked initialize and the read call, but not tools/list. mcp-adapter,
which fronts WordPress, maps invalid params to 200, and Moodle's
webservice_mcp answers a failing tool call with 200 and code -32603, so a read
that returned nothing could turn the check green.
rpc_error() extracts the error object of a JSON-RPC body, refused() is built
on it, and reject_error() fails initialize, tools/list and the read call on
both transports with the server's own message. A 200 whose body is not JSON
still carries no error object, so a provider that has not started keeps the
caller retrying as before.
The streamable_http initialize, and the one in the negative probes, sent no
clientInfo, which the MCP SDK requires. Home Assistant (mcp 1.26.0, stateless)
answers such an initialize with 200 and -32602 and serves the rest anyway, so
only the old status check let it pass. All three initialize calls now share one
INITIALIZE with clientInfo, as the SSE path already sent. Against a Home
Assistant-shaped endpoint on mcp 1.26.0 the SDK logs "InitializeRequest.params.
clientInfo Field required" for the old probe, and the fixed one prints OK.
Against HEAD, an error-on-200 at initialize and at the read call printed OK;
now all three steps print REJECTED naming the error. The MCP adapter in front
of sixteen of the nineteen streamable_http providers answers 200 only with a
result (svc-ai-mcp-adapter/files/python/server.py), so none of them changes
outcome.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hermes_hermes.1 ran on a worker while run_one.yml ran its script on the loop
host. resolve_container_id.sh finds nothing node-locally on the manager, falls
through to the cluster-wide branch and returns the task's ContainerID, which
only the owning node's daemon knows, so `container exec` answered
`No such container: 65801ca259a1` twenty times over 116 seconds. Run
33785198875, job 100756596935.
Resolve the owning node and run there. meta/tests.yml gains
container_service_key for the two roles that exec into their own container; a
role without the key keeps the loop host, and role_tests returns the default
for a missing path, so absence is a polarity rather than an error.
The reset at run_one.yml:27-30 is unconditional and precedes every
delegate_to. Without it the first loop iteration dies with
'CLI_HOST_NODE' is undefined, because _cli_apps starts with a role that has no
key; the same task closes the fact leak that include_tasks leaves open between
iterations. The resolve include is gated on an explicit length check rather
than truthiness, or a keyless role would hang in resolve_host_cid's converge
wait, and it caps that wait at 30 x 2s instead of inheriting 600 x 2s: the
deploy is long finished by the time the CLI stage runs, so failing fast beats
waiting twenty minutes.
Staging, render and run all delegate. Delegating only the run would turn
`No such container` into `No such file or directory`, since the assets would
stay on the loop host. Both templates now carry CLI_LOCAL_CID: with the exec
on the owning node the id is node-local, so container_address's deferred
`$( ... )` is not needed, and it never survived `set -a && . test.env` anyway,
which evaluates it once at source time.
The stage directory is removed on the delegate too. tasks/main.yml clears it
without delegation, i.e. only on the loop host, and the task's own name calls
its contents decodable DB secrets.
Measured locally rather than argued: self-delegation leaves file, copy with
content, copy with src and shell byte-identical, including mode; the loop
reset gives keyless roles an empty container and keeps CLI_HOST_NODE on the
loop host across five iterations; and container_service_key resolves to
hermes, openclaw and empty strings through the same walk cli.timeout already
uses in production.
Not fixed, and named so it is not forgotten: web-app-joomla, web-app-yourls
and web-app-penpot exec from an IS_STACK_HOST gate without resolve_host_cid,
so a task of theirs landing on a worker hits the same error from the same
source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cosmos generator linked a consumer to its provider only through an
'<role>' in group_names reference or an explicit provides: name. A hard
dependency has neither -- pinning the provider on unconditionally is
exactly what removes the Jinja reference the generator was reading -- so
web-svc-asset's bond to web-svc-file vanished from the diagram the moment
it became binding. Match a service key against the provider's own primary
key from the registry, in both directions.
Services the role turns off drew a solid 1:1 edge, claiming a mandatory
relationship for something that carries nothing at deploy time. They now
render as a red 0..0 edge; edge merging gained a real precedence so a
provider consumed through two keys stays live when only one is off.
Named colour, not hex: a '#' opens an entity code in the mermaid lexer and
turns the whole diagram into a parse error on GitHub.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A person credited with a link in one place and a bare name in another
reads as two different people and drops the reader's path to them.
sys-svc-engine showed it in two adjacent lines: the same name once bare,
once linked.
The new credits-attribution lint requires a name linked in any Credits
section to carry that link everywhere, and rejects one name pointing at
two URLs. It found 13 offenders, including a second contributor whose
attribution had drifted in test-e2e-cli.
The generator produced the defect on every run: its context carried
application_author but never application_author_url, so the template
always took the unlinked branch. Existing READMEs kept their correct
Credits because the generator only fills missing sections, which is why
239 roles were right and 12 wrong.
The name to URL map is derived rather than declared: galaxy_info.company
names the project owner, not the role author (web-app-mattermost is
authored by Alejandro Roman Ibanez but lists Kevin Veen-Birkenbach as
company), so only the Credits lines themselves carry the mapping.
Generator and lint share that derivation and cannot drift apart, and a
new contributor needs one linked credit rather than a code change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test.sh spawns its SOCKS forwarders with the bare python3 from PATH -
the baked /opt/venvs/infinito venv on swarm nodes - which neither
sys-pip-install (targets ansible_python_interpreter) nor the system pip
reaches. python3 -m pip hits exactly the interpreter test.sh sees;
--break-system-packages keeps the compose CI system python accepting it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roles declared under meta/main.yml dependencies now appear on both the
Dependencies and Dependents side, labelled with the glossary gear
symbol and connected by a fixed 1:1 edge; the legend explains it. All
role READMEs regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Edges now carry their relation: solid 1:1 for fixed consumption, dashed
0..1 for conditional (group-membership gated) consumption. Below every
diagram the generator renders a reading key for the edge kinds and the
node markers, with the glyphs drawn from the shared symbol glossary.
All role READMEs regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).
The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.
test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.
Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.
Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
svc-runner's CLI test runs a full nested reinstall deploy and needs
75+ minutes, so the uniform 3600s timeout killed it deterministically
in both modes. A new role_tests lookup reads roles/<id>/meta/tests.yml
(cli.timeout, scaled through the timeout lookup) with the existing
default as fallback, keeping the short hang-leash for every other CLI
test. svc-runner gets 4h and pins the nested deploy to variant 0; the
second matrix round only redeployed cdn+dashboard and added no
assertion value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
27 jobs of run 29147179243 burned their full 360-min budget. Three
systemic layers, all addressed here:
- The CLI e2e task ran the staged test.sh unbounded, and test.sh blocks
in 'systemctl start' on a oneshot backup unit with no JobTimeout; on a
wedged dockerd or a stalled NFS mount both freeze silently for hours.
The task now runs the script under 'timeout TEST_E2E_CLI_TIMEOUT'
(60 min via the timeout lookup, so TIMEOUT_FACTOR scales it) and the
unit starts are bounded by BKP_TEST_HEALTH_TIMEOUT.
- The converge probe re-ran the full per-service diagnostics (service ps,
service logs, inspect, journalctl) on EVERY retry, so 600 nominal
2-second attempts cost 25-33s each and silently outlived the step
timeout. The block is gone (the workflow-level container.py rescue
snapshot already collects it recursively on failure) and every docker
call is timeout-bounded. Same removal in the swarm collect script,
which keeps only the NFS and render evidence container.py does not see.
- Playwright-collect, rescue diagnostics and Cleanup had no
timeout-minutes, and teardown.sh calls docker against D-state nodes,
so a failed job hung there for hours until the job cap killed it
without artifacts. Every step is bounded now, every teardown call is
timeout-wrapped, and the deploy steps (swarm 270, compose 300) leave
room for diagnostics and cleanup below the 360-min cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-role existence asserts and the test script path were controller paths evaluated on the remote swarm node (docker connection, node checkout under /opt/infinito-nexus), so every svc-runner swarm job failed on a stat that could never succeed. Assert both required assets in one localhost-delegated loop, stage roles/<app>/files/test onto the node, run the staged script, and delegate the optional defaults/vars probes to the controller (they silently skipped before). svc-runner's test reaches its sources via RUNNER_SRC_DIR (compose: INFINITO_SRC_DIR dev-container mount; swarm: node checkout). The node checkout path itself moves into a single source of truth: INFINITO_NODE_SRC_DIR in default.env, consumed by the bootstrap/copy scripts, the swarm-playwright target, and the swarm inventory extras.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test-e2e-cli renders test.env at 0600 in a 0700 stage dir and always removes the stage tree (it holds base64 DB credentials). test-e2e-playwright + the e2e scripts resolve the node SOCKS from the single spot: playwright via lookup('tor_socks', 'host.docker.internal'), onion.sh takes it as a required arg, rerun-spec.sh composes it from INFINITO_TOR_SOCKS_PORT — no 9050 literal or backup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Byproducts of `make autoformat`: add YAML document-start markers to
svc-runner/test-e2e-cli meta files and rewrap the jellyfin galaxy_info
description. No semantic change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
