The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Home Assistant writes the owner password into the auth store on its volume
during onboarding and never reads the environment again. When the deploy
rotates credentials the container gets the new value and the store keeps the
old one, so the login that every later step needs is refused.
The provisioning script returned that refusal as an ordinary step, and the
/auth/token call that followed failed on a missing result key - a message about
the transport for a problem in the credentials. It now exits on any login flow
that does not reach create_entry, naming the account.
That turns a five-minute wait into an immediate branch: the retry loop stopped
only on success, so a password that will never be accepted was tried thirty
times. The loop now also breaks on the refusal, and only that one outcome is
allowed past failed_when - a crash or a traceback still fails the task, which
matters because the step after this reads a token out of stdout.
hass --script auth change_password reaches the store through the filesystem, so
it needs no working login. The hub holds the store in memory, hence the restart
before the same mint runs again.
e3caa074b2 fixed the account one layer in: the MCP service account, through
admin_change_password, which needs an admin token. This is the login that
issues that token, so without it that fix cannot be reached at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Home Assistant pinned the MCP service account to its read-only group when it created the account and never again, so an account promoted to administrator by hand kept every mutating tool that scope denies while the deploy stayed green, because nothing looked at the existing case. The group is now asserted on every run and a refusal aborts instead of passing.
Nextcloud minted an app password under a fixed managed name and never revoked the previous one, so every run that found the stored password rejected added another valid token while the old ones kept full account access, accumulating across deploys. The mint is now preceded by a listing and the revocation of every token carrying that name. Both new tasks mask their output, because the listing names the account and its token ids even though the JSON strips the secrets.
Prometheus ran with no query timeout, so its two-minute default applied: a query the adapter abandoned after fifteen seconds kept the server busy for the rest of it. The flag is now derived from the adapter's own limit and doubled, which gives the server room past the adapter's deadline while keeping the two from drifting apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every role-local MCP spec gates itself on the service flag and skips when the surface is off, so nothing ever checked that switching it off removes it. A surface still serving after the flag went false was indistinguishable from one that was never deployed: both read as a green skip. The gating helper had no inverse at all, so there was no way to assert what a disabled service must not do.
A shared assertion now probes the endpoint with both a GET and a POST while the flag is false and fails on any answer that shows something listening. It counts 401 and 403 as listening, which is the point: a refused credential still means the surface is there. The check is transport-agnostic because the roles differ in their enabled-state probes, over SSE, Streamable HTTP or plain JSON-RPC, while an endpoint that is gone answers none of them. Each role passes its endpoint URL as a thunk, since some resolve it only once a test is running.
The coverage lint requires the registration, and derives the role set from meta/mcp.yml rather than from a filename convention, which is how the tenth role surfaced: its spec is not named like the other nine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role-files-layout lint wants scripts under files/ sorted into
per-language folders, which dfe7a7fdd did for 24 of them. Playwright suites
are exempt: the runner copies the tree as it stands, discovery works by path
and the specs import their helpers relatively, so sorting by extension would
break both.
83 roles already carry the marker; hermes, homeassistant, litellm and
openclaw were missing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.
Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.
The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.
Merge work rather than authored here; not deployed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merged branch moved every role-local script out of files/ into a folder
named for its language (files/python/, files/php/, files/shell/), but the
moves lost their staging and never landed. HEAD still carried the old paths
while the worktree had the new ones.
Move the 24 remaining scripts and update the six places that resolve them:
the mcp-adapter instance/isolation/probe tasks, the litellm key task, the
adapter README and the endpoint-path lint. Every changed line in those
files is a path.
Also drop roles/sys-svc-compose/files/swarm/stack_ready.sh, a stale
duplicate of the copy already tracked under files/shell/swarm/.
The added comments are the branch's own `nocheck: mirrored-unit-test`
directives, which state why a script whose measurement only exists inside
its container cannot have a mirrored unit test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four roles created their MCP service account and never touched its password
again, while the same play authenticates as that account moments later. A
deployment whose declared password moved on therefore failed at the login,
not at the point the drift was introduced.
gitea now runs 'admin user change-password' before the token dance. The
--must-change-password=false flag is mandatory rather than cosmetic: at the
pinned v1.27.1 the flag defaults to true, so omitting it locks the account out
of the very API call that follows.
nextcloud resets through 'occ user:resetpassword --password-from-env' before
minting the app token that verifies the password.
snipe-it compared nothing at all - its docstring promised to pin the password
while the assignment sat inside the create branch. It now rehashes only when
Hash::check says the stored hash no longer matches, so the changed flag stays
honest.
home assistant swallowed a failed credential creation for accounts that already
existed and then logged in with them. It now calls admin_change_password and
falls back to creation only when the reply is credentials_not_found.
jellyfin and zammad already converged; wordpress, gitlab and moodle never
authenticate with this password and are left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both probes are driven with retries and an until condition that reads the
marker in stderr, so writing REJECTED ends the loop on the first attempt no
matter how many retries remain. Both wrote it for every failure, which turned
a provider that had not finished starting into a final judgement.
The same defect cost a full run elsewhere in this family: a 503 from a
starting service ended its task at attempts 1 of 30 and failed the deploy
after an hour.
Transient statuses and connection errors now report PENDING so the caller
keeps waiting, while 4xx and contract violations stay REJECTED. A stream that
carries no endpoint event is a contract violation, not a slow start, and is
kept terminal deliberately. Both probes also catch OSError now, which used to
surface as a traceback with no marker at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The assertions were embedded in each role's playwright.spec.js. Extracting them
follows how the other specs are laid out and completes the set: all eight MCP
server roles now register a guest spec, where only five did before.
Both assertions matter. A non-2xx alone would pass for an endpoint that answers
401 and still names its tools in the body, so the inventory is checked too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality rejected the new reclaim script for missing the executable
bit, and the S310 suppression in the Home Assistant token probe for
carrying no reason. A security suppression must state why it is safe, so
the loopback justification goes back on the urlopen call; ruff dropped
the second marker on Request(), where S310 never fires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role body runs per host, so the mint ran on every node in the play.
mint_long_lived_token deletes the existing token under the same client
name before creating a new one, so each node invalidated its
predecessor while the store persisted the first one. Clients then got a
401 on every call. A deploy-time probe now asks the hub whether the
stored token still authenticates, re-mints only when it does not, and
refuses the deploy when even the fresh token is rejected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guest spec requested /mcp_server/sse, a path Home Assistant does not
route. The hub answers 401 on any unknown path, so the assertion passed
without ever reaching the MCP surface. Upstream serves it at /api/mcp.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MCP provisioning was idempotent against our own token store only. When a reinstall leaves the hub volume in place but resets the secrets store, the two disagree: the store is empty while Home Assistant still knows the client_name. It then refuses the second long-lived token with {"code": "unknown_error"}, which says nothing about the cause, and the task burned all 30 retries on it.
The mint step now lists the refresh tokens, deletes every entry carrying our client_name, and only then creates the new one. A token whose store entry is gone cannot be used by anything anyway; it just holds the name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role declared a server-direction MCP service but never provisioned one, and the declared endpoint pointed at /mcp_server/sse, the legacy path. Home Assistant serves MCP under /api/mcp, and its mcp_server integration has no YAML schema, so the config entry only exists through the config-entries flow API.
The provisioning script drives that path over the container loopback: onboard the administrator when the hub is fresh (falling back to the login flow when it is not), mint a long-lived token over the websocket API, then create the mcp_server config entry with it. The token is persisted through sys-token-store, which is why meta/users.yml registers the key; a second role registering a token for the same user only works since the per-key user merge.
The whole path is skipped when a token already exists, so a redeploy costs one lookup. Handlers flush before it runs because the hub container has to exist before anything talks to its API.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first MCP server role: where the agent roles consume tools, this one
offers them. Its services.mcp block declares an internal streamable-HTTP
endpoint gated on an agent being deployed, so the surface only exists
when something can use it, and it stays read-only because a privileged
auth subject must not carry mutating tools.
Home Assistant rejects any request carrying X-Forwarded-For from an
undeclared proxy, answering 400 for every call behind the platform's
reverse proxy. The role therefore renders a configuration.yaml that
trusts the stack subnet, keeping default_config so mounting the file does
not drop the integrations Home Assistant generates on first start.
As a shared_net provider its compose file references its own network as
external, which nothing creates in compose mode, so the role pre-creates
it the way prometheus, minio and seaweedfs already do. The vhost declares
status_codes [200, 302, 405] because the health probe sends HEAD and the
hub answers 405.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
