Every role-local MCP spec gates itself on the service flag and skips when the surface is off, so nothing ever checked that switching it off removes it. A surface still serving after the flag went false was indistinguishable from one that was never deployed: both read as a green skip. The gating helper had no inverse at all, so there was no way to assert what a disabled service must not do.
A shared assertion now probes the endpoint with both a GET and a POST while the flag is false and fails on any answer that shows something listening. It counts 401 and 403 as listening, which is the point: a refused credential still means the surface is there. The check is transport-agnostic because the roles differ in their enabled-state probes, over SSE, Streamable HTTP or plain JSON-RPC, while an endpoint that is gone answers none of them. Each role passes its endpoint URL as a thunk, since some resolve it only once a test is running.
The coverage lint requires the registration, and derives the role set from meta/mcp.yml rather than from a filename convention, which is how the tenth role surfaced: its spec is not named like the other nine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role-files-layout lint wants scripts under files/ sorted into
per-language folders, which dfe7a7fdd did for 24 of them. Playwright suites
are exempt: the runner copies the tree as it stands, discovery works by path
and the specs import their helpers relatively, so sorting by extension would
break both.
83 roles already carry the marker; hermes, homeassistant, litellm and
openclaw were missing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.
Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.
The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.
Merge work rather than authored here; not deployed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The assertions were embedded in each role's playwright.spec.js. Extracting them
follows how the other specs are laid out and completes the set: all eight MCP
server roles now register a guest spec, where only five did before.
Both assertions matter. A non-2xx alone would pass for an endpoint that answers
401 and still names its tools in the body, so the inventory is checked too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guest spec requested /mcp_server/sse, a path Home Assistant does not
route. The hub answers 401 on any unknown path, so the assertion passed
without ever reaching the MCP surface. Upstream serves it at /api/mcp.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first MCP server role: where the agent roles consume tools, this one
offers them. Its services.mcp block declares an internal streamable-HTTP
endpoint gated on an agent being deployed, so the surface only exists
when something can use it, and it stays read-only because a privileged
auth subject must not carry mutating tools.
Home Assistant rejects any request carrying X-Forwarded-For from an
undeclared proxy, answering 400 for every call behind the platform's
reverse proxy. The role therefore renders a configuration.yaml that
trusts the stack subnet, keeping default_config so mounting the file does
not drop the integrations Home Assistant generates on first start.
As a shared_net provider its compose file references its own network as
external, which nothing creates in compose mode, so the role pre-creates
it the way prometheus, minio and seaweedfs already do. The vhost declares
status_codes [200, 302, 405] because the health probe sends HEAD and the
hub answers 405.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5