Each of these specs declared the service it exercises and then, as its first
action, signed in through the dashboard's OIDC button. In a round that ships
no identity provider that button does not exist, so the spec spent its whole
timeout waiting for it and reported the missing login rather than anything
about MCP. The junit already said as much, carrying the SSO_SERVICE_ENABLED
flag as the skip reason on the neighbouring tests.
Nine specs in the directory sign in that way; two already declared sso. The
other seven now do too, through the same gate they already imported.
Signing in as the deployed identity instead was not open to us: that helper
authenticates as the administrator whatever username it is handed, which
contradicts the non-member spec asserting the very endpoint the administrator
is asserted to reach. Nor can a native member stand in, because the account is
created by the OIDC login itself, so a round without one has no members to
grant anything to.
This buys the honest label at a price worth naming: a round that ships
seaweedfs without sso now reports green with no seaweedfs coverage, where it
used to report red for the wrong reason. Covering that needs a sign-in the
suite does not have yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GET /api/v1/tools/ builds its answer for an MCP server out of stored config alone - routers/tools.py:125-126 reads info off the connection row and never touches the network. So the three specs that read it pass unchanged against a provider that is down, refuses the deployed bearer, or serves nothing. What they establish is registration and the OIDC group to access-grant chain, not that the bridge carries.
POST /api/v1/configs/tool_servers/verify is the one route in 0.11.0 that does a real MCP initialize plus tools/list without an LLM, and it authenticates with the key in the body it is handed. The spec reads the stored connection and echoes it back, so the handshake runs over the deployed URL with the deployed credential through Open WebUI's own client, and asserts three things per server: it answers, it serves at least one tool, and the tool set contains the read probe its meta/mcp.yml declares.
Only bearer connections are verified. A system_oauth one resolves the caller's own upstream token per request and the administrator holds none, so it would fail on the caller rather than on the bridge; no provider declares auth: oidc today, but the filter keeps the spec honest when one does. The per-test budget scales with the number of registered servers instead of a fixed guess.
MCP_EXPECTED_TOOLS carries the declared read probe per server. Its rendering is unit-tested by extracting the block from the template itself rather than restating it, and decoded the way a spec decodes it - dotenv_quote emits an escaped JSON string that decodeDotenvQuotedValue unwraps before the spec parses the payload, so comparing after one decode would test a value no spec ever sees.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5