Commit Graph
7 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 4e46858202 feat(n8n): provision the MCP server and client, and close the auth gate
The oauth2 gate declared no allowed_groups at all while the role carried an mcp RBAC role nothing consulted, so it admitted anyone the proxy authenticated. It now admits the role's own administrator and mcp groups, and a lint derives the client set from meta/mcp.yml rather than a list, which is how the gap surfaced.

Two idempotence defects went with it. The run minted a public-API key, found it again next time and counted that as a change, so the task never converged; and it left that workflow-scoped key standing after every deploy although nothing read it. It is now minted per run and revoked before the run ends. The server workflow also connected nothing, and the trigger serves exactly the tools that reach it over an ai_tool connection, so it served none; it now carries one checked-in read tool with an explicit input schema.

The client side did not exist. It renders an agent workflow with one MCP Client Tool per discovered provider, each pinned to that provider's allowlist, because the node defaults to including every tool an upstream happens to serve. Credentials go through the internal API since the public one of this tag can only create and delete them, never converge an existing one. Every node type, typeVersion and connection literal was read from the pinned source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:46:57 +02:00
kevinveenbirkenbachandClaude Opus 5 9967c806bd fix(roles): follow the python scripts to their new folder
Twelve callers load their scripts with the relative form
lookup('file', 'name.py'), which resolves against the role's files/
directory. dfe7a7fdd moved those scripts into files/python/ without
updating them, so the MCP provisioning and probes of baserow, checkmk,
flowise, gitlab, homeassistant, jellyfin, n8n, openwebui and pretix, plus
n8n's AI gateway task, all resolved a path that no longer exists.

This is the second time the same relocation left callers behind. After
2aec4326a fixed the php paths, the claim that no stale references remained
came from grepping the worktree, which already carried the corrected paths;
HEAD still held the old ones. The check has to run against the commit.

Verified against HEAD this time: the eleven remaining stale relative lookups
are exactly the files in this bundle, and no other loading form (copy src,
script) carries a stale path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:51:10 +02:00
kevinveenbirkenbachandClaude Opus 5 8e8dbbd08c refactor(layout): fold the prefix-shared AI files into their own folders
test_prefix_groups_belong_in_subfolders flagged three groups added by this change
set: a shared delimiter-bounded prefix is a folder in disguise, so the group moves
into a folder named after the prefix and the prefix leaves the file names.

  sql/select/ai-{bot-user,service}.sql.j2  -> sql/select/ai/{bot-user,service}.sql.j2
  files/provision_{ai,mcp}.py              -> files/provision/{ai,mcp}.py
  files/ai_engine_{configure,endpoint}.php -> files/ai_engine/{configure,endpoint}.php

All six are recorded as renames, so history follows them. Every reference moves
with them: four templated query paths in mattermost's ai_gateway.yml, the two
lookup('file', ...) calls in n8n, the two in wordpress, and SCRIPT_PATH plus the
module docstring in the n8n unit test. lookup('file', ...) resolves relative to
the role's files/ directory, so the new argument is provision/mcp.py rather than
a repo-relative path.

Worth recording why this survived every gate: make quality was green on each run
this session, including immediately before the commits that introduced these
files. The lint enumerates through git ls-files, and the files were untracked, so
it could not see them. Committing them is what made them visible. A green gate
taken before new files are tracked says nothing about the gate after.

Verified: repository lints go from 88 passed with 1 failure to 89 passed, the
unit suite passes at 4448, and a repo-wide grep for every old path returns
nothing.
Not verified: no deploy has run. The lookup('file', ...) arguments and the
templated SQL paths resolve at play time, so a wrong path would surface as a task
failure rather than a test failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:19:26 +02:00
kevinveenbirkenbachandClaude Opus 5 8c3d6fcde1 fix(n8n): give the managed MCP trigger the bearer credential it enforces
The mcpTrigger node forces authentication to bearerAuth, because the upstream
`none` option would publish an unauthenticated SSE endpoint on the role's own
vhost. Setting that parameter is only half the contract: the node must also
reference an httpBearerAuth credential holding the token. Without one the
trigger answers every request to <path>/sse with

    500 No authentication data defined on node!

before the SSE transport is ever opened, while the workflow itself looks
correctly configured and reports active.

The provisioning chain gains a credential leg between the API key and the
workflow: POST /rest/login for the owner cookie, /rest/api-keys for the
deterministic managed key, /rest/credentials for the bearer, then the public
/api/v1 routes for the workflow. The credential is upserted under the
deterministic name in N8N_MCP_CREDENTIAL, which is why that constant landed with
the AI bundle: the variable has to exist before this references it.

The workflow body is now pushed on every run, including while the workflow is
active. Previously a node that had drifted from the definition stayed broken
behind its own active flag, because the reconciler saw an existing active
workflow and left it alone. Pushing unconditionally lets a wrong node heal.

Verified: the /rest credential route with the owner cookie is exercised by the
sibling provision_ai.py, which ran in a variant-2 round, so the mechanism this
depends on works even though this call has not been observed.
Not verified: no MCP request against n8n has been seen answering 200 since the
change. Two consequences are deliberate but real -- an operator's intentional
edit to the managed workflow is now overwritten on every deploy, not just a
drifted one, and because n8n never returns encrypted credential data, a token
that drifted from N8N_MCP_TOKEN cannot be detected, only overwritten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:16:56 +02:00
kevinveenbirkenbachandClaude Opus 5 43ca7f73ae feat(n8n): provision the managed openAiApi credential against the gateway
AI Agent nodes read their endpoint from a stored credential, not from the
environment, so the gateway base URL and this role's virtual key have to land in
n8n's credential store. files/provision_ai.py logs in as the owner and upserts an
openAiApi credential under the deterministic name infinito:litellm.

The write goes through the internal /rest routes because that is the only place
the pinned 1.95.3 exposes credential creation: the public /api/v1 key this role
already mints for MCP carries workflow scopes and cannot create a credential.
That is why this is a second script rather than another call inside
provision_mcp.py.

n8n encrypts credential data at rest and never returns it, so there is nothing
to compare against and the managed credential is overwritten rather than
reconciled. The script prints CHANGED or OK so the task can report change state
without pretending to have diffed a value it cannot read.

env.j2 sets only N8N_AI_ENABLED. An earlier revision of this work templated
N8N_INSTANCE_AI_MODEL_URL, which does not exist in 1.95.3 -- it was invented
rather than measured, and has been removed along with the requirement table row
that repeated it. The one real AI environment variable in that release,
N8N_AI_ASSISTANT_BASE_URL, expects n8n's own assistant service rather than an
OpenAI-compatible endpoint, so the credential store is the only usable surface.

vars/main.yml also carries N8N_MCP_CREDENTIAL_NAME. It belongs to the MCP work
rather than to this change, but a file cannot be split across commits and this
order is the safe one: the MCP tasks land later and find the constant already
present, where the reverse would reference a variable that does not exist yet.

Verified: the spec runs rather than skipping -- it appears in the variant-2
JUnit as "administrator: the managed openAiApi credential answers a prompt
through the in-cluster gateway".
Not verified: I confirmed it executed, not its verdict. The owner-cookie login
is also unexercised on an sso-only variant, and because the credential is
rewritten every run, an operator's manual edit is replaced without notice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:44:34 +02:00
kevinveenbirkenbachandClaude Opus 5 a423bde6cc refactor(mcp): unify client entrypoints and read provider tokens through one lookup
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.

Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.

The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.

The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.

The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:06:31 +02:00