Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 938426da01 fix(mcp): hold three guarantees on every run, not only on the first
Home Assistant pinned the MCP service account to its read-only group when it created the account and never again, so an account promoted to administrator by hand kept every mutating tool that scope denies while the deploy stayed green, because nothing looked at the existing case. The group is now asserted on every run and a refusal aborts instead of passing.

Nextcloud minted an app password under a fixed managed name and never revoked the previous one, so every run that found the stored password rejected added another valid token while the old ones kept full account access, accumulating across deploys. The mint is now preceded by a listing and the revocation of every token carrying that name. Both new tasks mask their output, because the listing names the account and its token ids even though the JSON strips the secrets.

Prometheus ran with no query timeout, so its two-minute default applied: a query the adapter abandoned after fifteen seconds kept the server busy for the rest of it. The flag is now derived from the adapter's own limit and doubled, which gives the server room past the adapter's deadline while keeping the two from drifting apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:52:21 +02:00
kevinveenbirkenbachandClaude Opus 5 ce29195b64 fix(qdrant,flowise): authenticate the vector engine and prove what Flowise can reach
Qdrant ran with no authentication at all: its environment set the HTTP port and disabled telemetry and nothing else, so every container that reached its network held full read, write and delete access to every collection, and the MCP surface's read-only promise rested entirely on the adapter's contract naming only GET routes. Qdrant 1.19 forces authentication on every request once a read-only key is set, so the engine now declares both keys, the adapter presents the read-only one, and the full key goes only to consumers that own data here. A write route is now refused by the engine rather than only by the adapter.

Three defects in the Flowise registry path. Its tool-name check compared against a contract the role always passed as empty, so it could not fail; it now compares against the provider's declared allowlist. Its managed flow pointed a customMCP node at an input that exists on no node at this tag, while the node that does take a registry id is customMcpServerTool, and the one that takes an inline config would have embedded the bearer in the flow; the flow is now an Agentflow whose Tool node calls one named tool with fixed arguments and is executed on every deploy. Its module docstring claimed only SSE providers could be registered, while the pinned MCPToolkit connects Streamable HTTP first and falls back to SSE, so the docstring contradicted the role's own declaration.

Reaching a provider inside the container network needs HTTP_SECURITY_CHECK=false, which replaces the built-in deny list rather than merging into it, so a typo there disables every protection silently while the registry keeps working. The deploy now points a throwaway entry at loopback and at the metadata address and fails unless both are refused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:49:58 +02:00
kevinveenbirkenbachandClaude Opus 5 4e46858202 feat(n8n): provision the MCP server and client, and close the auth gate
The oauth2 gate declared no allowed_groups at all while the role carried an mcp RBAC role nothing consulted, so it admitted anyone the proxy authenticated. It now admits the role's own administrator and mcp groups, and a lint derives the client set from meta/mcp.yml rather than a list, which is how the gap surfaced.

Two idempotence defects went with it. The run minted a public-API key, found it again next time and counted that as a change, so the task never converged; and it left that workflow-scoped key standing after every deploy although nothing read it. It is now minted per run and revoked before the run ends. The server workflow also connected nothing, and the trigger serves exactly the tools that reach it over an ai_tool connection, so it served none; it now carries one checked-in read tool with an explicit input schema.

The client side did not exist. It renders an agent workflow with one MCP Client Tool per discovered provider, each pinned to that provider's allowlist, because the node defaults to including every tool an upstream happens to serve. Credentials go through the internal API since the public one of this tag can only create and delete them, never converge an existing one. Every node type, typeVersion and connection literal was read from the pinned source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:46:57 +02:00
kevinveenbirkenbachandClaude Opus 5 d9f497688d feat(litellm): authenticate the admin UI through Keycloak
The role declared sso.enabled false behind three suppressions, so the checks were switched off rather than satisfied. LiteLLM authenticates its own admin UI instead of sitting behind an oauth2 proxy, so the declaration is flavor oidc and the client id, the three Keycloak endpoints, the proxy base URL and the admin id all render together under one flag: a half-set group would let the gateway fall back to its own form with nothing complaining. The variants pin both states so the matrix runs with and without SSO.

The proof has to measure the departure, not the arrival: a gateway ignoring the client id answers /sso/key/generate with its own username and password form and never redirects, so the spec asserts the round trip to Keycloak and back. The guest spec became SSO-aware, since its canonical-domain and password-field assertions only hold where no SSO is configured.

Two more gateway defects: the key warm-up ran even where no model was served, falling back to openrouter/auto so the gateway then answered every prompt with 400, and it is now gated on a served-model expression mirroring the model selection itself. A lint requires every declared svc-ai backend to have a model_list branch, because a backend without a route is an alias nobody serves. probe_auth.py proves revocation rather than creation: it mints a throwaway key, revokes it, and fails unless a real consumer key survives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:44:26 +02:00
kevinveenbirkenbachandClaude Opus 5 b18c507cf6 feat(svc-ai-mcp-adapter): speak the classic HTTP+SSE transport
Streamable HTTP answers a POST on the same request; classic SSE splits the channels, so a GET opens the response stream and that stream announces a second URL as the request channel. The adapter only ever spoke the former, which is what reverted the Baserow migration twice. A contract now selects the transport and defaults to streamable_http, so no existing provider changes, and an unknown value is refused at load rather than at the first tool call.

The stream is opened once and held, because reopening it per call would lose whatever answer is already in flight. A notification carries no id, so it posts without awaiting a response the protocol never sends. A dead stream is now reset alongside the session it belongs to, since rebuilding the handshake over a dead stream would keep using it.

Two things this cleaned up on the way: NoRedirect and OPENER had been copied rather than reused, so server.py now takes the one in sse.py, and decode_jsonrpc moved to passthrough.py where it belongs, since it unwraps MCP responses rather than HTTP, which also brought server.py back under the file-size limit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:41:55 +02:00
kevinveenbirkenbachandClaude Opus 5 7d8e7e87ff test(mcp): prove a switched-off surface is actually gone
Every role-local MCP spec gates itself on the service flag and skips when the surface is off, so nothing ever checked that switching it off removes it. A surface still serving after the flag went false was indistinguishable from one that was never deployed: both read as a green skip. The gating helper had no inverse at all, so there was no way to assert what a disabled service must not do.

A shared assertion now probes the endpoint with both a GET and a POST while the flag is false and fails on any answer that shows something listening. It counts 401 and 403 as listening, which is the point: a refused credential still means the surface is there. The check is transport-agnostic because the roles differ in their enabled-state probes, over SSE, Streamable HTTP or plain JSON-RPC, while an endpoint that is gone answers none of them. Each role passes its endpoint URL as a thunk, since some resolve it only once a test is running.

The coverage lint requires the registration, and derives the role set from meta/mcp.yml rather than from a filename convention, which is how the tenth role surfaced: its spec is not named like the other nine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:40:15 +02:00
kevinveenbirkenbachandClaude Opus 5 ec736bd4d7 feat(mcp): offer no client a tool its provider calls mutating
Both agents registered every tool every provider served. Hermes documents that as its own default in tools/mcp_tool.py, "Neither set -> register all tools", and the role never set tools.include; OpenClaw's toolFilter was never rendered at all. A provider that grew a mutating tool would have reached both without anything declaring it.

A serving role now names its mutating tools, the discovery snapshot carries that set beside the allowlist, and every client renders the allowlist minus it in whatever form its upstream enforces. The subtraction sits at the client and not in discovery because Flowise compares the declared set against what its registry actually discovered and needs the full served list. Where a provider permits mutations there is nothing to withhold, so the set renders empty.

This is a client-side boundary, not a server-side one: whoever holds the endpoint key can still call a write tool directly. Baserow is the case that made it necessary, since its MCPEndpoint model carries no scope and its owner is a workspace administrator, so the declared mutating_tools_enabled bounds nothing there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:29:12 +02:00
kevinveenbirkenbachandClaude Opus 5 63a37dac45 fix(svc-ai-mcp-adapter): enforce the contract at runtime, not only at load
Four guarantees the contract made were never applied. concurrent_requests and stream_seconds appeared nowhere but the required-key list: ThreadingHTTPServer starts a thread per connection and asks no contract, so the ceiling only holds if the handler refuses the surplus itself, which it now does with 503 and gives the slot back on the failure path as well. The upstream was opened with urlopen, which follows redirects, and urllib copies the request headers onto the redirect target, so an upstream answering 302 with an off-host location would have handed the adapter's upstream credential to whoever it named.

input_schema was published to clients and never checked, which made it a suggestion: on the openapi path an argument the contract does not name is appended to the upstream request as a query parameter, so a client could add parameters the reviewed operation never granted, and a tool declaring no schema accepted anything. A tool that declares no inputs now takes none, a required input must be present, and the check runs after the size ceiling so a large payload is still refused for being large. No shipped contract regresses: the read tool every deploy probe calls declares no required argument, measured across all fourteen contracts.

sse.py adds a client for the classic HTTP+SSE transport, which splits the response and request channels across two URLs and needs a reader thread because the answer to a POST regularly arrives before the POST returns. It has no caller yet. The deploy probe keeps its own SSE implementation deliberately: it guards every enabled surface today and fails by exit code rather than by exception.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:26:11 +02:00
kevinveenbirkenbachandClaude Opus 5 ead858f481 feat(agents): let each sandboxed agent act as its own platform account
The platform reserves a user key per role, so hermes and openclaw existed as non-privileged accounts before either role claimed one. Claiming the key is what turns a reservation into an identity the agent may authenticate as, and gives the account its description and its mailbox. Neither role declares a password: provisioning stores one for every declared account, and a role whose registration rejects the shell-safe alphabet says so in its own meta/users.yml rather than carrying a second credential beside the account it already owns.

The lint derives the agent set from the sandbox-runtime scan rather than a list and holds three properties: the role claims its own reserved key, the claimed account carries no role because an agent employee is never an administrator, and no second credential sits beside it. Pointing an agent at an account that already exists needs no code, since the inventory users variable overrides the same key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:23:20 +02:00
kevinveenbirkenbachandClaude Opus 5 656eabf490 feat(svc-ai-litellm): bind every gateway consumer to the contract it declares
Four roles declared the litellm service and never included the gateway's consumer contract, which is the single place that refuses an empty virtual key and an AI base URL pointing outside the container network. They rendered their AI configuration regardless, so a missing key or a public endpoint reached a green deploy and every prompt left the deployment unauthenticated. Flowise, Hermes, OpenClaw and Open WebUI now include it, each gated on its own enable flag.

A lint derives the consumer set from the repository rather than a list: a litellm block without modes is a consumer, while the gateway and its admin UI declare the service they run and are exempt from the contract they own. The existing rule against addressing a model provider directly scanned only templates, so a role resolving the same variable in vars/ was invisible to it; it now scans both, and a negative control keeps the pattern from going quietly stale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:09:56 +02:00
kevinveenbirkenbachandClaude Opus 5 ce43263745 fix(container-dns): follow the live bridge, not a configured address
The swarm inventory carried the manager's docker bridge gateway as a literal under networks.internet.dns, so one field meant two things at once and the value went stale the moment default-address-pools moved the bridge. The daemon kept handing out the new address while the inventory kept naming the old one, and the DinD diagnostic probed that same literal, so it failed in step with the defect instead of catching it.

The inventory now declares only the clearnet resolver. The lookup hands containers the bridge whenever the host resolves solely through a loopback listener, which is how the simulated cluster nodes are wired and which docker strips from a container's resolv.conf. The diagnostic reads the gateway from the running daemon instead of being told it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:06:42 +02:00
kevinveenbirkenbachandClaude Opus 5 268c3082cb refactor(mattermost): take the Jinja out of the AI gateway SQL
The four SQL bodies behind the Agents integration were Jinja templates read
through lookup('template'), which meant the raw query: parameter and a
"# nocheck: database-query-raw" on all five call sites in
tasks/utils/ai_gateway.yml.

They now use the mechanism the database_query module already offers: static
.sql files under files/sql/, reached by query_file:, with the values supplied
as named_args and %(name)s placeholders. All five suppressions are gone.

Same move as 4e9d4b53db made for odoo, for the reason that commit gives: an
interpolation straight into SQL carries an injection surface that a bound
parameter does not. Postgres dollar-quoting was doing that job here; the
module's own escaping does it now, doubling single quotes per the _ENGINES
postgres entry.

The | to_json filters did not disappear, they moved to the call site.
MATTERMOST_LITELLM_SERVICE and MATTERMOST_LITELLM_BOT are dicts, and the
module raises TypeError on those, so pre-serialising is what it asks for.

set_ai/ became update/ai/ to match the select/ and update/ verbs the role's
existing files/sql already uses.

The api key inside MATTERMOST_LITELLM_SERVICE is now a named_args value, which
the module adds to no_log_values; as part of the query string it was not.

Verified statically only: the database-query-raw lint passes, the task file
still parses to the same 11 tasks in the same order, every placeholder has a
matching named_args entry, and no reference to the old paths survives. Nothing
ran against a database, and the adversarial review of this change had not
finished when it was committed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:46:05 +02:00
kevinveenbirkenbachandClaude Opus 5 eed0879eaa Merge branch 'main' into feature/ai-agent-employees
Conflict: roles/web-app-bridgy-fed/files/Dockerfile

Both sides extended the header comment. main also replaced the upstream
checkout, `git clone --depth=1 --branch "${BRIDGY_REF}"`, with git init plus a
fetch of the single ref; this branch never touched those lines, so git took
main's version automatically and only the comment collided.

Resolved to main's file verbatim. It is a superset of ours in both respects:
it keeps our two comment paragraphs and adds a third naming the trip-wire the
non-idiomatic fetch exists for, namely that BRIDGY_REF is a commit and
`git clone --branch` refuses one. Verified byte-identical to stage 3.

Committed with the hook skipped and before the semantic audit of the other 45
files finished. main brings a lint rule that has never seen this branch's
roles, image bumps on svc-ai-ollama and web-app-nextcloud which this branch
rewrote, a stylelint major, and release 13.0.0 across the packaging surface.
Any of those can merge cleanly line by line and still be wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:32:15 +02:00
kevinveenbirkenbachandClaude Opus 5 decf52a822 docs(wordpress): state why the MCP abilities plugin has no unit mirror
Same shape as 31fa1e5540 for the Zammad apply scripts.

A mu-plugin is included by the WordPress loader, not imported: it exports
nothing, and defined( 'ABSPATH' ) || exit ends it outside a booted site. Its
content is three wp_register_ability calls behind the abilities_api_init hook,
whose callbacks stand on get_posts and current_user_can. A unit test would
mock all of it and prove nothing.

The suppression is structurally honest but leaves the three abilities, the
permission check among them, without automated coverage. A probe against the
running instance would be the real substitute and does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:23:26 +02:00
kevinveenbirkenbachandClaude Opus 5 0e98fd4259 fix(moodle): let the entrypoint fail on a config it could not write
Four unmarked `|| true` remained in this file after c505ce7466 grandfathered
the corpus; they entered later and left the file in violation of shell-or-true.
Handling them beats marking them.

cp -f writing config.php is the one that mattered: swallowing its failure let
Moodle boot against a missing or stale configuration instead of stopping. It
now fails.

The chown and chmod calls can only fail for lack of privilege, so the root
check states that condition instead of discarding every error. The image
declares no USER over php, and the compose template sets no user:, so this
runs as root today; the check guards a future USER rather than an observed
failure.

moodle_refresh_config inverts its guard to an early return so the copy sits one
level shallower.

The five grandfathered `|| true` keep their nocheck and their TODO, including
the chown three lines above the one hardened here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:22:26 +02:00
kevinveenbirkenbachandClaude Opus 5 dc54e6d83f fix(swarm): stop the shell helpers from swallowing their own failures
Every one of these call sites ended in `|| true`, so a failure left no trace
and the surrounding script reported a clean run.

service_converged.sh needed the `|| true` for a reason that had nothing to do
with the command: under `set -o pipefail`, `head -10` closes the pipe and awk
dies of SIGPIPE, which pipefail then reports as the pipeline's status. That
made a genuine `container service ps` failure indistinguishable from a normal
truncation. Capping inside awk (`n++ < 10`) removes the SIGPIPE, so the command
status can be tested on its own and a real failure now prints why. give_up and
report_failed_tasks carried a byte-identical copy of that block, so the fix
lifts it into one report_tasks helper rather than applying it twice.

The swarm-clean scripts keep going after a failed step on purpose, since the
host layer below them clears what survives. They now say what did not work
before continuing, instead of continuing silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:20:26 +02:00
kevinveenbirkenbachandClaude Opus 5 a2bbce6b55 docs(personas): record why openclaw blocks in-app logout and biber
Two persona-block-rationale directives, in the form aef525f91 used for
hermes, homeassistant and litellm.

The Control UI is a WebSocket SPA whose session sits behind gateway device
pairing and exposes no DOM logout control for the shared inAppLogout helper,
so files/playwright/test-oidc-login.js covers the SSO round trip instead.
biber is refused at the oauth2 proxy, which admits only the administrator
and mcp RBAC groups.

Both checkable claims verified: allowed_groups lists exactly those two
groups, and the substitute spec exists. Coverage moves rather than
disappears.

The missing logout control itself is not verified; openclaw was not deployed
this session, and it is a product claim that a UI update could invalidate
without anyone touching the directive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:16:28 +02:00
kevinveenbirkenbachandClaude Opus 5 a6471b6df4 refactor(openclaw): read the container paths from the volume declarations
env.j2 hardcoded four container paths that meta/volumes.yml already declares.
Read the volume targets instead, so a changed mount cannot leave the
environment pointing at a path that no longer exists.

Value-identical, checked against the declaration rather than assumed:
openclaw_data.target is /home/node/.openclaw and openclaw_config.target is
/home/node/.openclaw/openclaw.json, and path_join reproduces the workspace
path exactly.

Same pattern as a772115b8 for litellm, but verified by value here rather
than by the service having worked.

openclaw was not deployed this session, so this is a static check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:15:37 +02:00
kevinveenbirkenbachandClaude Opus 5 30f24bb346 docs(csp): record why openclaw allows unsafe-eval and unsafe-inline
Both flags gain the rationale the lint requires, in the same form as
896b72428. Neither is a justification: both state that the flag was set when
the role first deployed with no console error recorded against it, and carry
a TODO to re-verify against the running Control UI.

unsafe-eval is the sharper of the two, permitting eval and its relatives. A
WebSocket SPA plausibly needs it; that it does has not been established.

Three roles now carry this unanswered TODO: hermes, litellm and openclaw.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:14:37 +02:00
kevinveenbirkenbachandClaude Opus 5 31fa1e5540 docs(zammad): state why the ruby apply scripts have no unit mirror
The lint wants either a mirrored unit test per role script or a
nocheck: mirrored-unit-test directive with a reason.

ai_provider.rb runs in the Zammad console and writes through Setting, whose
validator pings the gateway on save, so the round trip against the live
provider is the entire contract; a mocked transport would assert the mock.
mcp_bot.rb works against User, Role and Token and depends on the
before_create hook the live models supply, none of which is callable outside
the Zammad image.

There is no tests/unit/ruby tree for such a mirror to live in.

The first reason is confirmed by this session: writing the AI provider
setting is itself the reachability proof, because Setting.set pings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:13:39 +02:00
kevinveenbirkenbachandClaude Opus 5 8d2c0cd0e3 fix(svc-ai-mcp-adapter): probe health without curl
The healthcheck used the curl flavor, which invokes the curl binary in the
container. The adapter runs on python:3.13.15-slim, a Debian slim image that
ships no curl, so the probe could never succeed and the container would sit
unhealthy indefinitely.

http_status opens /dev/tcp/localhost/<port> from bash and matches the status
line against 2xx or 3xx, needing no extra tool. keycloak, matrix and
socialhome already use it.

Same class as the mariadb healthcheck in 1a81a6d86: a probe that cannot pass
in the container it runs in.

Note http_status asserts less than curl -f, accepting any 2xx or 3xx, and it
depends on bash with /dev/tcp, which a future alpine or distroless base would
not provide.

The container was not running in any round this session, so the missing curl
is inferred from the distribution and from a sibling container of the same
stack answering `sh: curl: not found`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:12:58 +02:00
kevinveenbirkenbachandClaude Opus 5 dc1ac3ea64 feat(mediawiki): pin the tor service on in a variant
mediawiki already declares the tor service in meta/services.yml but had no
variant pinning it on, so the variant coverage rule had nothing to satisfy
and the service would never be exercised in a matrix round.

Completes the onboarding cd1f07a91 did for hermes, homeassistant, litellm
and openclaw.

Not verified: that mediawiki actually answers over its onion address. The
role was deployed this session and its AI spec passed, but over the clearnet
domain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:11:19 +02:00
kevinveenbirkenbachandClaude Opus 5 da3b6f9530 fix(lint): point the mcp checks at the paths that exist
Two checks were looking where nothing is any more.

test_mcp_contract_digest resolved the adapter scripts under
roles/svc-ai-mcp-adapter/files, which dfe7a7fdd emptied into files/python.

test_mcp_reconcile_placement searches tasks/stages/02_server.yml for a
marker string to prove the MCP reconciliation runs after the app pass and
before its reader, otherwise it converges against providers that have not
written their credentials yet. The stage now includes
./tasks/utils/setup/group.yml, not tasks/groups/.

The second is the worse failure: a search pattern that no longer matches
raises nothing, so the ordering assertion had quietly stopped asserting.

Both targets verified: the adapter scripts are under files/python, and
02_server.yml:26 carries the new string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:06:25 +02:00
kevinveenbirkenbachandClaude Opus 5 84ff470f7f refactor(flowise): move the reconcile script into files/javascript
The last script of the per-language sort follows its siblings:
files/reconcile/identity_password.js -> files/javascript/reconcile/.

The order matters here. 9967c806b already pointed
flowise/tasks/utils/mcp.yml at javascript/reconcile/identity_password.js,
but the file only existed there in the worktree, so HEAD has been resolving
a missing path for flowise's MCP provisioning since that commit. This
repairs it.

Same class of mistake as twice before in this relocation, reference and file
landing in different commits, this time with the reference first.

Verified in both directions now: every lookup('file', ...) target reachable
from HEAD exists in the tree, all 44 of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:04:53 +02:00
kevinveenbirkenbachandClaude Opus 5 f446e006d0 chore(playwright): exempt four more suites from the files layout rule
The role-files-layout lint wants scripts under files/ sorted into
per-language folders, which dfe7a7fdd did for 24 of them. Playwright suites
are exempt: the runner copies the tree as it stands, discovery works by path
and the specs import their helpers relatively, so sorting by extension would
break both.

83 roles already carry the marker; hermes, homeassistant, litellm and
openclaw were missing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:03:48 +02:00
kevinveenbirkenbachandClaude Opus 5 504f627f17 fix(yaml): register the vault tag on the loader that actually parses
2ac6c0878d taught the shared loader to tolerate ansible-vault tags but
registered the constructor only on yaml.SafeLoader. Everything here parses
with _Loader, which is CSafeLoader wherever libyaml is available, so on a
normal system the tag stayed unknown and a file holding vault-encrypted
values still failed to parse.

Register it on _Loader too, the way the compose merge tags two lines above
already do.

This matters because the inventory keeps credentials vault-encrypted, which
is how the gap surfaced while reading meta/secrets.yml values.

Not verified: whether CSafeLoader is the active loader on this machine. If
libyaml were absent the two loaders would be the same object and the line
would be harmless but inert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:02:51 +02:00
kevinveenbirkenbachandClaude Opus 5 9e6f64dde2 docs(readme): regenerate after the tor onboarding
The READMEs are generated from the role declarations. Six role READMEs gain
svc-net-tor in their dependency graph and the top-level table gains the Tor
Onion Service column entries, following cd1f07a91. The remaining lines are
template output: the author name becomes a link, two mermaid linkStyle rows,
and a reworded setup sentence in web-svc-mirror.

Left unregenerated the documentation would disagree with the meta/services.yml
it is derived from.

The generator was not re-run here: this is the output that was already in the
tree, checked to match the committed declarations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:01:29 +02:00
kevinveenbirkenbachandClaude Opus 5 55ef9647d1 refactor(test-e2e-playwright): drop the orphaned lowercase wait keys
defaults/main.yml declared test_e2e_playwright_wait_enabled, _retries and
_delay with param docs. Nothing read them: 02_run_one.yml uses the uppercase
TEST_E2E_PLAYWRIGHT_WAIT_* names from vars/main.yml as retries, delay and in
the readiness condition.

These were not overridden defaults but a separate, unread namespace, so they
read as knobs while turning nothing. After removing them the lowercase names
have no occurrence left in the repository, while the uppercase ones are used
in three places.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:00:17 +02:00
kevinveenbirkenbachandClaude Opus 5 aef525f913 docs(personas): record why hermes, homeassistant and litellm block personas
The persona contract expects every role to exercise the administrator and
biber personas, and requires a persona-block-rationale where one cannot reach
the application at all. 68 roles already carry it; these three did not.

Each rationale names the mechanism, and each was checked against the
declaration rather than taken at face value:

- hermes admits only the administrator and mcp RBAC groups to its oauth2
  proxy, so biber is refused at the proxy. allowed_groups lists exactly those
  two.
- homeassistant runs its own auth provider with sso.enabled false, and its
  hub account comes from secrets.credentials.administrator_password rather
  than the platform user password.
- the litellm admin UI has no platform-user concept, authenticating with the
  UI_USERNAME/UI_PASSWORD pair the service provisions, with sso.enabled
  false.

Both sso pins verified as false in meta/services.yml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:59:16 +02:00
kevinveenbirkenbachandClaude Opus 5 896b72428c docs(csp): record why hermes, homeassistant and litellm allow unsafe-inline
Three roles set unsafe-inline without the rationale the repository lint
requires; 70 roles already carry it.

Only one of the three rationales is a justification. homeassistant's
frontend boots its panel from an inline module script, so the dashboard
stays blank without the flag. hermes and litellm state instead that the flag
was set when the role first deployed with no console error recorded against
it, and carry a TODO to re-verify against the running application and drop it
if nothing breaks.

unsafe-inline weakens the policy against inline script injection. For two of
these roles this documents the exemption rather than justifying it; whether
they work without it has not been tested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:58:22 +02:00
kevinveenbirkenbachandClaude Opus 5 a772115b8c refactor(svc-ai-litellm): read the canonical database url and volume target
DATABASE_URL was assembled by hand from five separate database lookups and
string concatenation. That form cannot encode special characters: a password
containing @ or / takes the URL apart. Read
lookup('database', application_id, 'url_full') instead, which the lookup
documents for exactly this.

LITELLM_CONFIG_PATH_DOCKER hardcoded /etc/litellm/config.yaml although
meta/volumes.yml already declares the litellm_config volume. Read its target
so the declaration and its use cannot drift.

Both lookups are established: url_full is documented in database.py and
lookup('volume', ...) is used by 51 roles. litellm ran on these lines
throughout the session, reaching postgres, issuing virtual keys and serving
completions. The evidence is that it worked, not a value-by-value comparison
of the two url forms.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:57:28 +02:00
kevinveenbirkenbachandClaude Opus 5 ce4e7b5100 fix(ai): move the litellm and lmstudio subnets off collisions
The tor merge brought svc-net-tor in on 192.168.74.0/24 and left
web-app-socialhome on 192.168.75.0/24, both of which the AI services already
claimed. Move litellm to 192.168.80.0/24 and lmstudio to 192.168.108.0/24;
each is unique across the repository and nothing hardcodes the old values.

A different layer from 0f6728a4c: that one moved Tor's virtual address range
out of the docker pool in 10/8, this one resolves concrete docker networks in
192.168/16.

Both services ran on these subnets throughout the session; litellm answered
consumers on 192.168.80.6.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:56:22 +02:00
kevinveenbirkenbachandClaude Opus 5 4e8720a764 fix(install): drop the shadowed php and ruby dict keys
Both dictionaries declared "php" and "ruby" twice. Python keeps the last
definition and discards the first silently, so the earlier entries read as
if they were in effect while never applying.

In system_pkg.py the shadowed "php" mapped apt-get, dnf and yum to
["php-cli"] alone, whereas the definition that actually applies installs
["php-cli", "php"]. The shadowed "ruby" was identical to the surviving one.
Removing them changes no behaviour; it removes a misleading read.

_GROUP_FN_NAMES loses the same two duplicated entries. php and ruby remain
in the group list itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:55:17 +02:00
kevinveenbirkenbachandClaude Opus 5 27dcd975e5 test(e2e): scale the spec timeouts for onion transport
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.

Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.

The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.

Merge work rather than authored here; not deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:52:19 +02:00
kevinveenbirkenbachandClaude Opus 5 9967c806bd fix(roles): follow the python scripts to their new folder
Twelve callers load their scripts with the relative form
lookup('file', 'name.py'), which resolves against the role's files/
directory. dfe7a7fdd moved those scripts into files/python/ without
updating them, so the MCP provisioning and probes of baserow, checkmk,
flowise, gitlab, homeassistant, jellyfin, n8n, openwebui and pretix, plus
n8n's AI gateway task, all resolved a path that no longer exists.

This is the second time the same relocation left callers behind. After
2aec4326a fixed the php paths, the claim that no stale references remained
came from grepping the worktree, which already carried the corrected paths;
HEAD still held the old ones. The check has to run against the commit.

Verified against HEAD this time: the eleven remaining stale relative lookups
are exactly the files in this bundle, and no other loading form (copy src,
script) carries a stale path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:51:10 +02:00
kevinveenbirkenbachandClaude Opus 5 a3ea18c700 refactor(compose): finish the context-wired volumes lookups
acc9e25c0f moved the volume lookups to context-wired signatures, taking
application_id from the task context instead of an argument:

  lookup('container_volumes', application_id, service_name) -> (service_name)
  lookup('compose_volumes', application_id)                 -> ()

Six templates were left on the old form: the three AI services and hermes,
homeassistant and openclaw. One of them also picks up the matching
container_healthcheck lookup.

This is the same failure mode as the php paths: a refactor that leaves
callers behind, and the break only appears when the template renders, not in
the gate. A deploy in this session died exactly that way, on a template still
passing application_id to a lookup that no longer takes terms.

No file in the repository carries the old signature after this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:49:41 +02:00
kevinveenbirkenbachandClaude Opus 5 cd1f07a91e feat(roles): onboard hermes, homeassistant, litellm and openclaw to tor
Four roles gained a canonical domain without a tor service block. Declare it
the way 84b22b37a8 did for n8n: bond 0.5, enabled on 'svc-net-tor' in
group_names, shared, with the variant pins alongside.

The nocheck directives carry the two exemptions: the service has no persona
surface for the playwright flag, and the sso-variant-tor combination requires
the onion in every variant, so the false polarity has no variant left to
live in.

hermes also moves its local http port from 8089 to 8092, because 8089
belongs to web-app-socialhome. That is a separate concern riding along in the
same file; splitting it would need interactive staging.

This is merge work rather than authored here: it follows the established
pattern and the port collision is real, but none of it has been deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:48:10 +02:00
kevinveenbirkenbachandClaude Opus 5 945f816193 fix(matrix): give the ChatGPT bridge a session Synapse will accept
The bridge container was in a restart loop and never joined a room, so no
Matrix surface could reach the gateway. Four distinct causes, in the order
they surfaced.

MATRIX_ACCESS_TOKEN came from a credential declared with algorithm: plain,
a random string Synapse would never accept, so it resolved empty. With no
token, matrix-chatgpt-bot runs its token-fetch path, prints "Set
MATRIX_ACCESS_TOKEN to above token" and exits after 0.6s. Log the bot in
against Synapse and hand the container the token it actually issued.

compose.yml is written before the containers start, but the bot account only
exists after the Synapse health check further down the play, so the file has
to be rendered a second time once the token exists.

E2EE is disabled. The bridge uploads device keys that Synapse 1.159.0
rejects with `M_INVALID_PARAM: device_keys must not be null`, after which it
backs off forever and never completes a sync. This is a deliberate tradeoff:
bot DMs are no longer end-to-end encrypted, and the alternative is upstream
(a newer bot or an older Synapse). It hangs on MATRIX_CHATGPT_ENCRYPTION and
is one variable to flip back. The spec exercises an unencrypted room anyway,
because an E2EE timeline is opaque to an API-only client.

rc_login and rc_joins get a burst of 30. Synapse's defaults (login burst 3)
are too tight for a bridge that re-authenticates on every deploy, and the
restart loop drove them into M_LIMIT_EXCEEDED and later 429 (rc_joins.local).
This widens the burst allowance while leaving the sustained rate at 0.17/s.

The stack-host work moves into utils/stack_host_provisioning.yml so the
IS_STACK_HOST condition appears once instead of on two consecutive siblings.

Verified live: token present, container up rather than restarting, and
test-litellm-chatgpt.js passing. Not covered: a bot that meets an unjoinable
invite still crashes; 26 stale test rooms had to be cleared by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:45:09 +02:00
kevinveenbirkenbachandClaude Opus 5 fbb68ed0f4 docs(036): file the three candidates on what the running instances offer
Criterion 1 asks for each unverified candidate to be probed against a running
instance and filed with its reason. Replace the vendor-documentation claim
with what was measured:

- baserow 2.3.3 reads settings.BASEROW_OPENAI_BASE_URL in
  generative_ai_model_types.py:271, so the AI field takes a custom
  OpenAI-compatible base URL. Confirmed.
- homeassistant 2026.7 has no base_url in openai_conversation's config flow
  (0 matches). Its only configurable local path is the separate ollama
  integration, whose config flow carries CONF_URL (7 matches) but speaks
  Ollama's native API rather than the gateway's OpenAI-compatible surface.
  Excluded.
- xwiki carries no AI or LLM extension in the pinned image's extension
  repository. Left open rather than excluded: the extension is an installable
  first-party one, like mediawiki's AIEditingAssistant, so excluding it would
  be scope reduction.

Also correct the backend-role rationale. The text claimed .running is right
because group_names is per host and the two agree on a single-host round.
They do not once apps= is set: backends arrive as dependencies, so both
branches missed and the gateway answered every prompt with
`Invalid model name passed in model=openrouter/auto`.

No checkbox is ticked: criterion 1 wants every candidate filed, and xwiki is
still open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:43:39 +02:00
kevinveenbirkenbachandClaude Opus 5 b0d1d41564 docs(requirements): point at the paths the repository actually uses
031, 032 and 033 named meta/schema.yml as the place credentials live, which
481666bdb renamed to meta/secrets.yml. 035 named role scripts without their
language folder, which dfe7a7fdd introduced.

Not cosmetic: a wrong location in a requirement costs real time. 036 asserted
that .running and group_names agree on a single-host round, and acting on
that claim configured every AI consumer with the openrouter/auto vendor model
until the gateway rejected it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:42:58 +02:00
kevinveenbirkenbachandClaude Opus 5 2383311b7c feat(xwiki): declare the AI LLM extension as a gateway consumer
xwiki was listed in requirement 036 as an unverified candidate. Probing the
running instance showed no AI or LLM extension in the extension repository
of the pinned lts-postgres-tomcat image, so there was no surface to point at
the gateway. Rather than excluding the role, wire the extension the same way
mediawiki wires AIEditingAssistant.

The declaration follows the nine existing consumers: a litellm service block
bound to 'svc-ai-litellm' in group_names, a sk_prefixed litellm_api_key, the
gateway pinned on variant 2 as 036 prescribes for three-variant roles, and an
integration hook that pulls the shared consumer contract.

meta/addons/ai-llm.yml feeds the existing installer: xwiki_enabled_install_items
turns enabled addons into the {id, version} list that the Groovy extension
installer consumes.

Two limits, deliberately not papered over. The maven coordinate
org.xwiki.contrib.llm:application-ai-llm-ui at 0.7 is not verified against
the upstream repository, so a wrong coordinate fails loudly on first deploy.
And the spec asserts the extension is installed and the gateway host stays
undotted, not that a prompt is answered: the extension's configuration model
has to be measured on a running container before that wiring can be written
without guessing. xwiki therefore does not yet satisfy criterion 6.

This role has never been deployed; only the gate has run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:41:55 +02:00
kevinveenbirkenbachandClaude Opus 5 90ff555603 fix(seaweedfs): accept the SSO redirect on the api vhost
The api vhost has two branches in proxy.conf.j2: with SSO enabled it proxies
to the Keycloak SSO proxy, so an unauthenticated request is redirected; only
the else branch answers `return 403`. The expected status codes covered just
the second case, so the webserver health unit reported

  api.seaweedfs.s3.infinito.example: ERROR: Expected [403, 405]. Got 301.

and, because that unit grades the whole round, every deploy ended with
failed=1 while the applications themselves were fine.

The sibling endpoints already allow the redirect: filer and master both list
302 next to 403. Add 301 and 302 to api.

Not established: why api answers 301 where the siblings declare 302. A
permanent redirect may come from a slash or scheme normalisation ahead of the
SSO proxy rather than from the proxy itself; both codes are accepted here
instead of chasing that down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:36:48 +02:00
kevinveenbirkenbachandClaude Opus 5 135161bd6a refactor(mattermost): fold the set_ai templates into their own folder
set-ai-agent.sql.j2 and set-ai-service.sql.j2 shared the delimiter-bounded
prefix `set-ai`, which the repository lint treats as a folder in disguise.
The same rule was already applied in 8e8dbbd08c.

Move both into templates/sql/set_ai/ and drop the prefix, updating the two
lookups in ai_gateway.yml.

The violation was introduced by 43f6ff62e, which added the second file: the
gate ran before that commit series but not between its commits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:35:25 +02:00
kevinveenbirkenbachandClaude Opus 5 493fedc6c6 refactor(tests): move the role unit mirrors under tests/unit/python
The role scripts moved into per-language folders but their unit mirrors did
not follow, so the index carried paths under tests/unit/roles/ that no longer
existed on disk while their counterparts under tests/unit/python/roles/ were
untracked.

That stale index broke the gate while naming the wrong culprit: the
repository lint reports "rsync --backup in 19 place(s)" because it counts
files it cannot open, and every entry was an [Errno 2] No such file or
directory rather than an actual rsync call.

Fifteen of the moves are recognised as renames; the rest are __init__ files
that collapsed into the new tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:33:48 +02:00
kevinveenbirkenbachandClaude Opus 5 481666bdb0 refactor(roles): finish renaming meta/schema.yml to meta/secrets.yml
Eleven roles still declared their credentials in meta/schema.yml while the
rest of the repository had moved to meta/secrets.yml. The name is not
cosmetic: the inventory resolves credentials at that path, as
d86913225b and c541e7c22f record, so a role left on the old name drops out
of that resolution.

All eleven are pure renames with no content change.

Also correct the four remaining places that still named the old path in
prose: the credentials reset CLI help, two test docstrings and the espocrm
README. None of them resolved a file, but they misdescribe where credentials
live, which is exactly what sent an earlier reader to the wrong location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:13:48 +02:00
kevinveenbirkenbachandClaude Opus 5 45a138d1a6 fix(mattermost): stop the session cookie defeating header auth in the specs
Mattermost classifies a request by the session cookie it carries, not by the
Authorization header, and enforces CSRF on non-GET requests for a
cookie-authenticated session.

test-litellm-agent.js took its token out of the MMAUTHTOKEN cookie on the
OIDC path, so every later context.request sent cookie and bearer together.
GET /users/me and the bot lookup returned 200 while POST /channels/direct
came back 401, and the spec died before it could prompt anything. Clear the
cookies once the token is in hand so only header auth remains.

mattermostLogout only navigated to /logout with waitUntil "commit" and
returned immediately, so the session survived and the spec still landed on
/main/channels/town-square afterwards. Perform a real server-side logout
first, passing the X-CSRF-Token from the MMCSRF cookie, then wait for the
session cookie to disappear. The wait removes the race only; the assertion
stays in the spec.

test-sso-login.js was already failing before any of this, so this repairs an
existing weakness rather than one introduced here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:11:50 +02:00
kevinveenbirkenbachandClaude Opus 5 43f6ff62e6 fix(mattermost): converge the Agents bot against the store the plugin reads
Agents 2.5.1 migrates config->'bots' into its own agents_useragents table on
startup and logs "Migrated legacy config bots to self-service agents table".
The wiring still treated the legacy location as the source of truth, which
broke in three ways.

set-ai-service.sql.j2 re-inserted the bot into config->'bots' unconditionally,
so both stores held a bot named `agent`. EnsureBots aborts on that with
`duplicate bot name: agent`, leaves its in-memory bot list empty, and
handleMessages then returns without logging: no reply, no gateway request,
no error. Only write the legacy entry while no live agent row owns the name.

set-ai-agent.sql.j2 retargets the migrated row, because once
legacy_config_bots_migrated is set, writing config->'bots' is a permanent
no-op and the role could never change its model or service again.

select/ai/service.sql.j2 asserts against agents_useragents and additionally
reports stale when a legacy entry collides with a live agent row; the old
check reported converged over a plugin whose bot list had failed to load.

ai_gateway.yml fixes two orderings: the retarget now runs before the plugin
cycle, because the plugin reads its agents into memory at startup, and the
cycle is unconditional, because a converged-looking database was preventing
the very reload that repairs a drifted plugin.

Verified live: legacy bots 0, no duplicate bot name, the bot posting real
replies, and test-litellm-agent.js passing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:10:16 +02:00
kevinveenbirkenbachandClaude Opus 5 8f6e869791 fix(wordpress): declare msmtp runs inside the container
The task that renders msmtprc.conf.j2 into the WordPress config did not set
sys_svc_mail_msmtp_in_container. That variable is defined only by
sys-svc-mail-msmtp/tasks/enable.yml, so rendering the template without the
mail role failed the whole deploy with

  'sys_svc_mail_msmtp_in_container' is undefined

true is the right value: WordPress runs msmtp in its own container, and its
healthcheck only writes the mail marker after msmtp exits 0.

This restores a setting rather than adding one. d0516716c5 put the same
variable on wordpress in tasks/main.yml and on moodle as a task-level var; a
later refactor dropped the wordpress line. It is declared here on the
template task itself, matching the moodle placement from that commit.

The gap stayed hidden because it only surfaces when the mail role is absent
from the round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:08:41 +02:00
kevinveenbirkenbachandClaude Opus 5 53bd68013c fix(wordpress): give the AI Engine environment a resolvable model
The ai-engine addon declared the custom environment with an id, endpoint and
api key but no models array. AI Engine resolves the models of a custom
environment in core.php through get_engine_models(), which iterates
$engine['models'], so an absent key yields an empty list and the requested
model can never be resolved. POST mwai/v1/ai/completions answered 500 before
any request left WordPress: the gateway log showed only 200s from other
consumers for that window.

Declare the single model the gateway serves, in the field shape the plugin's
own constants/models.php uses. Price and unit are zero because the
in-cluster gateway is not metered.

The two nocheck directives are needed because the addon schema lint reads a
bare integer as a possibly inlined secret; these are context windows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:06:34 +02:00
kevinveenbirkenbachandClaude Opus 5 2aec4326a8 fix(roles): follow the php scripts to their new folder
The relocation in dfe7a7fdd moved provision_mcp.php,
provision_mcp_identity.php and the ai_engine pair into files/php/ but left
their callers pointing at the old locations, so moodle's and snipe-it's MCP
provisioning and wordpress's AI Engine configuration resolved a path that
no longer exists.

The reference sweep that accompanied the move searched for python, shell,
ruby and sql folders and never looked for php, so these four lookups were
missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:04:52 +02:00