Commit Graph
51 Commits
Author SHA1 Message Date
evangelostsak 5092aab4c2 feat(web-app-openclaw): let a deployment put its hosts into meshes 2026-10-16 03:40:19 +02:00
kevinveenbirkenbachandClaude Opus 5 328113e7c3 refactor(ai): declare the litellm consumer contract once
Fourteen roles repeated the same two lines to say they use the gateway: a
`shared` flag gated on group_names, and an `enabled` flag conjoining that gate
with LITELLM_CHAT_MODEL_SERVED. Both are now single-sourced.

`shared` becomes literal true. The flag means the provider is a shared instance
rather than one deployed per consumer, and a central gateway is always that;
the dynamic form only said "shared if deployed", which `enabled` already
decides. The 50 shared pins under `litellm:` leave meta/variants.yml with it.
The tree had already noticed: 20 of those 50 lines carried
`# nocheck: variants-static-override`, two of them with copy-pasted text about
container_backup. This narrows e5de4e085a for one service key rather than
reversing it; every other dynamic flag keeps the group_names form.

`enabled` becomes `{{ LITELLM_USABLE }}`, defined in group_vars/all/16_ai.yml.

That shorthand is only possible because is_explicit_truth learns about it.
The resolver reads a dependency out of the flag TEXT — "in group_names" in
value — so a constant hides the role id and the provider silently leaves the
static closure of every role referencing it. Measured: the old expression
resolves True, `{{ LITELLM_USABLE }}` resolves False. GROUP_GATED_CONSTANTS
names the constants whose own definition carries the gate, and the docstring
states the obligation: a SPOT of this kind that is not listed there drops its
provider without a word.

The wider fix, accepting any Jinja as dynamic, was measured and rejected: 35
other flags carry a Jinja value without the marker (MCP sidecars, javascript
and SSO gates, recaptcha), and they would all enter closures they are not in
today.

A literal shared flag is no longer dynamic, so test_dynamic_flags wants the
`# nocheck: dynamic-flag` its own message documents for legitimate exceptions;
where a marker already existed the rule list is extended rather than a second
comment added.

Behaviour is unchanged where it counts. The closure keeps litellm with the same
app counts (wordpress 26, openwebui 23, hermes 23, mediawiki 22, flowise 23),
unrelated Jinja still resolves False, and the truth table of the new enabled
form matches the old one in all four served/deployed combinations. Three roles
resolve without litellm, and `git show HEAD:` proves that predates this change:
zammad pins enabled false in variant 0 at line 39, nextcloud at line 97.

Six unit tests cover the resolver, including one asserting that every
registered constant is actually defined in group_vars, so an entry cannot
become a dead promise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:23:59 +02:00
kevinveenbirkenbachandClaude Opus 5 4ac07edaea refactor(meta): move the info-media exemption to the file it speaks about
The marker names the reason a role has no upstream homepage or demo video, and
it sat in meta/main.yml because a comment-only info.yml parses to an empty
document, which the schema lint rejected outright. So the exemption lived one
file away from the two fields it explains, and a reader looking for the video
found neither the field nor the reason it was missing.

The schema lint now accepts an empty document exactly when it carries the
marker, and that is what allows the move. The media lint reads the marker from
info.yml rather than main.yml, and its failure message names that same file, so
the fix it suggests lands where the absent field would have gone.

A hundred and two roles follow. Every occurrence was a standalone comment on
the second line of main.yml, directly under the document start, so the move is
a line delete and an insert at the same offset. Seventy-three roles already had
an info.yml and gain the line; twenty-nine get one whose whole content is the
document start and the marker. All hundred and two reasons carry over verbatim,
including the thirty-five bespoke ones.

Nothing downstream sees a change. Of the four readers of meta/info.yml, three
coerce a null parse through `or {}` behind an isinstance guard, and
utils/cache/applications.py skips a falsy parse before assignment, so
applications.<role>.info stays absent for the twenty-nine exactly as it was
while the file did not exist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 22:26:38 +02:00
kevinveenbirkenbachandClaude Opus 5 96408f23a0 perf(healthcheck): let the start window outlast a real boot
start_interval is the cadence while start_period runs, and only while it runs.
Once the window closes the probe falls back to interval, so a service that
becomes ready just after it pays a whole tick anyway and the cadence never
reached it. Six of the seven flavors were shaped that way, Curl worst: a thirty
second window against a one minute interval.

Measured across 151 converge waits in four job logs, 63 sit exactly on a tick
boundary, 36 in the thirty second step and 27 in the sixty second one.
Attributed by application, web-app-seaweedfs is the largest at 1501s over
sixteen waits with a median of 57.7s: it probes with curl and fell straight into
the gap. web-app-zammad probes with connect, whose window is fifteen minutes,
and its 154s median is real boot time with nothing to recover.

So the window moves to ten minutes and the steady state interval to two, and the
flavor overrides that sat below them go: Curl, Wget and Connect lose interval
1m, Http its twenty second window, Nc its ten. Connect keeps fifteen minutes.

Neither number costs anything while a container works. start_period delays
nothing, it only stops failures counting, and interval no longer touches
convergence at all since that hangs on the window alone; it is steady state
load, which drops from 120 probes an hour per service to 30.

Forty six role files follow. Twenty seven declarations go that repeat the new
default or undercut it, an undercut window being the defect itself. Eight
windows longer than ten minutes keep their value and now say why. The literal
blocks and the three Dockerfiles this repository builds spell every timing out,
because outside the lookup an omission takes docker's value and not the
platform's.

A fourth lint rule keeps the shape: start_period must outlast one interval,
checked against the effective values, so raising interval without the window
cannot quietly bring the cliff back.

Two faults in the lint surfaced while building it and are fixed here. It
compared timings as strings, so 600s and 10m read as different values, and it
matched the healthcheck key exactly, so a line carrying a trailing nocheck was
skipped whole, which is how the two swarm-init partials went unexamined.

One regression, bounded. A container that starts and never turns healthy is
neither rejected nor errored, so stack_ready.sh only calls it fatal after three
killed tasks, and each cycle now spans the longer window. That reaches twenty
minutes rather than six, capped there by swarm_stack_converge_retries at 600
with a two second delay, and it lands only on runs that were failing anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 21:54:26 +02:00
kevinveenbirkenbachandClaude Opus 5 d0a8608bfa perf(healthcheck): give every probe the start cadence 42ebd9ea71 added
42ebd9ea71 gave the healthcheck SPOT start_interval, the cadence docker uses
while start_period is still running. A block a template writes out itself
bypasses the SPOT and never gained it. Measured across the repository: 46 such
blocks, 40 of them already carrying start_period, not one carrying
start_interval.

Without it a probe fires only every interval, so a service ready after two
seconds is reported healthy at the next tick and a stack converges no faster
than its coarsest quantisation. That lands in sys-svc-compose's swarm converge
wait, 1551.7s per CI job over 38 stack deploys.

Twenty five of the blocks move into meta/services.yml and render through
container_healthcheck. Each keeps an explicit test rather than picking a flavor:
a flavor would replace the command instead of carrying it across.

The move is lossless by construction. probes.py block() writes
overrides.get(key, default) for interval, timeout, retries, start_period and
start_interval, so a declaration repeating the literal values renders the same
block plus the missing cadence. Checked per block against the old values: the
only difference is start_interval 5s, nothing changed, nothing lost.

Twenty one stay literal. Seventeen interpolate a template variable, whose value
resolves in a scope meta/services.yml does not share, so moving them would be a
rewrite rather than a transcription. Four interpolate nothing and still cannot
move: svc-db-memcached and svc-db-redis probe every second in their swarm
service partial, where a 5s start cadence would report healthy later than the
block already does, and the sys-svc-rdbms and keycloak sso_proxy partials render
for the consuming role's application_id, so their declaration has no single
meta/services.yml to live in. Each carries a nocheck naming its reason.

All twenty one gain start_interval. Three also gain start_period, without which
the key does nothing at all, since it only applies while the start period runs.
That widens their failure tolerance rather than narrowing it: bluesky from 90 to
120 seconds, magento from 50 to 80, the mariadb partial from 180 to 210.

pixelfed spelled its probe as a bare string and now spells it CMD-SHELL. Docker
reads both as a shell command; only the YAML shape changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 20:31:07 +02:00
kevinveenbirkenbachandClaude Opus 5 e0048f157b fix(litellm): gate consumers on a served model, not on the gateway being deployed
The gateway publishes only models it has a backend for and renders model_list: [] when neither ollama nor lmstudio is deployed and no OpenRouter key is set. LITELLM_CHAT_MODEL fell back to openrouter/auto without checking that key, so consumers were configured for a model the gateway does not carry and every prompt failed with a 400.

Observed in the Mattermost#1 CI job: litellm.log carries 12 occurrences of "Invalid model name passed in model=openrouter/auto" next to 400 Bad Request, and tests/test-litellm-agent.js failed on every attempt while the rest of the suite passed.

Make the fallback conditional on the key so the name is empty when nothing is served, promote LITELLM_CHAT_MODEL_SERVED from the role vars to group_vars where consumers can see it (it was the third copy of this logic and the only correct one), and switch the 14 consumers from "litellm is deployed" to "litellm serves a model". The Playwright service flag reads the same lookup, so the spec skips together with the Ansible gate. The bool filter carries weight: the folded scalar yields the string False, which Jinja treats as truthy.

A new unit test renders both expressions against Jinja and pins the invariant that a model name exists exactly when a model is served, including the case where ollama preloads embedding models only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 19:48:32 +02:00
kevinveenbirkenbachandClaude Opus 5 9fb8f63286 fix(test-e2e-cli): exec on the node that holds the container
hermes_hermes.1 ran on a worker while run_one.yml ran its script on the loop
host. resolve_container_id.sh finds nothing node-locally on the manager, falls
through to the cluster-wide branch and returns the task's ContainerID, which
only the owning node's daemon knows, so `container exec` answered
`No such container: 65801ca259a1` twenty times over 116 seconds. Run
33785198875, job 100756596935.

Resolve the owning node and run there. meta/tests.yml gains
container_service_key for the two roles that exec into their own container; a
role without the key keeps the loop host, and role_tests returns the default
for a missing path, so absence is a polarity rather than an error.

The reset at run_one.yml:27-30 is unconditional and precedes every
delegate_to. Without it the first loop iteration dies with
'CLI_HOST_NODE' is undefined, because _cli_apps starts with a role that has no
key; the same task closes the fact leak that include_tasks leaves open between
iterations. The resolve include is gated on an explicit length check rather
than truthiness, or a keyless role would hang in resolve_host_cid's converge
wait, and it caps that wait at 30 x 2s instead of inheriting 600 x 2s: the
deploy is long finished by the time the CLI stage runs, so failing fast beats
waiting twenty minutes.

Staging, render and run all delegate. Delegating only the run would turn
`No such container` into `No such file or directory`, since the assets would
stay on the loop host. Both templates now carry CLI_LOCAL_CID: with the exec
on the owning node the id is node-local, so container_address's deferred
`$( ... )` is not needed, and it never survived `set -a && . test.env` anyway,
which evaluates it once at source time.

The stage directory is removed on the delegate too. tasks/main.yml clears it
without delegation, i.e. only on the loop host, and the task's own name calls
its contents decodable DB secrets.

Measured locally rather than argued: self-delegation leaves file, copy with
content, copy with src and shell byte-identical, including mode; the loop
reset gives keyless roles an empty container and keeps CLI_HOST_NODE on the
loop host across five iterations; and container_service_key resolves to
hermes, openclaw and empty strings through the same walk cli.timeout already
uses in production.

Not fixed, and named so it is not forgotten: web-app-joomla, web-app-yourls
and web-app-penpot exec from an IS_STACK_HOST gate without resolve_host_cid,
so a task of theirs landing on a worker hits the same error from the same
source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 02:55:08 +02:00
kevinveenbirkenbachandClaude Opus 5 a432535847 fix(openclaw): a failed exec is not a passing probe
The CLI test discarded the exec status with `|| true` and judged the run on
its output alone: non-empty, and not carrying a `failed to start server`
line. `Error response from daemon: No such container: <id>` satisfies both,
so the script echoed the daemon's error, printed ALL CHECKS PASSED and
exited 0. CI reported green for a container it never reached.

That path was live in run 33785198875 job 100756596935, where hermes_hermes.1
sat on a worker while roles/test-e2e-cli ran the exec on the manager. The
harness half of that failure is a separate change; this one is about the
predicate, which would have reported success for any unreachable container in
any mode.

Capture the status and require it. The output guard stays as it is: no
positive success marker is introduced, because nothing in the repository
records what `mcp probe` prints when it succeeds, and a guessed marker can
break a working test. web-app-hermes keys on `Tools discovered:` for the same
reason the other way round, but that string is its own.

The condition gains a conjunct, so no case that was checked before is
unchecked now. The residual runs the other way: should the probe ever exit
non-zero while still producing usable output, a run that passed will now
fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 02:14:50 +02:00
kevinveenbirkenbachandClaude Opus 5 346ad875c6 refactor(mcp): drop the two provenance fields nothing reads
supported_version and minimum_version sat in 24 roles and were read by no template, no task and no lookup. Their only consumer was the audit report version column, which turned false the moment an automated image bump ran: qdrant reported a verified version nobody had verified, two releases long. Both leave MCP_KEYS, so a role that declares them again fails the schema lint instead of carrying a field with no effect.

Requirement 025 mandated minimum_version as a MUST and 035 carried it in an example; both are corrected here, otherwise the repository would contradict its own code.

The gate then surfaced five dead docs.mattermost.com links. Mattermost moved to /administration-guide/ and /deployment-guide/, and every replacement was probed for 200 rather than guessed. The MCP page is gone with no successor, so mattermost now points at mcpserver/README.md in the Agents repository, the source rather than a rendered mirror.

Both concerns share three files - mattermost's meta/mcp.yml, requirement 025, and the generated audit.md, which derives from a source of each - so splitting them would commit an artifact that does not match its sources.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 19:31:08 +02:00
kevinveenbirkenbachandClaude Opus 5 a458ef2965 fix(mcp): reject a provider with no callable tool
Hermes and OpenClaw wrapped the tool allowlist in a guard that dropped the key when the callable set was empty. Both upstreams read an absent filter as "register every tool the server serves", so a provider resolving to no tools got the widest surface instead of none. Nextcloud keeps its contract in tools.categories with no allowlist and admits both clients, so it was the live case.

The templates now render the filter unconditionally, the way the n8n client node already pins include to selected; the two tests that asserted the omission are inverted. mcp_servers additionally rejects such a provider with the fatal code tools_unenforceable: a client filters by tool name, so a runtime-composed surface reaches it as no surface at all, and a server the agent can never call is worse wired in than refused with a reason.

A deployment pairing Nextcloud with either agent now aborts naming the provider, where it previously handed the agent every Context Agent tool unfiltered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 16:19:08 +02:00
kevinveenbirkenbachandClaude Opus 5 4986e80dd4 feat(readme): ask the reader for a domain the deployment can resolve
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.

Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.

The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.

A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:30:25 +02:00
kevinveenbirkenbachandClaude Opus 5 4164f37981 fix(mcp): give every MCP round an identity provider to sign in with
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".

The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.

Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.

The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.

nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 15:51:24 +02:00
kevinveenbirkenbachandClaude Opus 5 0b13fb73d0 style: apply make autoformat
Two leftovers of my own. The sweep that removed the reverse client entries
appended a newline where it cut a block, which at end of file leaves a blank
line after the last key in four services.yml. And the two cache-read nocheck
comments in the bond editor were written past the line limit, so the
formatter breaks their read_text calls across three lines and carries the
comment to the closing paren.

The wrap does not cost the suppression: test_no_uncached_file_reads matches
the statement, not the line the directive sits on, and passes on this form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:24:47 +02:00
kevinveenbirkenbachandClaude Opus 5 92f97a53ad docs(readme): regenerate after the reverse client edges came out
The dependency diagrams are drawn from the service edges, so dropping the
client-side homeassistant and baserow declarations changes both ends of each
relation. The four client roles lose the provider node, its service node and
the edge between them; homeassistant loses its whole Dependents subgraph,
because nothing pulls it in from that side any more.

The provider-side edges that carry the MCP pairing are untouched, which is
the point: the relation is now drawn once, from the provider, instead of
from both ends.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:23:49 +02:00
kevinveenbirkenbachandClaude Opus 5 1e6e343268 refactor(mcp): declare a client-provider pairing once, on the provider
Four MCP clients each carried a services entry for homeassistant, and
openwebui and flowise one for baserow, with matching run_after lines. Those
predate the admission refactor: since then the co-deploy edge lives on the
provider, where mcp-consumer-services-entry enforces it, and homeassistant
and baserow already carry an entry for every client they admit. The client
copies stated the same relation a second time.

The asymmetry gives them away. Open WebUI consumes twenty MCP servers and
declared three of them; the other seventeen work because the bridge is built
from MCP_DISCOVERED_SERVERS, never from the client's own services map.

Drop the run_after lines in the same move: a run_after target must be backed
by a services entry, so one cannot go without the other. Hermes and OpenClaw
lose the whole key, homeassistant being its only entry, because a bare
run_after parses as null rather than an empty list.

Keep qdrant on flowise, which renders QDRANT_URL and QDRANT_API_KEY into the
container and is a real dependency, and keep prometheus everywhere as the
ordinary metrics edge.

Open WebUI stops dragging both providers into every round it joins, which
brings moodle variant 2 from 64.75 GB back to 62.75 GB, under the budget the
preceding commit pushed it over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:03:22 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 ffdef28da3 refactor(mcp)!: derive a surface's presence from its peers
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.

The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.

The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.

Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.

Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 16:13:01 +02:00
kevinveenbirkenbachandClaude Opus 5 83c1c729de refactor(mcp)!: derive admission from the client
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.

The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.

BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 18:33:22 +02:00
kevinveenbirkenbachandClaude Opus 5 c9298033e6 fix(roles): define the entity_name these five roles read
Role vars do not cross role boundaries, so flowise, hermes, homeassistant,
openclaw and openwebui read nothing where they template entity_name into
resolve_service_key. Seventy-seven other roles set it on the line below
application_id.

Ansible does not stop for it. In compose the undefined value renders into task
names as "<< error 1 - 'entity_name' is undefined >>" and the play carries on,
because that path resolves through resolve_compose_cid and never reads the key.
Swarm templates it into a real argument and dies on the first task, and with
any_errors_fatal that ends the play. Reproduced both ways against a replica of
resolve_host_cid.yml: compose ok=4 failed=0, swarm "'entity_name' is undefined".

Nothing could have supplied it either - role vars reach neither a sibling
include nor the parent, so the value was always undefined rather than sometimes
borrowed from whichever role ran before.

The rule that comes with it compares consumption against definition, which no
existing check does; the four scanners under tests/lint/ansible/variables all
run the other way. It skips a file another role pulls in by absolute path,
since that file executes in the including role's scope and reads that role's
value, and it matches the variable rather than get_entity_name, the filter whose
name ends in it. Both exemptions cost a false finding each while I was writing
it. It caught two roles the review had missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 00:37:24 +02:00
kevinveenbirkenbachandClaude Opus 5 f70717ad45 fix(openclaw): name the model the way the gateway validates it
The provider block ec736bd4d7 added carries only an id per model. OpenClaw
validates the whole file before serving anything and rejected it:

  Gateway failed to start: Invalid config at /home/node/.openclaw/openclaw.json:
  models.providers.litellm.models.0.name: Invalid input

A rejected config is not a degraded start. The gateway retries, trips its own
breaker after four unclean boots, and the container ends at exit 128. The
deploy notices only at the very end, when the container health check gives up
after 25 attempts - twenty-five minutes and a role away from the cause.

The model itself was fine: the rendered file on the host held
[{"id": "qwen2.5:0.5b"}], so this is the missing key and not the
LITELLM_CHAT_MODEL fallback. name now sits beside id with the same value.
Which of the two reaches the API is the upstream's business; identical values
cannot send a different model than the one agents.defaults names.

It shipped because nothing rendered it. All three existing cases in
test_mcp_tool_filter.py pass OPENCLAW_LITELLM_ENABLED=False, so the block was
skipped and its shape was never compared to anything. The new test renders it
with LiteLLM on, and against the old template it fails on KeyError: 'name'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 23:06:41 +02:00
kevinveenbirkenbachandClaude Opus 5 9166e98405 feat(openclaw): wire the agent's models through the platform gateway
The config template already rendered a models.providers block behind an enable flag, but the five variables that block reads did not exist, so the first deploy would have failed on an undefined variable. They follow the same shape Flowise, Hermes and Open WebUI already carry: the flag from the service declaration, the base URL from the gateway's local address, the key from the role's own credential.

This also closes a window the earlier gateway-contract commit opened, since that one added an include gated on OPENCLAW_LITELLM_ENABLED while the variable itself was still uncommitted. The resolved model-wiring entry drops out of the role's TODO.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:53:36 +02:00
kevinveenbirkenbachandClaude Opus 5 ec736bd4d7 feat(mcp): offer no client a tool its provider calls mutating
Both agents registered every tool every provider served. Hermes documents that as its own default in tools/mcp_tool.py, "Neither set -> register all tools", and the role never set tools.include; OpenClaw's toolFilter was never rendered at all. A provider that grew a mutating tool would have reached both without anything declaring it.

A serving role now names its mutating tools, the discovery snapshot carries that set beside the allowlist, and every client renders the allowlist minus it in whatever form its upstream enforces. The subtraction sits at the client and not in discovery because Flowise compares the declared set against what its registry actually discovered and needs the full served list. Where a provider permits mutations there is nothing to withhold, so the set renders empty.

This is a client-side boundary, not a server-side one: whoever holds the endpoint key can still call a write tool directly. Baserow is the case that made it necessary, since its MCPEndpoint model carries no scope and its owner is a workspace administrator, so the declared mutating_tools_enabled bounds nothing there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:29:12 +02:00
kevinveenbirkenbachandClaude Opus 5 ead858f481 feat(agents): let each sandboxed agent act as its own platform account
The platform reserves a user key per role, so hermes and openclaw existed as non-privileged accounts before either role claimed one. Claiming the key is what turns a reservation into an identity the agent may authenticate as, and gives the account its description and its mailbox. Neither role declares a password: provisioning stores one for every declared account, and a role whose registration rejects the shell-safe alphabet says so in its own meta/users.yml rather than carrying a second credential beside the account it already owns.

The lint derives the agent set from the sandbox-runtime scan rather than a list and holds three properties: the role claims its own reserved key, the claimed account carries no role because an agent employee is never an administrator, and no second credential sits beside it. Pointing an agent at an account that already exists needs no code, since the inventory users variable overrides the same key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:23:20 +02:00
kevinveenbirkenbachandClaude Opus 5 656eabf490 feat(svc-ai-litellm): bind every gateway consumer to the contract it declares
Four roles declared the litellm service and never included the gateway's consumer contract, which is the single place that refuses an empty virtual key and an AI base URL pointing outside the container network. They rendered their AI configuration regardless, so a missing key or a public endpoint reached a green deploy and every prompt left the deployment unauthenticated. Flowise, Hermes, OpenClaw and Open WebUI now include it, each gated on its own enable flag.

A lint derives the consumer set from the repository rather than a list: a litellm block without modes is a consumer, while the gateway and its admin UI declare the service they run and are exempt from the contract they own. The existing rule against addressing a model provider directly scanned only templates, so a role resolving the same variable in vars/ was invisible to it; it now scans both, and a negative control keeps the pattern from going quietly stale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:09:56 +02:00
kevinveenbirkenbachandClaude Opus 5 a2bbce6b55 docs(personas): record why openclaw blocks in-app logout and biber
Two persona-block-rationale directives, in the form aef525f91 used for
hermes, homeassistant and litellm.

The Control UI is a WebSocket SPA whose session sits behind gateway device
pairing and exposes no DOM logout control for the shared inAppLogout helper,
so files/playwright/test-oidc-login.js covers the SSO round trip instead.
biber is refused at the oauth2 proxy, which admits only the administrator
and mcp RBAC groups.

Both checkable claims verified: allowed_groups lists exactly those two
groups, and the substitute spec exists. Coverage moves rather than
disappears.

The missing logout control itself is not verified; openclaw was not deployed
this session, and it is a product claim that a UI update could invalidate
without anyone touching the directive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:16:28 +02:00
kevinveenbirkenbachandClaude Opus 5 a6471b6df4 refactor(openclaw): read the container paths from the volume declarations
env.j2 hardcoded four container paths that meta/volumes.yml already declares.
Read the volume targets instead, so a changed mount cannot leave the
environment pointing at a path that no longer exists.

Value-identical, checked against the declaration rather than assumed:
openclaw_data.target is /home/node/.openclaw and openclaw_config.target is
/home/node/.openclaw/openclaw.json, and path_join reproduces the workspace
path exactly.

Same pattern as a772115b8 for litellm, but verified by value here rather
than by the service having worked.

openclaw was not deployed this session, so this is a static check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:15:37 +02:00
kevinveenbirkenbachandClaude Opus 5 30f24bb346 docs(csp): record why openclaw allows unsafe-eval and unsafe-inline
Both flags gain the rationale the lint requires, in the same form as
896b72428. Neither is a justification: both state that the flag was set when
the role first deployed with no console error recorded against it, and carry
a TODO to re-verify against the running Control UI.

unsafe-eval is the sharper of the two, permitting eval and its relatives. A
WebSocket SPA plausibly needs it; that it does has not been established.

Three roles now carry this unanswered TODO: hermes, litellm and openclaw.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:14:37 +02:00
kevinveenbirkenbachandClaude Opus 5 f446e006d0 chore(playwright): exempt four more suites from the files layout rule
The role-files-layout lint wants scripts under files/ sorted into
per-language folders, which dfe7a7fdd did for 24 of them. Playwright suites
are exempt: the runner copies the tree as it stands, discovery works by path
and the specs import their helpers relatively, so sorting by extension would
break both.

83 roles already carry the marker; hermes, homeassistant, litellm and
openclaw were missing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:03:48 +02:00
kevinveenbirkenbachandClaude Opus 5 9e6f64dde2 docs(readme): regenerate after the tor onboarding
The READMEs are generated from the role declarations. Six role READMEs gain
svc-net-tor in their dependency graph and the top-level table gains the Tor
Onion Service column entries, following cd1f07a91. The remaining lines are
template output: the author name becomes a link, two mermaid linkStyle rows,
and a reworded setup sentence in web-svc-mirror.

Left unregenerated the documentation would disagree with the meta/services.yml
it is derived from.

The generator was not re-run here: this is the output that was already in the
tree, checked to match the committed declarations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:01:29 +02:00
kevinveenbirkenbachandClaude Opus 5 27dcd975e5 test(e2e): scale the spec timeouts for onion transport
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.

Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.

The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.

Merge work rather than authored here; not deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:52:19 +02:00
kevinveenbirkenbachandClaude Opus 5 a3ea18c700 refactor(compose): finish the context-wired volumes lookups
acc9e25c0f moved the volume lookups to context-wired signatures, taking
application_id from the task context instead of an argument:

  lookup('container_volumes', application_id, service_name) -> (service_name)
  lookup('compose_volumes', application_id)                 -> ()

Six templates were left on the old form: the three AI services and hermes,
homeassistant and openclaw. One of them also picks up the matching
container_healthcheck lookup.

This is the same failure mode as the php paths: a refactor that leaves
callers behind, and the break only appears when the template renders, not in
the gate. A deploy in this session died exactly that way, on a template still
passing application_id to a lookup that no longer takes terms.

No file in the repository carries the old signature after this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:49:41 +02:00
kevinveenbirkenbachandClaude Opus 5 cd1f07a91e feat(roles): onboard hermes, homeassistant, litellm and openclaw to tor
Four roles gained a canonical domain without a tor service block. Declare it
the way 84b22b37a8 did for n8n: bond 0.5, enabled on 'svc-net-tor' in
group_names, shared, with the variant pins alongside.

The nocheck directives carry the two exemptions: the service has no persona
surface for the playwright flag, and the sso-variant-tor combination requires
the onion in every variant, so the false polarity has no variant left to
live in.

hermes also moves its local http port from 8089 to 8092, because 8089
belongs to web-app-socialhome. That is a separate concern riding along in the
same file; splitting it would need interactive staging.

This is merge work rather than authored here: it follows the established
pattern and the port collision is real, but none of it has been deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:48:10 +02:00
kevinveenbirkenbachandClaude Opus 5 481666bdb0 refactor(roles): finish renaming meta/schema.yml to meta/secrets.yml
Eleven roles still declared their credentials in meta/schema.yml while the
rest of the repository had moved to meta/secrets.yml. The name is not
cosmetic: the inventory resolves credentials at that path, as
d86913225b and c541e7c22f record, so a role left on the old name drops out
of that resolution.

All eleven are pure renames with no content change.

Also correct the four remaining places that still named the old path in
prose: the credentials reset CLI help, two test docstrings and the espocrm
README. None of them resolved a file, but they misdescribe where credentials
live, which is exactly what sent an earlier reader to the wrong location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:13:48 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 575b9a463c fix(web-app-openclaw): keep one writer on the agent's shared state
The service declared no replica count, so it took the topology default of
three and spread them over two nodes while its state directory is one NFS
export shared by all of them. The agent keeps that state in SQLite, and the
swarm run shows what concurrent writers do to it: one replica exits with the
database reported as not a database, another cannot even create the state
directory.

Those exits are what started the restart storm the run then failed on, and
they are runtime independent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:30:31 +02:00
kevinveenbirkenbach 6923c5f361 fix(openclaw): allow eval in script-src so the CSP health check can pass
The shipped frontend bundle calls eval() from its runtime config chunk,
which script-src blocks, so the CSP health unit failed twenty-five times
in a row and aborted the deploy. The application itself answers 200 and
loads every asset.

This widens the policy rather than fixing the cause, which sits in the
upstream bundle; four other roles carry the same allowance for the same
reason. The failure was masked while the adapter build failed earlier in
the same job.
2026-08-10 13:31:48 +02:00
kevinveenbirkenbach b57a728320 feat(openclaw): admit the mcp group at the SSO gate
The agent drives MCP clients, so holders of its application-scoped mcp
role must reach it, which completes the set of three client roles whose
gate now admits that group. The README follows the renamed assert task.
2026-08-10 00:07:58 +02:00
kevinveenbirkenbach 77896fba12 refactor(mcp): expose every role's MCP work at tasks/utils/mcp.yml
The reconciler addresses a client with tasks_from: utils/mcp.yml, which
only works when every role offers its MCP work at the same path. The
files were named per role, and on the two agents the work sat inline in
the entry point, where nothing but a full role run could reach it.

The filter docstring pointed at the pre-rename path and now names the
file that exists.
2026-08-09 23:57:43 +02:00
kevinveenbirkenbach 67feed6332 feat(rbac): declare the mcp role on every agent that carries clients
An mcp grant lets its holder drive clients that reach admitted providers
with the credentials the deployment issued, not the caller's own, so the
role is declared per application and never spans them.

The unit test pins that boundary directly: a scoped grant reaches only
the named application, an unscoped one reaches nothing, and an ordinary
role still spans everything. The Keycloak realm builder, the LDAP entry
builder and the Open WebUI reconciler all resolve membership through the
one module this covers.
2026-08-09 23:53:41 +02:00
kevinveenbirkenbachandClaude Opus 5 a423bde6cc refactor(mcp): unify client entrypoints and read provider tokens through one lookup
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.

Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.

The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.

The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.

The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:06:31 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbachandClaude Opus 5 46021802b3 docs(mcp): state how hermes and openclaw prove their configured servers
Neither client exposes an administrator-visible server list the way Open WebUI
does, so their configured set is proven at deploy time instead: hermes runs
'hermes mcp test <server>' per server, openclaw probes the agent's mcp surface
in tasks/01_mcp_assert.yml. OpenClaw's assert already existed but was documented
nowhere, so reading the role suggested it had no verification at all.

Both sections also name what the Playwright spec covers, which is the
complementary case: the instance holding the MCP credentials refuses an
unauthenticated caller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:13:04 +02:00
kevinveenbirkenbachandClaude Opus 5 e9af031215 feat(hermes,openclaw): consume the discovered MCP servers
Render every discovered server into the agent's own config, filtered to
the auth schemes a client can actually present, so no entry carries a
credential the server would reject.

Render hermes' config after the stack exists and before the handlers
flush; the other order fired compose up while the config was still
missing.

Own the config file as the container user. Rendered as root it stays
unreadable to the agent, which then falls back to its built-in defaults
silently and reports only that the server is unknown, pointing the reader
at the wrong layer.

Assert what these clients expose: neither ships an admin surface listing
its configured servers, so the specs hold the API to refusing an
unauthenticated caller, which is what keeps the per-server bearers out of
reach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:26:32 +02:00
kevinveenbirkenbach f39073c95c Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/ai-agent-employees
# Conflicts:
#	roles/web-app-flowise/meta/services.yml
2026-07-30 15:39:29 +02:00
kevinveenbirkenbachandClaude Opus 5 e3481b7745 feat(openclaw): reach the declared MCP servers and prove it
MCP_DISCOVERED_SERVERS resolves every server-direction role to its internal URL, the bearer the server role minted, and its transport, dropping any server without a token so no client renders a connection it cannot authenticate.

OpenClaw moves from 2026.3.11 to 2026.7.1 because the pinned version has no configuration surface for outbound MCP servers at all: its validator rejects mcpServers, the shipped bundle carries no bundle-mcp, and the ACP path refuses per-session servers while pointing at a gateway config that does not exist there. 2026.7.1 reads mcp.servers, which the template now renders. The gateway command gains its run subcommand because the bare gateway no longer starts anything.

Reachability was the missing half: openclaw sat on its own networks only, so the hub name resolved through the DNS wildcard to the ingress instead of the container. The hub overlay now declares an mcp consumer key, which attaches every role carrying services.mcp.

A probe task runs openclaw mcp probe after the handlers flush and fails the deploy when a configured server does not answer. Without it the previous run reported failed=0 while the client could not reach anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 15:40:19 +02:00
kevinveenbirkenbachandClaude Opus 5 e3aba6bf9b feat(svc-virt-kata): enforce the isolation the agent tier promises
The sandbox tier carried no check of its own. Three agent replicas once ran on
the shared kernel while every test stayed green, because nothing between the
runtime selection and the running container ever asked what the container
actually got. Each guard below sits where the promise first breaks.

A node is declared a sandbox target by a label that routes every sandboxed
service to it. It now has to prove it can carry them first: the node reads its
own default runtime and refuses the label while that is still runc. Doing it
before the label rather than after is the point, since the label is what makes
the promise.

Compose has no node-wide default to inspect, so the agent roles read the
runtime the container actually received once the stack is up. The check is
skipped in embodied mode, where the runtime is dropped on purpose because a
microVM between the agent and the hardware it drives defeats the point, and it
keys on the same services.kata switch that already drives the placement
constraint and the compose pin.

A runtime is only as good as the template around it. The new lint refuses a
privileged container, a host network, pid, ipc or user namespace, and a host
docker socket bind in any role that declares a kata service. It fails when it
finds no sandboxed role at all, so renaming the service cannot silently retire
the guard.

It caught its own subject on the first run: both agent templates emitted
privileged: true next to the device allowlist, which grants every device and
made the allowlist decorative. The requirement is explicit that no device
outside the allowlist reaches the agent, so the flag is gone and the role
README no longer claims a limit that was not enforced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 13:11:28 +02:00
kevinveenbirkenbachandClaude Opus 5 9d3eb4a79c fix(sys-svc-container): register the runtimes the host actually has
`svc-virt-kata` is pulled in as a shared service of the agent roles, so it also
executes on hosts that are in no inventory group of that name. Gating the
runtime registration on `'svc-virt-kata' in group_names` therefore left such a
host with runsc installed and its node labelled kata-capable, while
`daemon.json` registered nothing and the daemon kept running everything under
runc: the exact isolation breach the label promises is closed. The stat probes
in `daemon_config.yml` decide instead, and they cannot be fooled by an
inventory that says otherwise.

Nothing else in the tree installs or references runsc, so a shim on disk is
proof that this role put it there.

The agent compose templates pin their runtime on the same `services.kata`
switch that already drives the swarm placement constraint, instead of pinning
unconditionally against a daemon config written under a different condition.
Both roles declare that key themselves, so the lookup is the strict form: a
deleted flag now aborts the render rather than silently starting the agent
without a sandbox.

The new unit test covers what nothing covered before, which is how the runc
breach went unnoticed: an absent shim is never registered, a default runtime is
always a registered one, the stack host keeps the shared kernel, compose never
sets a daemon-wide default, and registration follows the probe rather than the
inventory group.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:06:41 +02:00
kevinveenbirkenbachandClaude Opus 5 fc16056c8b fix(svc-virt-kata): apply the sandbox runtime on swarm workers
`docker stack deploy` drops the per-service `runtime:` key, so a swarm worker
can only carry the sandbox as its daemon-wide `default-runtime`. That was never
set: the role reached the manager only, no node was labelled, and one of three
agent replicas executed model-generated code under plain runc.

Setting it from the base render is not possible either. The container role
writes `daemon.json` before the runtime binary exists, and dockerd refuses to
start when a runtime path does not resolve ("stat /usr/local/bin/runsc: no such
file or directory"). The role's own attempt to re-render afterwards was a silent
no-op, because `include_role: sys-svc-container` hits the
`run_once_sys_svc_container` guard.

Move the probes plus the render into `daemon_config.yml` and let the role
re-enter it via `tasks_from` after installing runsc. Only runtimes whose binary
the probe found are registered, and `default-runtime` is written only for a
present shim on a non-stack host, so the stack host keeps the shared kernel for
the platform's own services.

The placement constraint compares against a fixed literal because Docker
constraints have no presence test: label the node `kata-capable=true` rather
than with the runtime name, otherwise a Kata node fails a `== runsc`
comparison. Without a worker outside the stack host no node is eligible at all,
so refuse the deploy instead of leaving the service pending forever.

`select_runtime.yml` is gone: hermes and openclaw included it by absolute path
out of a foreign role, and one probe in the container role now serves every
consumer.

Verified live on a three node cluster: worker `DefaultRuntime=runsc`, both
workers labelled, manager unlabelled, service constrained, and the running
replica reports `runtime=runsc`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 04:00:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 08a6dd157c feat(svc-ai-robot): embody an agent on a dedicated device
Requirement 034 inverts the isolation model of the other agent roles: the
boundary is the whole machine rather than a kernel boundary inside it.
Adding this role to a device switches the agent from isolated to
embodied.

The agent then runs privileged with exactly the devices from
ROBOT_DEVICE_ALLOWLIST bound into its container, and the kernel-isolating
runtime is dropped, because a microVM between the agent and the hardware
it is meant to drive defeats the purpose. The allowlist lives in
group_vars so the agent templates can read it; a role variable is not
visible across roles.

The admission checks run inside the agent role before it deploys rather
than in this role afterwards. The agent is declared shared, so the
service loader brings it up during the constructor stage while svc-ai
roles only run at the following step; a check placed here would sit
behind the deployment it exists to prevent.

The dedicated-device check now compares against something group_names
actually carries: any other web-app role on the host. The previous list
held Docker node labels, which are a different namespace from Ansible
inventory groups, so that intersection could never match and the failure
message always blamed a multi-node swarm.

Which agent embodies the device follows from the topology: both agent
flags take the dynamic in-group_names form, Hermes Agent wins when both
are present, and the matrix pins one per round so neither path rots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:16:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8b5312445b docs(roles): write the AI and runtime role documentation
The eight new roles shipped with the README template unfilled: every one
of them described itself as "[Name](https://example.com/) is an
application" with a single "Describe a capability" feature. They were
implemented, deployed and tested; their documentation was placeholder
text.

Each role now carries a Description, Overview and Features written
against its own services, tasks and templates, with TODO.md as the
binding list of what is not wired. That caught a false claim: OpenClaw
advertised routing through the shared gateway, but it reads model
providers from openclaw.json rather than the environment, so the gateway
only shapes its deploy topology. The claim is gone.

The overview table derives its Name column from each README H1, and all
eight had kept the humanised fallback. Worst of these, both LiteLLM roles
rendered as "Litellm" on adjacent rows, so a reference in the Integrated
with column could not say which one it meant; they are now the Gateway
and the Admin UI. The rest take their official spelling.

The galaxy descriptions all repeated the product name the Name column
already carries, one named a role id in prose and one used a hyphen as a
dash. svc-virt-kata also claimed to install Kata, which it does not: it
installs gVisor and registers an already-present Kata shim.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:21:25 +02:00