Commit Graph
15 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 a432535847 fix(openclaw): a failed exec is not a passing probe
The CLI test discarded the exec status with `|| true` and judged the run on
its output alone: non-empty, and not carrying a `failed to start server`
line. `Error response from daemon: No such container: <id>` satisfies both,
so the script echoed the daemon's error, printed ALL CHECKS PASSED and
exited 0. CI reported green for a container it never reached.

That path was live in run 33785198875 job 100756596935, where hermes_hermes.1
sat on a worker while roles/test-e2e-cli ran the exec on the manager. The
harness half of that failure is a separate change; this one is about the
predicate, which would have reported success for any unreachable container in
any mode.

Capture the status and require it. The output guard stays as it is: no
positive success marker is introduced, because nothing in the repository
records what `mcp probe` prints when it succeeds, and a guessed marker can
break a working test. web-app-hermes keys on `Tools discovered:` for the same
reason the other way round, but that string is its own.

The condition gains a conjunct, so no case that was checked before is
unchecked now. The residual runs the other way: should the probe ever exit
non-zero while still producing usable output, a run that passed will now
fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 02:14:50 +02:00
kevinveenbirkenbachandClaude Opus 5 4986e80dd4 feat(readme): ask the reader for a domain the deployment can resolve
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.

Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.

The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.

A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:30:25 +02:00
kevinveenbirkenbachandClaude Opus 5 92f97a53ad docs(readme): regenerate after the reverse client edges came out
The dependency diagrams are drawn from the service edges, so dropping the
client-side homeassistant and baserow declarations changes both ends of each
relation. The four client roles lose the provider node, its service node and
the edge between them; homeassistant loses its whole Dependents subgraph,
because nothing pulls it in from that side any more.

The provider-side edges that carry the MCP pairing are untouched, which is
the point: the relation is now drawn once, from the provider, instead of
from both ends.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:23:49 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 83c1c729de refactor(mcp)!: derive admission from the client
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.

The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.

BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 18:33:22 +02:00
kevinveenbirkenbachandClaude Opus 5 9e6f64dde2 docs(readme): regenerate after the tor onboarding
The READMEs are generated from the role declarations. Six role READMEs gain
svc-net-tor in their dependency graph and the top-level table gains the Tor
Onion Service column entries, following cd1f07a91. The remaining lines are
template output: the author name becomes a link, two mermaid linkStyle rows,
and a reworded setup sentence in web-svc-mirror.

Left unregenerated the documentation would disagree with the meta/services.yml
it is derived from.

The generator was not re-run here: this is the output that was already in the
tree, checked to match the committed declarations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:01:29 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbach b57a728320 feat(openclaw): admit the mcp group at the SSO gate
The agent drives MCP clients, so holders of its application-scoped mcp
role must reach it, which completes the set of three client roles whose
gate now admits that group. The README follows the renamed assert task.
2026-08-10 00:07:58 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbachandClaude Opus 5 46021802b3 docs(mcp): state how hermes and openclaw prove their configured servers
Neither client exposes an administrator-visible server list the way Open WebUI
does, so their configured set is proven at deploy time instead: hermes runs
'hermes mcp test <server>' per server, openclaw probes the agent's mcp surface
in tasks/01_mcp_assert.yml. OpenClaw's assert already existed but was documented
nowhere, so reading the role suggested it had no verification at all.

Both sections also name what the Playwright spec covers, which is the
complementary case: the instance holding the MCP credentials refuses an
unauthenticated caller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:13:04 +02:00
kevinveenbirkenbachandClaude Opus 5 e9af031215 feat(hermes,openclaw): consume the discovered MCP servers
Render every discovered server into the agent's own config, filtered to
the auth schemes a client can actually present, so no entry carries a
credential the server would reject.

Render hermes' config after the stack exists and before the handlers
flush; the other order fired compose up while the config was still
missing.

Own the config file as the container user. Rendered as root it stays
unreadable to the agent, which then falls back to its built-in defaults
silently and reports only that the server is unknown, pointing the reader
at the wrong layer.

Assert what these clients expose: neither ships an admin surface listing
its configured servers, so the specs hold the API to refusing an
unauthenticated caller, which is what keeps the per-server bearers out of
reach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:26:32 +02:00
kevinveenbirkenbachandClaude Opus 5 e3481b7745 feat(openclaw): reach the declared MCP servers and prove it
MCP_DISCOVERED_SERVERS resolves every server-direction role to its internal URL, the bearer the server role minted, and its transport, dropping any server without a token so no client renders a connection it cannot authenticate.

OpenClaw moves from 2026.3.11 to 2026.7.1 because the pinned version has no configuration surface for outbound MCP servers at all: its validator rejects mcpServers, the shipped bundle carries no bundle-mcp, and the ACP path refuses per-session servers while pointing at a gateway config that does not exist there. 2026.7.1 reads mcp.servers, which the template now renders. The gateway command gains its run subcommand because the bare gateway no longer starts anything.

Reachability was the missing half: openclaw sat on its own networks only, so the hub name resolved through the DNS wildcard to the ingress instead of the container. The hub overlay now declares an mcp consumer key, which attaches every role carrying services.mcp.

A probe task runs openclaw mcp probe after the handlers flush and fails the deploy when a configured server does not answer. Without it the previous run reported failed=0 while the client could not reach anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 15:40:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 08a6dd157c feat(svc-ai-robot): embody an agent on a dedicated device
Requirement 034 inverts the isolation model of the other agent roles: the
boundary is the whole machine rather than a kernel boundary inside it.
Adding this role to a device switches the agent from isolated to
embodied.

The agent then runs privileged with exactly the devices from
ROBOT_DEVICE_ALLOWLIST bound into its container, and the kernel-isolating
runtime is dropped, because a microVM between the agent and the hardware
it is meant to drive defeats the purpose. The allowlist lives in
group_vars so the agent templates can read it; a role variable is not
visible across roles.

The admission checks run inside the agent role before it deploys rather
than in this role afterwards. The agent is declared shared, so the
service loader brings it up during the constructor stage while svc-ai
roles only run at the following step; a check placed here would sit
behind the deployment it exists to prevent.

The dedicated-device check now compares against something group_names
actually carries: any other web-app role on the host. The previous list
held Docker node labels, which are a different namespace from Ansible
inventory groups, so that intersection could never match and the failure
message always blamed a multi-node swarm.

Which agent embodies the device follows from the topology: both agent
flags take the dynamic in-group_names form, Hermes Agent wins when both
are present, and the matrix pins one per round so neither path rots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:16:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8b5312445b docs(roles): write the AI and runtime role documentation
The eight new roles shipped with the README template unfilled: every one
of them described itself as "[Name](https://example.com/) is an
application" with a single "Describe a capability" feature. They were
implemented, deployed and tested; their documentation was placeholder
text.

Each role now carries a Description, Overview and Features written
against its own services, tasks and templates, with TODO.md as the
binding list of what is not wired. That caught a false claim: OpenClaw
advertised routing through the shared gateway, but it reads model
providers from openclaw.json rather than the environment, so the gateway
only shapes its deploy topology. The claim is gone.

The overview table derives its Name column from each README H1, and all
eight had kept the humanised fallback. Worst of these, both LiteLLM roles
rendered as "Litellm" on adjacent rows, so a reference in the Integrated
with column could not say which one it meant; they are now the Gateway
and the Admin UI. The rest take their official spelling.

The galaxy descriptions all repeated the product name the Name column
already carries, one named a role id in prose and one used a hyphen as a
dash. svc-virt-kata also claimed to install Kata, which it does not: it
installs gVisor and registers an already-present Kata shim.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:21:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bcd32463a2 feat(web-app-openclaw): deploy the OpenClaw agent behind an SSO proxy
The second agent of requirement 032, pinned to the same isolating runtime
as hermes. Unlike hermes it exposes a browser Control UI rather than a
bearer API, so it takes the oauth2 proxy flavour with its own sso port;
a proxy in front of a browser surface is correct where it would break
OpenAI-compatible clients.

Three deploy failures shaped this role. OpenClaw maps no environment
variables onto config keys, so it restart-looped on 'Missing config' and
then on 'non-loopback Control UI requires allowedOrigins'; both now come
from a rendered openclaw.json mounted as a config. That render must run
after sys-stk-full, because it notifies compose up and the handler would
otherwise fire before the compose file exists. And the process runs as
root: OpenClaw refuses a config file not owned by the running uid and
writes cron state into its home, neither of which a non-root user can do
against the platform's root-owned volume and config mount. gVisor is the
isolation boundary here, not the in-guest uid.

The administrator persona is blocked: the Control UI is a WebSocket SPA
behind gateway device pairing with no DOM logout control for the shared
helper. The SSO round-trip itself is covered by test-oidc-login.js, which
drives the full unauthenticated to Keycloak to authenticated path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 19:09:57 +02:00