Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 79bf95b751 fix(svc-storage-nfs-server): serve ganesha on every default distro
The ganesha package and service maps were indexed by
ansible_facts['os_family'] but keyed with distribution names, so the
Fedora and Ubuntu entries were unreachable and os_family RedHat matched
nothing. Host deploys on Fedora aborted with "object of type 'dict' has
no attribute 'RedHat'".

Key both maps by os_family and cover the whole default distro matrix:

- RedHat: Fedora ships nfs-ganesha in its base repo. Its 9.16 build is
  linked against NTIRPC_7.2 symbols the current libntirpc does not
  export, so ganesha.nfsd exits 127; pull libntirpc from updates-testing
  to restore a runnable pair.
- RedHat on EL: nfs-ganesha is in no base, AppStream, CRB or EPEL repo,
  only in the CentOS Storage SIG, so enable that repository first.
- Archlinux: no official package exists at all. Build it from the AUR
  and ship the systemd unit, because the AUR package installs binaries
  only.

Drop the unreachable distribution keys here and in the client role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:25:49 +02:00
kevinveenbirkenbachandClaude Opus 5 b20a70646f fix(swarm-test): give the sandbox tier a second node in the lab
Dedicating a single worker to the sandbox left the drain step with nowhere to
put the agent: draining the only labelled node failed with `no suitable node
(scheduling ...)`. Two labelled workers are also what the requirement asks for,
so the earlier single-worker choice traded one defect for another.

The cost is deliberate: the inverse placement constraint pushes every
non-sandbox service off both workers, so the stack host carries the rest of the
platform. That is the topology the requirement describes, and whether the stack
host holds up under it is what the next swarm run measures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 12:25:13 +02:00
kevinveenbirkenbach c0a3a69b1a Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees
# Conflicts:
#	utils/tests/swarm/extend_inventory.py
2026-07-27 11:41:10 +02:00
kevinveenbirkenbachandClaude Opus 5 abbd774e8b refactor(swarm-test): read the backup placement in extend_inventory
The inventory groups and the remote-2-local provider list are the same fact about which node holds a backup repository. extend_inventory now consumes backup_repos.repo_placements instead of restating the closure checks, so the two cannot drift apart again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:29:47 +02:00
kevinveenbirkenbachandClaude Opus 5 0bde90e47d fix(swarm-test): derive the DR drill backup providers from the closure
The drill configured svc-bkp-remote-2-local with a fixed manager plus NFS provider list for every app, while the manager only runs a backup repository when the app's include closure pulls svc-bkp-volume-2-local or svc-bkp-secrets-2-local in. For an app whose container_backup consumer is statically off, such as svc-dns-unbound, user-backup never ran on the manager, so the pull was rejected with 'Permission denied (publickey,password)' and the fail-if-any wrapper took the whole unit down at drill step 4/9. Four roles reach that step today: svc-dns-unbound, svc-db-redis, svc-registry-cache and svc-registry-docker.

backup_repos.py expresses the placement once and matrix derives the provider list from it per round. derive_includes accepts the round's variant map because INFINITO_APP_VARIANTS only reaches the subprocess callers. The extras file keeps the device paths the drill reads back and loses the provider list nothing read from it, and the drill banners stop claiming backup legs that were never installed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:29:47 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ac7872e56 style(deploy): apply autoformat to the deploy round handler
Import ordering and one comprehension reflowed by make autoformat. No
behaviour change; kept out of the orphan-prune commit so that diff stays
readable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 11:29:47 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f62d432873 fix(deploy): apply the disable orphan-prune on the deploy side too
init.py prunes the include set twice: it drops the roles named by disable
and then re-walks the dependency graph so providers that only existed to
serve a disabled role are dropped as well. deploy/cli.py filtered only
the named roles, so its deploy-id set stayed a superset of what the
inventory actually contains.

With disable=matomo,dashboard,prometheus,email,css the init side pruned
svc-db-redis, svc-db-mariadb, web-svc-cdn, web-svc-file, web-svc-asset
and web-svc-simpleicons; the deploy side still passed them to
validate_application_ids, which aborted the run with 'not present in
inventory' before any role deployed.

Both sides now call prune_orphans_after_disable with the same
round-merged services map, so the sets match. Document the two-sided
contract in the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 11:29:47 +02:00
Alejandro Roman 871b00670a Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-27 11:25:42 +02:00
kevinveenbirkenbach d67aea7f6b Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees
# Conflicts:
#	utils/tests/swarm/extend_inventory.py
2026-07-27 11:25:20 +02:00
kevinveenbirkenbachandClaude Opus 5 32e39ceb58 refactor(swarm-test): read the backup placement in extend_inventory
The inventory groups and the remote-2-local provider list are the same fact about which node holds a backup repository. extend_inventory now consumes backup_repos.repo_placements instead of restating the closure checks, so the two cannot drift apart again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:24:04 +02:00
kevinveenbirkenbachandClaude Opus 5 1f87e632f9 refactor(swarm-test): read the backup placement in extend_inventory
The inventory groups and the remote-2-local provider list are the same fact about which node holds a backup repository. extend_inventory now consumes backup_repos.repo_placements instead of restating the closure checks, so the two cannot drift apart again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:18:53 +02:00
kevinveenbirkenbachandClaude Opus 5 d7cd4a2a53 fix(swarm-test): derive the DR drill backup providers from the closure
The drill configured svc-bkp-remote-2-local with a fixed manager plus NFS provider list for every app, while the manager only runs a backup repository when the app's include closure pulls svc-bkp-volume-2-local or svc-bkp-secrets-2-local in. For an app whose container_backup consumer is statically off, such as svc-dns-unbound, user-backup never ran on the manager, so the pull was rejected with 'Permission denied (publickey,password)' and the fail-if-any wrapper took the whole unit down at drill step 4/9. Four roles reach that step today: svc-dns-unbound, svc-db-redis, svc-registry-cache and svc-registry-docker.

backup_repos.py expresses the placement once and matrix derives the provider list from it per round. derive_includes accepts the round's variant map because INFINITO_APP_VARIANTS only reaches the subprocess callers. The extras file keeps the device paths the drill reads back and loses the provider list nothing read from it, and the drill banners stop claiming backup legs that were never installed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:16:07 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e365ac4664 style(deploy): apply autoformat to the deploy round handler
Import ordering and one comprehension reflowed by make autoformat. No
behaviour change; kept out of the orphan-prune commit so that diff stays
readable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 11:16:04 +02:00
kevinveenbirkenbachandClaude Opus 4.8 82ff7fe29e fix(deploy): apply the disable orphan-prune on the deploy side too
init.py prunes the include set twice: it drops the roles named by disable
and then re-walks the dependency graph so providers that only existed to
serve a disabled role are dropped as well. deploy/cli.py filtered only
the named roles, so its deploy-id set stayed a superset of what the
inventory actually contains.

With disable=matomo,dashboard,prometheus,email,css the init side pruned
svc-db-redis, svc-db-mariadb, web-svc-cdn, web-svc-file, web-svc-asset
and web-svc-simpleicons; the deploy side still passed them to
validate_application_ids, which aborted the run with 'not present in
inventory' before any role deployed.

Both sides now call prune_orphans_after_disable with the same
round-merged services map, so the sets match. Document the two-sided
contract in the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 11:16:00 +02:00
kevinveenbirkenbachandClaude Opus 5 d6b00f5540 fix(swarm-test): derive the DR drill backup providers from the closure
The drill configured svc-bkp-remote-2-local with a fixed manager plus NFS provider list for every app, while the manager only runs a backup repository when the app's include closure pulls svc-bkp-volume-2-local or svc-bkp-secrets-2-local in. For an app whose container_backup consumer is statically off, such as svc-dns-unbound, user-backup never ran on the manager, so the pull was rejected with 'Permission denied (publickey,password)' and the fail-if-any wrapper took the whole unit down at drill step 4/9. Four roles reach that step today: svc-dns-unbound, svc-db-redis, svc-registry-cache and svc-registry-docker.

backup_repos.py expresses the placement once and matrix derives the provider list from it per round. derive_includes accepts the round's variant map because INFINITO_APP_VARIANTS only reaches the subprocess callers. The extras file keeps the device paths the drill reads back and loses the provider list nothing read from it, and the drill banners stop claiming backup legs that were never installed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:07:04 +02:00
Alejandro Roman 29b4e9f9ed fix(backup): sql-dump the dedicated stalwart DB — postgis image name defeated baudolo's postgres detection 2026-07-27 10:18:44 +02:00
kevinveenbirkenbachandClaude Opus 5 80f37e9af8 fix(swarm-test): name containerd in the wedged-container recovery hint
Restarting docker does not clear a container whose shim never sent an exit
event: that shim belongs to containerd, a separate unit that survives the
restart. Measured on a host carrying two such remnants - docker.service had
been restarted hours earlier while containerd.service had been up for three
days, and `rm -f`, plain `rm` and `stop -t 1` all still failed afterwards.

The hint also has to say what the restart costs. It wipes every exec instance,
so following it during a deploy kills the run: an act deploy died after 1h48m
with `failed to inspect exec: No such exec instance`, and the log carried no
ansible error at all, only post-steps failing in milliseconds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:13:07 +02:00
kevinveenbirkenbachandClaude Opus 5 efe828b7a4 fix(swarm-test): detach NFS before the node containers die
Every teardown left one node container behind, always the same one, and the
warning blamed a kernel D-state. The cause was a budget: the client-unmount
step gave all four containers one shared 120s allowance, the unmount chain
costs up to three attempts per mount, and the third container in the list never
got its turn. That container was then killed with its mount still attached.

Killing was the second half of it. The node image sets STOPSIGNAL SIGRTMIN+3 so
that `docker stop` triggers a clean systemd poweroff, but the teardown only ever
ran `docker kill`, which skips it and leaves the nested engine and its task
containers to die mid-flight. Those task containers hold binds of the NFS
superblock inside their own mount namespaces, where an unmount in the node
namespace cannot reach them, so the nested engine has to go down first.

Under act the ansible controller is the act job container itself, so its own
NFS mount is invisible to the host mountpoint check and survived every cleanup.

Three more paths reached the same containers without any of this: the
cross-cluster reaper removed them with no unmount at all and then reported the
wedges it had just manufactured, the bootstrap pre-clean detached only inside
the NFS server, and the escape hatch filtered its candidates down to
`name=nfs-server` so it could never see the worker that actually survives.

The removal verdict itself was wrong: a single inspect fired immediately after
`rm -f` reports containers the daemon is still removing, and two of the three
warnings in the observed run were false alarms.

Verified by tearing a live five-node cluster down: all five containers removed,
no remnant, no warning.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:12:01 +02:00
kevinveenbirkenbachandClaude Opus 5 dfb65b16c8 fix(svc-virt-kata): make the sandbox tier operator-named
A sandbox node runs gVisor as its daemon-wide default runtime and every
container scheduled there inherits it, so keycloak failed to start on one.
Swarm has no taints, which leaves the inverse placement constraint as the only
lever: sandbox consumers are pinned to the labelled nodes, everything else is
kept off them. Both halves were measured on a live cluster first, because a
docker constraint has no presence test and rejects an empty comparison value:
`!= true` does match a node without the label, and three replicas under it
avoided two labelled workers entirely.

That makes the set of labelled nodes load-bearing for the whole platform, and
it was decided by accident. Without an `svc-virt-kata` inventory group the
service loader runs the role on every host, so every worker becomes a sandbox
node and the rest of the platform is evacuated onto the stack host. The role
now refuses to run in swarm mode until the operator names those nodes.

With the group present the opposite gap opened: the provisioner puts every role
group on the single `--host` it is given, so the role lands on the manager
alone, the label task skips itself there, and no node ends up labelled. The old
assert only proved that a worker exists, which says nothing about where the
role ran, so the deploy passed the check and then hung until the convergence
wait gave up. The assert now reads the labels back from the stack host.
`docker node ls --filter label=` matches engine labels, not swarm node labels;
the filter has to be `node.label`.

The constraint itself is emitted only where a sandbox tier exists, so a deploy
without an agent role is untouched by any of this, and the lab dedicates a
single worker: labelling both would leave the manager as the only home for
every other service.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:08:36 +02:00
kevinveenbirkenbachandClaude Opus 5 9d3eb4a79c fix(sys-svc-container): register the runtimes the host actually has
`svc-virt-kata` is pulled in as a shared service of the agent roles, so it also
executes on hosts that are in no inventory group of that name. Gating the
runtime registration on `'svc-virt-kata' in group_names` therefore left such a
host with runsc installed and its node labelled kata-capable, while
`daemon.json` registered nothing and the daemon kept running everything under
runc: the exact isolation breach the label promises is closed. The stat probes
in `daemon_config.yml` decide instead, and they cannot be fooled by an
inventory that says otherwise.

Nothing else in the tree installs or references runsc, so a shim on disk is
proof that this role put it there.

The agent compose templates pin their runtime on the same `services.kata`
switch that already drives the swarm placement constraint, instead of pinning
unconditionally against a daemon config written under a different condition.
Both roles declare that key themselves, so the lookup is the strict form: a
deleted flag now aborts the render rather than silently starting the agent
without a sandbox.

The new unit test covers what nothing covered before, which is how the runc
breach went unnoticed: an absent shim is never registered, a default runtime is
always a registered one, the stack host keeps the shared kernel, compose never
sets a daemon-wide default, and registration follows the probe rather than the
inventory group.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:06:41 +02:00
kevinveenbirkenbach 87dce12024 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-07-27 04:45:47 +02:00
kevinveenbirkenbachandClaude Fable 5 eacaa4a53b fix(ci): honor tests.yml skip in guide role pool
guide_select built its random pool from invokable + lifecycle only, so
the Guide job could pick roles whose meta/tests.yml declares every
deploy mode untestable (svc-storage-nfs-client, which hard-asserts a
reachable NFS server). Filter the pool by the role's declared skip for
its guide mode; a malformed tests.yml now fails selection loudly
instead of being silently ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 04:42:31 +02:00
kevinveenbirkenbachandClaude Opus 5 20f744459a style(web-app-bigbluebutton): name the tag task by intent
The task name repeated the `jwilder/dockerize:latest` literal that the
command below it already carries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 04:04:19 +02:00
kevinveenbirkenbachandClaude Opus 5 fc16056c8b fix(svc-virt-kata): apply the sandbox runtime on swarm workers
`docker stack deploy` drops the per-service `runtime:` key, so a swarm worker
can only carry the sandbox as its daemon-wide `default-runtime`. That was never
set: the role reached the manager only, no node was labelled, and one of three
agent replicas executed model-generated code under plain runc.

Setting it from the base render is not possible either. The container role
writes `daemon.json` before the runtime binary exists, and dockerd refuses to
start when a runtime path does not resolve ("stat /usr/local/bin/runsc: no such
file or directory"). The role's own attempt to re-render afterwards was a silent
no-op, because `include_role: sys-svc-container` hits the
`run_once_sys_svc_container` guard.

Move the probes plus the render into `daemon_config.yml` and let the role
re-enter it via `tasks_from` after installing runsc. Only runtimes whose binary
the probe found are registered, and `default-runtime` is written only for a
present shim on a non-stack host, so the stack host keeps the shared kernel for
the platform's own services.

The placement constraint compares against a fixed literal because Docker
constraints have no presence test: label the node `kata-capable=true` rather
than with the runtime name, otherwise a Kata node fails a `== runsc`
comparison. Without a worker outside the stack host no node is eligible at all,
so refuse the deploy instead of leaving the service pending forever.

`select_runtime.yml` is gone: hermes and openclaw included it by absolute path
out of a foreign role, and one probe in the container role now serves every
consumer.

Verified live on a three node cluster: worker `DefaultRuntime=runsc`, both
workers labelled, manager unlabelled, service constrained, and the running
replica reports `runtime=runsc`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 04:00:59 +02:00
kevinveenbirkenbach 1f30544f06 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-07-26 21:53:54 +02:00
kevinveenbirkenbachandClaude Fable 5 caae36a716 fix(setup): prune stale generated group files
tasks/groups/*-roles.yml are gitignored, so branch switches leave
orphans behind; a stale svc-virt-roles.yml injected a phantom group
into the stage order and referenced a role absent on this branch.
Remove any group file not in the current invokable set before
regenerating, and drop narration comments flagged by the comment lint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:46:19 +02:00
kevinveenbirkenbachandClaude Fable 5 cd26d6ec0c fix(web-app-bigbluebutton): use container wrapper for dockerize pre-pull
Raw docker pull/tag in the bootstrap tasks fail the no-raw-docker gate;
route both through the engine-agnostic container wrapper, shorten the
84-char tag task name for the emoji/length lint, and add the dockerize
node to the README services graph.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:46:19 +02:00
kevinveenbirkenbachandClaude Fable 5 8916ee0ac3 fix(web-app-bigbluebutton): mirror the dockerize helper
Upstream generate-compose runs docker run jwilder/dockerize with no
tag pin, no pre-pull and no retry: the only direct Docker Hub pull in
the whole deploy, and a 15s Hub timeout killed a 62-minute swarm run
(30161503176 job bbb-0). Declare dockerize as a disabled pseudo
service so images-mirror-missing and mirrors.yml cover it, pre-pull
the resolved ref with retries and retag it to the exact name the
script requests, so the local image is used and Hub is never
contacted in CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:43:07 +02:00
kevinveenbirkenbachandClaude Fable 5 64c3f32d2d fix(web-app-moodle): fill password after toggle re-render
core/togglesensitive fetches its template async and replaces the
password input via outerHTML, serializing the empty value attribute
and discarding anything typed before the swap. Swarm timing lost that
race 3/3 (empty password= in every POST body of run 30161503176),
compose won it. Both specs now wait for the post-replacement
.toggle-sensitive-wrapper input before filling. profile-readonly also
asserts login success; it previously passed vacuously on the login
page where none of the 19 profile fields exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:42:42 +02:00
kevinveenbirkenbachandClaude Fable 5 3c60469c15 fix(web-app-zammad): gate wizard on pk-seq reset
Setting.any? turns true after the first of ~22 seed files, while
user_nr_1 inserts user id=1 explicitly and DbHelper.import_post resets
the sequences only after all seed files. An AutoWizard run inside that
window gets nextval=1 and dies on users_pkey (swarm variant-2 failure
in run 30161503176). users_id_seq.is_called becomes true exactly at
the reset, so the gate now waits for completed init.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:42:27 +02:00
kevinveenbirkenbachandClaude Fable 5 ffefcfe7e3 fix(web-app-bigbluebutton): mirror the dockerize helper
Upstream generate-compose runs docker run jwilder/dockerize with no
tag pin, no pre-pull and no retry: the only direct Docker Hub pull in
the whole deploy, and a 15s Hub timeout killed a 62-minute swarm run
(30161503176 job bbb-0). Declare dockerize as a disabled pseudo
service so images-mirror-missing and mirrors.yml cover it, pre-pull
the resolved ref with retries and retag it to the exact name the
script requests, so the local image is used and Hub is never
contacted in CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:39:13 +02:00
kevinveenbirkenbachandClaude Fable 5 4935622486 fix(web-app-moodle): fill password after toggle re-render
core/togglesensitive fetches its template async and replaces the
password input via outerHTML, serializing the empty value attribute
and discarding anything typed before the swap. Swarm timing lost that
race 3/3 (empty password= in every POST body of run 30161503176),
compose won it. Both specs now wait for the post-replacement
.toggle-sensitive-wrapper input before filling. profile-readonly also
asserts login success; it previously passed vacuously on the login
page where none of the 19 profile fields exist.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:37:49 +02:00
kevinveenbirkenbachandClaude Fable 5 b8dc2c1999 fix(web-app-zammad): gate wizard on pk-seq reset
Setting.any? turns true after the first of ~22 seed files, while
user_nr_1 inserts user id=1 explicitly and DbHelper.import_post resets
the sequences only after all seed files. An AutoWizard run inside that
window gets nextval=1 and dies on users_pkey (swarm variant-2 failure
in run 30161503176). users_id_seq.is_called becomes true exactly at
the reset, so the gate now waits for completed init.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:36:57 +02:00
kevinveenbirkenbach a3ef5c835d Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-07-25 22:18:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 08a6dd157c feat(svc-ai-robot): embody an agent on a dedicated device
Requirement 034 inverts the isolation model of the other agent roles: the
boundary is the whole machine rather than a kernel boundary inside it.
Adding this role to a device switches the agent from isolated to
embodied.

The agent then runs privileged with exactly the devices from
ROBOT_DEVICE_ALLOWLIST bound into its container, and the kernel-isolating
runtime is dropped, because a microVM between the agent and the hardware
it is meant to drive defeats the purpose. The allowlist lives in
group_vars so the agent templates can read it; a role variable is not
visible across roles.

The admission checks run inside the agent role before it deploys rather
than in this role afterwards. The agent is declared shared, so the
service loader brings it up during the constructor stage while svc-ai
roles only run at the following step; a check placed here would sit
behind the deployment it exists to prevent.

The dedicated-device check now compares against something group_names
actually carries: any other web-app role on the host. The previous list
held Docker node labels, which are a different namespace from Ansible
inventory groups, so that intersection could never match and the failure
message always blamed a multi-node swarm.

Which agent embodies the device follows from the topology: both agent
flags take the dynamic in-group_names form, Hermes Agent wins when both
are present, and the matrix pins one per round so neither path rots.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:16:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c9bbc33654 style(deploy): apply autoformat to the deploy round handler
Import ordering and one comprehension reflowed by make autoformat. No
behaviour change; kept out of the orphan-prune commit so that diff stays
readable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:26:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ba6176eb7c docs(roles): apply the dash and Credits fixes to two missed READMEs
Two single lines the staging filters skipped. desk-nextcloud kept the em
dash because the exclusion pattern meant for web-app-nextcloud matched it
too, and web-app-flowise kept the bare author name because it was sorted
into the prose bundle and never returned to the Credits one.

Both rules were already enforced everywhere else, so this only closes the
last file each.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:23:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 12ddb4c0d2 fix(web-app-hermes): restore the sso and logout lint opt-outs
The sso and logout rules are real opt-out mechanisms, named as such in
the failure messages of test_sso and test_logout_dashboard. Replacing
them with dynamic-flag silenced a different rule and broke both lints,
plus test_env_services_match as a consequence.

All three rules apply to these entries: the role opts out of SSO because
an oauth2 proxy in front of a bearer-token API breaks OpenAI-compatible
clients, the static false is deliberate rather than topology-dependent,
and no Playwright spec gates on either service, so rendering an env flag
for them would be noise. web-app-homeassistant already carries the same
three-token form for the same situation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:22:23 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8b5312445b docs(roles): write the AI and runtime role documentation
The eight new roles shipped with the README template unfilled: every one
of them described itself as "[Name](https://example.com/) is an
application" with a single "Describe a capability" feature. They were
implemented, deployed and tested; their documentation was placeholder
text.

Each role now carries a Description, Overview and Features written
against its own services, tasks and templates, with TODO.md as the
binding list of what is not wired. That caught a false claim: OpenClaw
advertised routing through the shared gateway, but it reads model
providers from openclaw.json rather than the environment, so the gateway
only shapes its deploy topology. The claim is gone.

The overview table derives its Name column from each README H1, and all
eight had kept the humanised fallback. Worst of these, both LiteLLM roles
rendered as "Litellm" on adjacent rows, so a reference in the Integrated
with column could not say which one it meant; they are now the Gateway
and the Admin UI. The rest take their official spelling.

The galaxy descriptions all repeated the product name the Name column
already carries, one named a role id in prose and one used a hyphen as a
dash. svc-virt-kata also claimed to install Kata, which it does not: it
installs gVisor and registers an already-present Kata shim.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:21:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 479a55ef5c style(readme): drop the em dash from the role README template
House style bans em and en dashes. Established role READMEs carry none,
but the template had one in its Quick Setup line, so it had been added
after those roles were generated and reappeared in every regenerated
README since. Replacing it with a colon propagates to the 54 roles the
generator refreshed.

The Credits condition switches from an "is defined" test to a truthiness
test because the generator now always sets application_author_url,
passing None for an unknown author. Left as it was, the branch would have
rendered a broken [Name](None) link instead of falling back to the bare
name.

roles/categories.yml loses the same dashes from the svc.virt description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:19:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5cf0b1c543 feat(readme): enforce consistent Credits attribution
A person credited with a link in one place and a bare name in another
reads as two different people and drops the reader's path to them.
sys-svc-engine showed it in two adjacent lines: the same name once bare,
once linked.

The new credits-attribution lint requires a name linked in any Credits
section to carry that link everywhere, and rejects one name pointing at
two URLs. It found 13 offenders, including a second contributor whose
attribution had drifted in test-e2e-cli.

The generator produced the defect on every run: its context carried
application_author but never application_author_url, so the template
always took the unlinked branch. Existing READMEs kept their correct
Credits because the generator only fills missing sections, which is why
239 roles were right and 12 wrong.

The name to URL map is derived rather than declared: galaxy_info.company
names the project owner, not the role author (web-app-mattermost is
authored by Alejandro Roman Ibanez but lists Kevin Veen-Birkenbach as
company), so only the Credits lines themselves carry the mapping.
Generator and lint share that derivation and cannot drift apart, and a
new contributor needs one linked credit rather than a code change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:18:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 66a1259842 refactor(web-app-nextcloud): point integration_openai at the shared gateway
Nextcloud reached its OpenAI-compatible backend through Flowise, which
only forwarded to the LiteLLM sidecar that Flowise no longer runs. The
consumer flag, the activation condition and the endpoint now bind to
svc-ai-litellm directly, one hop shorter.

The endpoint switches from Flowise' public TLS URL to the internal
gateway address plus /v1, so a container-to-container call no longer
travels through the reverse proxy.

The api_key stops crossing a role boundary: it was read from
web-app-flowise credentials.llm_master_key, handing Nextcloud
administrative access to the whole gateway, and is now this role's own
per-consumer virtual key.

The litellm variant pin sits on variant 7 rather than 3 because adding
the gateway to variant 3 pushed that round past the matrix memory budget.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:16:52 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0c4d74cc7c feat(web-app-homeassistant): expose the automation hub as an MCP server
The first MCP server role: where the agent roles consume tools, this one
offers them. Its services.mcp block declares an internal streamable-HTTP
endpoint gated on an agent being deployed, so the surface only exists
when something can use it, and it stays read-only because a privileged
auth subject must not carry mutating tools.

Home Assistant rejects any request carrying X-Forwarded-For from an
undeclared proxy, answering 400 for every call behind the platform's
reverse proxy. The role therefore renders a configuration.yaml that
trusts the stack subnet, keeping default_config so mounting the file does
not drop the integrations Home Assistant generates on first start.

As a shared_net provider its compose file references its own network as
external, which nothing creates in compose mode, so the role pre-creates
it the way prometheus, minio and seaweedfs already do. The vhost declares
status_codes [200, 302, 405] because the health probe sends HEAD and the
hub answers 405.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:15:18 +02:00
kevinveenbirkenbachandClaude Opus 5 6e14d12b59 perf(test-e2e-playwright): stage shared assets once per run
The Playwright image was pulled once per role, and the shared helpers
and persona modules were staged file by file. The pull now happens once
for the whole run, and the personas tree is copied in a single task.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:38 +02:00
kevinveenbirkenbachandClaude Opus 5 b442a670f9 refactor(svc-net-tor): extract onion reachability tasks
The three onion-reach tasks repeated the same egress-router and
node-onion guard; they now live in onion_reach.yml behind a single
include condition.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:38 +02:00
kevinveenbirkenbach 0e64865297 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 5 5318df2e9d feat(ci): resolve image refs from services.yml SPOT
moby/buildkit was hardcoded in four places across two workflows and catthehacker/ubuntu in two act scripts, none of them reachable by the image mirror: utils/docker/image/discovery.py only scans roles/**/meta/services.yml. So the one image the CI build cannot proceed without was pulled straight from Docker Hub, unmirrored, which is what broke the fedora build in run 30160077828.

Declare both as mirror-only services on svc-runner, following the test-e2e-playwright precedent, so the existing mirror pipeline picks them up unchanged. New scripts/meta/resolve/image/service_ref.sh reads image+version back out of the declaration and is the single source for consumers outside Ansible; its mode argument is mandatory (upstream never touches the network, mirror prefers the GHCR tag and falls back to upstream). The fallback is why the mirror job can keep running in parallel with the build instead of gating it.

ACT_PLATFORM_IMAGE moves to an env handler so act consumers read the declared value via BASH_ENV, and workflow.sh now demands it instead of defaulting to a literal. Operator overrides still win through eb.setdefault.

Verified: service_ref.sh in all four paths (upstream, mirror hit, missing mode, invalid mode); the mirror path matches GHCRProvider.image_base() character for character and resolves against the live registry anonymously; ACT_PLATFORM_IMAGE resolves through the real BASH_ENV mechanism; make quality green on all four targets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 5 5d8253f004 fix(web-app-gitea): hoist delegate_to to apply:
Commit 99bc9a94d gave every top-level task in cleanup/ldap.yml and cleanup/oidc.yml an identical delegate_to, tripping the include-uniform-apply-keyword lint and failing CI run 30160077828.

apply: on include_tasks wraps the included tasks in a Block, and delegate_to is Block-valid, so hoisting it to the include site is semantically identical: register still resolves on the original host, the per-task when: gate is unaffected, and the include-site when: does not leak into the block. Leaf-only keywords (failed_when/changed_when) stay per-task per 64eef3440.

Matches the established pattern in keycloak, mediawiki, mastodon, matomo, espocrm, collabora and shopware. setup/ldap.yml and setup/oidc.yml deliberately keep the per-task form, since their undelegated set_fact must not run on the hosting node.

make quality green: test-external, test-integration, test-lint, test-unit all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bcd32463a2 feat(web-app-openclaw): deploy the OpenClaw agent behind an SSO proxy
The second agent of requirement 032, pinned to the same isolating runtime
as hermes. Unlike hermes it exposes a browser Control UI rather than a
bearer API, so it takes the oauth2 proxy flavour with its own sso port;
a proxy in front of a browser surface is correct where it would break
OpenAI-compatible clients.

Three deploy failures shaped this role. OpenClaw maps no environment
variables onto config keys, so it restart-looped on 'Missing config' and
then on 'non-loopback Control UI requires allowedOrigins'; both now come
from a rendered openclaw.json mounted as a config. That render must run
after sys-stk-full, because it notifies compose up and the handler would
otherwise fire before the compose file exists. And the process runs as
root: OpenClaw refuses a config file not owned by the running uid and
writes cron state into its home, neither of which a non-root user can do
against the platform's root-owned volume and config mount. gVisor is the
isolation boundary here, not the in-guest uid.

The administrator persona is blocked: the Control UI is a WebSocket SPA
behind gateway device pairing with no DOM logout control for the shared
helper. The SSO round-trip itself is covered by test-oidc-login.js, which
drives the full unauthenticated to Keycloak to authenticated path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 19:09:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5d72ddc357 feat(web-app-hermes): deploy the Hermes agent under an isolating runtime
The first agent workload of requirement 032. Its compose service pins
runtime: to the fact svc-virt-kata publishes, so model-generated code
runs behind a kernel or syscall boundary rather than plain runc. The pin
goes through compose_only because docker stack deploy discards the key;
swarm relies on the node-default runtime and a placement constraint
instead.

Hermes is not a dashboard. The container starts its gateway and serves an
OpenAI-compatible API on 8642, which shapes three decisions: the env
block configures API_SERVER_*, inference points OLLAMA_BASE_URL at the
LiteLLM gateway (Hermes' Ollama client speaks the same wire format), and
the vhost declares status_codes [200, 404] because the API server maps
only /health and /v1/*, so the bare root answers 404 by design and the
webserver health unit would otherwise fail the host.

SSO is off deliberately: an oauth2 proxy in front of a bearer-token API
breaks OpenAI-compatible clients. Authentication is API_SERVER_KEY, and
the guest spec asserts that /v1/models without it is rejected while the
administrator spec asserts the key unlocks the listing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 19:08:32 +02:00