A lookup plugin receives group_vars through variables.get() untemplated, and
group_vars/all/18_swarm.yml:2 defines DEPLOYMENT_MODE as a Jinja expression.
The comparison against 'swarm' could therefore never match, so every swarm
deploy fell through to _COMPOSE_HOST and baked
socks5h://host.docker.internal:9050 into Nextcloud's proxy system value.
svc-net-tor publishes SOCKS with mode: host, so that listener exists on the
manager alone. With three Nextcloud replicas against one Tor node the failure
is node-determined, not intermittent: the 104 'cURL error 7 ... over proxy
host.docker.internal' lines in run 31340965955 split 56 on wrk-02, 48 on
wrk-01 and 0 on mgr-01, which is what made the OIDC callback 500 on 2 of 3
replicas and read as a 39% flake.
Templating first yields socks5h://tor:9050 in swarm, which every replica
reaches over the overlay it already joins through the onion_sso consumer rule.
Compose is unchanged: a single-node inventory renders the expression to
'compose', and a suppressed template error falls through to the same branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
08_users/realm.yml seeded the realm from `lookup('users') | non_reserved_users`
while 08_rbac_groups_realm.yml built group memberships from the raw
`lookup('users')`. The membership builder has its own guards, but both are dead
against real data: across all 269 default users, none carries a `reserved` key
and none has an empty password. So membership was effectively unfiltered while
seeding filtered on `accounts`.
helpdesk and no-reply (accounts: [mailbox], roles: [bot]) were therefore
demanded as members of roles/web-app-mailu/bot but never created, and
ensure_group_path_members.sh exits 1 on the first miss. Four jobs -- Mattermost 1,
Friendica 1, Matrix 1, OpenProject 1 -- each burned five cross-node recovery
attempts on a data failure no reschedule could fix.
Rather than adding the filter at the site that forgot it, remove the ability to
forget: lookup('users_non_reserved') resolves users and applies the predicate in
one place, and all four consumers now read it -- the two Keycloak user tasks, the
RBAC group builder, and OPENLDAP_USERS, which had it right all along. The filter
stays as the pure predicate the lookup calls.
Reproduced offline against the real user model, byte-identical to the CI member
list: before, bot demanded helpdesk and no-reply; after, members are exactly the
four seeded accounts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
KEYCLOAK_REDIRECT_URIS derived the scheme from Keycloak's OWN TLS state and
stamped that single scalar onto every SSO consumer. With svc-net-tor
deployed Keycloak is onion-primary, so resolve_enabled returns False and
lookup('tls', ...) yields "http" -- which then went onto clearnet apps too.
Keycloak registered http://matrix.infinito.example/* while Synapse sends
https://.../_synapse/client/oidc/callback, and its redirect-URI match is
scheme-exact: 90 invalid_redirect_uri events, an error page instead of a
login form, and a 60s selector timeout that looked like a Playwright bug.
jitsi hit the same wall on https://meet.infinito.example/oauth2/callback.
Resolve per consumer instead, via resolve_enabled: an .onion is always
plaintext, a clearnet app follows its own server.tls.enabled, and only
then the global TLS_ENABLED. That override existed and was silently
discarded -- passing a pre-computed scheme string threw it away, which is
the same "what I am" instead of "what the consumer is" mistake one layer
up. resolve_enabled handles the onion case itself, so the filter's own
is_onion_domain branch is gone rather than duplicated.
Move it from a filter to a lookup while at it: the thing needs domains,
applications and TLS_ENABLED anyway, so resolving them inside collapses
the call site to lookup('redirect_uris') and drops
KEYCLOAK_REDIRECT_FEATURES, which listed the same path twice. A lookup
under plugins/lookup/ is also plainly importable, so the test loses the
sys.modules stubbing apparatus a role-local filter needed -- 60 lines net
smaller.
KEYCLOAK_WEB_ORIGINS is regex-derived from these URIs and inherits the
fix. Verified against the real applications view: 69 URIs, all https with
TLS_ENABLED true and all http with it false -- the same 69 the failing run
registered as plaintext.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
libcurl refuses a .onion host by name (RFC 7686, error 6) before it
consults /etc/hosts, extra_hosts or DNS, so the platform's DNS+iptables
onion egress cannot serve a server-side backchannel at all. Measured:
curl --noproxy '*' -> exit 6
curl --resolve auth.<onion>:80:127.0.0.1 … -> exit 6
--resolve fills curl's own cache strictly earlier than getaddrinfo and
is still refused, which makes sys-svc-container's onion_oidc_socks
template -- extra_hosts, despite its name -- a no-op for every libcurl
client. Only a configured proxy makes curl skip the check.
Since 9d91fbea8 emptied proxy.config.php.j2 there is no such proxy, while
svc-net-tor's exclusive/primary defaults replace Keycloak's clearnet
domain and KC_HOSTNAME pins the issuer to the onion. Nextcloud's
oidc_login discovery call therefore dies inside PHP, /apps/oidc_login/oidc
returns 500, and oidc_login_auto_redirect sends /login straight there.
Restoring the old socks5h://host.docker.internal:9050 alone would make it
worse: tor runs as a single manager-pinned replica publishing SOCKS with
mode: host, so only the replica on that node could connect and a
deterministic failure would become a 2-in-3 flake. In swarm, though, tor
is NOT host-networked -- that branch is compose-only -- and it already
binds SocksPort 0.0.0.0, so the shared overlay is a usable path from
every node.
Give svc-net-tor a shared_net overlay with alias 'tor' and dial it by
service name. Attachment is gated by a new consumer kind, onion_sso: the
consumer's services.sso.enabled AND the provider's services.tor.enabled.
Both are group-membership derived, so the conjunction is exactly "the
issuer is an onion" and stays false in clearnet deployments -- which
matters because the service registry is not filtered to deployed roles.
The existing kinds could not express it: keying on sso alone would also
hold without tor, and keying on tor would miss Nextcloud's variant 2,
which disables its own onion yet still needs the client path.
tor_socks_proxy keeps the swarm/compose split in one place so the six
roles sharing this defect need not restate it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three svc-db-* credential tests each hand-rolled the same transport
gate in their test.env.j2:
(lookup('config', application_id, 'services.tor.enabled') | bool
and lookup('config', application_id, 'services.<entity>.exposed') | bool)
Three copies of one predicate, each naming its own entity key, and each
free to drift from the onion mapping that actually decides reachability.
Extract it into plugins/lookup/tor_reachable.py, split the same way as
its sibling tor_ports.py: a plain is_reachable() for callers holding the
merged applications view, plus a thin LookupModule resolving it through
lookup('applications'). Because collect_exposed_ports() scans
services.values(), the entity name disappears from the templates
entirely -- they collapse to
TOR_ENABLED={{ lookup('tor_reachable', application_id) | lower }}
Rework the integration guard accordingly: it now asserts that every
svc-db-* template delegates to the shared lookup, and drives its
base/variant agreement table through the real is_reachable() instead of
re-implementing the predicate. Add unit coverage for the new lookup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reconciler addresses a client with tasks_from: utils/mcp.yml, which
only works when every role offers its MCP work at the same path. The
files were named per role, and on the two agents the work sat inline in
the entry point, where nothing but a full role run could reach it.
The filter docstring pointed at the pre-rename path and now names the
file that exists.
A client rendered an endpoint for every provider the inventory knew,
including providers this run never deploys, so it carried connections
nothing was serving. Discovery now intersects with
application_closure(deployment.whitelist), the same set the service
loader preloads from. An unresolvable closure filters nothing, because
dropping every provider would look exactly like a deployment without
any.
credential_missing stays non-fatal in the app pass, where a provider may
still be waiting to write its secret, and becomes fatal at the
reconciliation stage, where every provider has already run.
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.
Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.
The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.
The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.
The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_ram_gb reimplemented the /proc/meminfo parse that utils/env/runtime.py owns: same pseudo-file, same MemTotal scan, same line.split()[1], same swallow-and-fall-back guard. It now calls mem_total_mb().
The error semantics map exactly. The helper returns 0 for an unreadable file, a malformed line and a missing field; all three previously produced 4.0, and 0 is falsy, so the conservative fallback still covers every one of them.
One real delta: the helper floors kB to MB before the GB conversion, so _ram_gb loses up to 0.001 GB. That can only move the worker count where MemTotal/2 sits within half a megabyte of an exact 1.5 GiB multiple, and it moves it downwards, which is the direction this sizing already prefers.
Touching the file pulled its ten pre-existing narration comments into the diff-scoped comment lint. They were deleted, not relabelled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the category catalogue: this branch added a virt category for
the sandboxed container runtimes, the other did not.
Keeping our side alone would have closed the conflict and broken the build. The
same merge introduces a test requiring every category to declare a hanzi label,
unique among its siblings, and virt predates that rule and carries none. It now
declares 沙箱, checked against its ten siblings for a collision.
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
The mcp role sat in the administrator's unscoped roles list. That list carries
no application, and the RBAC builders admit any user whose roles contain the
name, so the single entry granted mcp in every deployed application at once.
It is replaced by application_roles, a mapping of application to roles, which
the user cache now reads and passes through. Both directory builders resolve
membership through one shared function, so the Keycloak and LDAP paths cannot
drift.
The member list never reached its consumer. mcp_group_members yielded username
strings while the Open WebUI reconciler calls member.get("email"), so the first
real grant would have crashed the deploy. It went unnoticed because the
reconciler returns early on an empty list and nobody held the role. The filter
now yields username and email, which is what the reconciler needs to match
accounts Open WebUI created from an OIDC claim, whose display name need not be
the username. A test drives the real filter output through the real script so
the two sides cannot diverge again.
The Playwright scenario walks the boundary end to end: biber is served no MCP
tools, the administrator adds him to the provider's LDAP mcp group in LAM, and
after signing in again the tools are served. LAM is reached only when it is
present in the inventory, so no service edge or variant is added for it.
The administrator spec inverts its last assertion. With the role scoped, an
administrator holds no application's mcp group, and BYPASS_ADMIN_ACCESS_CONTROL
is false, so the tools must not be served to them either.
A provider overlay keyed on services.mcp.enabled and .shared admitted every
MCP-capable role, servers included, because those flags carry no direction. Each
provider's network therefore collected every other provider and the result was a
mesh in which any sidecar could reach any application. The new mcp_client
consumer kind additionally requires services.mcp.direction to be client or both,
and the four provider overlays switch to it.
A sidecar rendered its networks with the same lookup as its provider and
inherited that provider's consumer attachments, which it neither needs nor
should have. own_network_only keeps it on the provider network alone. A provider
whose own topology is default_net contributes no attachment line at all, and a
bare networks: key is not a list, so the flag returns nothing in that case and
lets the sidecar fall back to the project default, which is that provider's own
network.
Defines what a services.mcp block may declare and enforces it: classification,
transport, auth, implementation and the provenance fields that make an upstream
claim checkable. The lint refuses an unknown key anywhere in the block and
refuses a provider whose allowed consumers cannot speak its transport or its
auth scheme.
Discovery resolves each provider's credential and endpoint per consumer and
per deployment, rejecting a server with a named reason. Transport, auth and
endpoint faults are fatal; a missing credential is not, because it also occurs
when the provider is simply not provisioned yet in the same play.
The filters render the Authorization header from the declared scheme, build
Open WebUI's tool-server connections with their group grants, and derive the
members of those groups from the RBAC assignments. Role scoping gains
application-scoped roles, so an mcp role only counts when it is granted on a
specific application.
roles_with_service gains a scope and a direction filter, and drops the canonical
domain requirement for direction-filtered entries: an MCP URL is built from the
service key and the internal port, so a provider without a public vhost stays
valid.
A whitelist-scoped update preloaded and fully executed every provider role
the host is a member of, because the gate only asked whether the role was in
group_names. On this inventory that meant a single-app update built compose
contexts, rendered configs and vhosts for 17 unrelated applications.
reachable_roles() is lifted out of applications_if_group_and_all_deps() so the
same traversal can start from an arbitrary seed set instead of group_names.
The old function now calls it and filters as before, so every existing caller
keeps its result. A new application_closure lookup exposes it to Ansible, and
sys-service-loader gates each preload on the whitelist's closure. An empty
whitelist yields an empty closure, which the gate reads as the full inventory
scope, so a no-apps deploy is unchanged.
Measured on mode=update apps=web-app-snipe-it: 42:49 before, 17:18 after, with
ok tasks dropping from 4903 to 2030. Nothing outside the closure ran, and
snipe-it's real dependencies (mariadb, redis, openldap, postgres, keycloak,
mailu, prometheus, seaweedfs, cdn, logout) all still did.
The closure also gates the required branch, deliberately. `required` reads as
a global property but plugins/lookup/service.py derives it from group_names,
so it means "some provisioned app needs this" and admits nearly every service.
Exempting it was measured at 42:39 against 17:18 for the same deploy, which is
the entire saving. A service whose provider lies outside the closure is by
definition required by an application outside the deploy scope.
Out of scope: seeding applications_current_play from the closure. That shrinks
CURRENT_PLAY_DOMAINS_ALL, which cert SANs, openresty vhosts, DNS records and
the webserver health check all consume at full inventory scope.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both creates: guards pointed at paths that could never match, so the
bootstrap and setup:install shells re-ran on every deploy.
The first guard went through docker_volume_path, which hardcodes
/var/lib/docker/volumes/<name>/_data and misses any relocated docker
data-root. The second was worse: path_join over the bare volume name
produced a relative path, resolved against the Ansible working
directory, which matched nowhere at all.
Both now derive from the mountpoint the daemon reports. The stack-host
gate moves to a single include, mirroring web-app-akaunting.
With its last caller converted, docker_volume_path is deleted. It could
not be repaired: a Jinja filter runs on the controller and cannot know a
remote host's data-root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both creates: guards pointed at paths that could never match, so the
bootstrap and setup:install shells re-ran on every deploy.
The first guard went through docker_volume_path, which hardcodes
/var/lib/docker/volumes/<name>/_data and misses any relocated docker
data-root. The second was worse: path_join over the bare volume name
produced a relative path, resolved against the Ansible working
directory, which matched nowhere at all.
Both now derive from the mountpoint the daemon reports. The stack-host
gate moves to a single include, mirroring web-app-akaunting.
With its last caller converted, docker_volume_path is deleted. It could
not be repaired: a Jinja filter runs on the controller and cannot know a
remote host's data-root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Since the swarm-NFS migration two chains copied the same bytes: svc-bkp-nfs-2-local
reads the export from the NFS server's own filesystem, while baudolo on the manager
copied the same files again THROUGH the NFS mount. That was a migration leftover -
svc-bkp-nfs-2-local arrived in adb8bcf69 (2026-07-10) and nobody taught the older role
to step back - and the role README already forbade it as a rule nobody enforced. In the
DR drill the duplicate costs three times over: the pulled tree, the device copy and the
restore root.
swarm_nfs_backed() in utils/storage/nfs.py is now the single answer to "does the swarm
NFS rewrite back this volume". plugins/filter/compose_volumes.py (which performs the
rewrite) and this role's backup_volume lookup (which feeds baudolo's
--volumes-no-backup-required) both call it, so the two cannot drift. On the current tree
that excludes 83 of 158 type: volume entries.
Four conditions keep the exclusion from ever dropping a volume nobody else captures:
* a manager-pinned role's volumes are not on the export (web-app-seaweedfs holds
Nextcloud's object store, deliberately node-local per its meta/services.yml)
* a role forcing another mode via compose_mode_force is not on the export either.
That override is per role (roles/sys-svc-compose/defaults/main.yml:3), so it is
resolved per role inside the predicate from roles/<id>/vars/main.yml, not read once
from ambient scope. web-app-bigbluebutton pins compose statically;
web-app-matrix computes it, and a value the predicate cannot resolve counts as
forcing - unknown mode keeps the volume backed up
* an entry opting out with nfs: false stays node-local
* the exclusion applies at all only when the svc-bkp-nfs-2-local inventory group is
populated. That role enters a play by group membership alone
(tasks/groups/svc-bkp-roles.yml:19-22) and svc-storage-nfs-server ships a variant
with nfs_backup: {enabled: false}, so without the gate an export whose repository
nobody deployed would lose 83 volumes from both capture paths
Every condition fails toward the old double capture: wasted space, never lost data.
Compose mode is untouched - the exclusion cannot fire without swarm and an nfs backend.
DEPLOYMENT_MODE and storage are templated through the lookup's templar, because
group_vars/all/18_swarm.yml:2 defines the mode as a Jinja expression; comparing the raw
source string would silently disable the whole exclusion.
Twelve new tests pin the contract, including the two failure modes that would be
invisible in production: an untemplated mode, and an exclusion without an export
repository. Both were verified by mutation - removing either guard makes its test fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four roles declared auth_subject: user while handing the client an
account-bound credential. Nextcloud and Home Assistant mint against the
administrator account, Mattermost issues the administrator's personal access
token, and Baserow creates a dedicated mcp@<domain> account with its own
workspace. The declarations now name what the minting tasks do; Baserow's
README additionally claimed the administrator owns its endpoint, which it
never did.
The tools declarations follow one convention instead of two: the field records
the deployment's intent and the README names the missing enforcement, which is
what GitLab already did and what Baserow and Mattermost now do.
The discovery lookup carries auth_subject through so clients can see the
subject. No consumer branches on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.
The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.
Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.
The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.
Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two places decided independently what the pre-created <entity> network is called
inside the compose project. render.py emits the key; create.yml asserted it was
always `default`. They diverged in July and nobody noticed, because compose
5.3.1 resolved networks by name and never read the claim.
Compose 5.4.0, published 2026-08-03T14:39:12Z, resolves them by label. For
web-app-seaweedfs - the one deployed role that provides its own shared_net and
therefore renders an unnamed `default` - the mislabelled `seaweedfs` network is
adopted as the project default, `seaweedfs_default` is never created, and the
containers fail to start against it. That failed web-app-nextcloud 0 and 3 in
run 30830377785.
_shared_network_key() is now the single derivation. render_compose_networks
takes its branch from it, so the rendering decision IS the key decision;
shared_network_compose_key() exports the same value, the shared_network_key
lookup hands it to Ansible, and create.yml stamps it: `seaweedfs` for a
self-provider, `default` for everyone else. Both true, both from the source that
renders the file.
Verified on one host with only the compose binary swapped:
5.4.0 network=default -> FAIL, network <project>_default not found
5.4.0 network=<entity> -> OK
5.3.1 network=default -> OK
5.3.1 network=<entity> -> OK
so this is not a fix that trades one version for the other.
test_key_matches_the_key_the_renderer_emits renders the block and asserts the
reported key appears in it, for both role shapes and both modes. The drift that
sat unnoticed for a month now breaks a test.
The move of the attachment computation into utils/networks/attachments.py rides
along because render.py would otherwise be broken between two commits: it is a
pure relocation, re-exported so no importer changes, and it brings the file from
352 to 243 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compose_networks, container_networks and compose_external_networks each rebuilt
the same Ansible-side context before calling their renderer: resolve the
variables, settle the deployment mode, load applications, build the service
registry, and close over the config and database lookups. Around forty lines,
three times, each with its own copy of _resolve_var. A change to how the
registry or the closures are obtained could reach one lookup and miss another.
build_context() returns that context once. honour_mode_force carries the only
real difference between the callers: compose_external_networks never consulted
compose_mode_force, and still does not.
The lookups keep only what is theirs - term validation, their own arguments,
their renderer call - and drop from 94/90/87 lines to 59/54/53.
No assertion changed in the tests; only the mock targets moved with the seam,
from the plugin modules to utils.networks.lookup_context. Twelve legacy comments
in those test files lost their grandfathering when the files entered the lint's
diff scope: two became docstrings naming the call contract, the rest were
narration and are gone.
Behaviour-preserving: the same 13 lookup tests pass before and after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
No role declared it, no client rendered it and no probe read it. The key only
put a "health_path": None into every discovery entry and suggested a capability
that does not exist.
Removing it from the vocabulary, the lookup and the lookup test has to happen
together: the schema lint would keep accepting it, the entry would keep carrying
the None, and the test compares the exact key set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-stk-full passes `application_id` on as a role var holding the Jinja
expression, and the lookup read it raw from the play vars. It then looked up an
application named `{{ sys_stk_full_application_id }}`, found no healthcheck and
blamed the role for not declaring one, which web-svc-logout does.
The container_ports lookup already resolves this, so take its idiom. The
re-trust has to follow the str() coercion: coercing strips the trust tag and
templar.template would silently no-op.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pre-pull task set `async` unconditionally while gating only `poll`. With
async disabled, `poll` became `omit` and ansible fell back to its built-in 15s
poll while `async` still routed the task through async_wrapper: ansible
collected the job itself and then ran `async_status mode=cleanup`, deleting the
job file that tasks/utils/async/reap.yml expects to read. The reap reported a
successful `ollama pull` as a failed async job, failing web-app-nextcloud 2 in
run 30781723102.
Both keys are now switched off together, matching all six peer call sites. On
the resulting synchronous path the reap's `async_reap_tolerate` list no longer
applies, so the tolerated "up to date" answer is carved out in `failed_when`
instead. It keys on `failed` rather than `rc`, so a module-level failure that
carries no rc is not swallowed.
async_failures._which() now honours loop_control.loop_var: the fired result
records the chosen name under `ansible_loop_var`, so a batch failure names
`smollm2:135m` instead of the bare job id.
A second lint case asserts that async and poll carry `omit` together - the rule
this call site was the only one of seven to break.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
container_healthcheck resolves application_id from the play vars
instead of taking it as a positional term. Vars handed over by
include_role arrive unrendered, so under sys-stk-full the lookup got
the literal '{{ sys_stk_full_application_id }}', resolved no app and
refused to render the dashboard compose file.
Template such values before use. container_hostname had the same
defect: it is a Jinja string in every role's vars and was baked
verbatim into the curl Host header. Raise when a value survives
templating unrendered, since ansible-core returns untrusted templates
as-is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.
Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:
healthcheck:
flavor: curl
path: health/ready
start_period: 20m
`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.
Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.
That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.
Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.
Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.
Timing cost is roughly one to two minutes on a deploy measured in hours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a blocker vocabulary so a role whose upstream MCP path this
deployment cannot use says why instead of reporting an absence. Without
it OpenProject reads as "upstream has no MCP", which is false: the server
ships, it is Enterprise-only. A blocker and a served surface are mutually
exclusive, so a role cannot claim both.
Add endpoint.key_credential and endpoint.suffix for servers that address
a session through the URL path. Baserow streams from
{base}/mcp/{key}/sse, and endpoint fields must stay literal, so the
composition belongs in the discovery layer. A server that declares a path
key without a resolvable value is dropped rather than handed to clients
half-built.
Carry the upstream provenance in the role block, so the audit derives it
instead of keeping a second copy.
Require an explicit mutating_tools_enabled: false for a privileged
auth_subject; the rule previously only fired when the key was present and
true, so omitting it slipped an administrator-scoped surface through.
Move the MCP filters into plugins/filter/mcp/ so they stop sharing a name
prefix with their siblings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The consumer count came from roles_with_service, which builds dashboard card
targets and therefore requires a single canonical domain. web-app-matrix
declares domains.canonical as a mapping (synapse, element) and was dropped;
in a deployment where it was the only consumer the count reached zero, the
server started with -volume.max=7, the unnamed default collection took all
seven slots on first write, and every S3 PUT for the matrix collection failed
with "No writable volumes and no free volumes left". Synapse answered 500 on
media upload and no avatar ever reached the bucket. web-app-openwebui carries
the same mapping shape and the same latent failure.
Consumers are now enumerated by the binding itself: a role is a consumer when
its merged config binds the engine and marks it shared, whatever the role is
called and however many domains it serves. The S3 identity file and the volume
budget read that one list, so the identities granted write access and the
slots budgeted for their collections cannot drift apart.
Selection reads the merged applications config rather than meta/services.yml,
because an operator may bind a store through the applications inventory
variable alone; that path is covered by a test so a later optimisation cannot
quietly reintroduce a naming or declaration assumption.
f956de889 made the slot ceiling generous, which removed the accidental brake the old autosizing had provided: while the ceiling was derived from free disk space, growth stopped long before the disk filled. With a fixed ceiling the only thing between growth and ENOSPC is the free-space guard, and its stock 1% is 300 MB on a 30 GB data root - less than the 1024 MB unit SeaweedFS allocates in, so a single growing volume can outrun it.
The floor is now expressed as whole allocation units rather than a percentage, which is meaningless without knowing the disk size. Two units: one for the growth in flight, one for concurrent writers. The size limit is stated alongside it so both come from one constant instead of the floor deriving from a value the image injects - verified that our flag wins, since -master.volumeSizeLimitMB=512 makes the master report 512 MB.
-volume.minFreeSpace replaces the deprecated -volume.minFreeSpacePercent, per the image's own help text. The rendered command was run against chrislusf/seaweedfs:4.39: master, volume server and filer all start.
Left open deliberately: web-app-seaweedfs declares min_storage: 304MB, which cannot hold with a 2 GiB floor. That value feeds the CI storage filter in scripts/meta/resolve/apps.sh, where 60GB is the budget, so nothing is dropped today - but the declaration understates what the role now needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
a3fcd3e7f stopped the entrypoint from autosizing the ceiling out of free disk space, but replaced it with a flat 64 that I adopted without deriving. Each bucket is a SeaweedFS collection and the master grows one in batches of seven - 'create 7 volume' in the artifact, copy_1 = 7 in the image's own master scaffold - so 64 covers nine collections. Twenty-five roles declare the service, and the tenth consumer would have hit 'No writable volumes and no free volumes left' again, just later and without the disk-space link.
The ceiling now comes from volume_slots(): one grow batch per bucket plus one for the default collection, with the consumer count read from roles_with_service. Note what that count is: the consumer gate gets whether seaweedfs is in group_names, not whether the consumer itself is installed, so the value is derived from the repository rather than from the host. It grows with the codebase, which is what the ceiling has to keep up with.
The collection count is a required parameter with no default. A silent fallback would be worse here than a loud failure: without the flag the stock entrypoint injects -volume.max=0 and the autosizing trap is back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role passed no -volume.max, so the stock entrypoint injects -volume.max=0, which means 'derive the ceiling from free disk space'. That derivation charges every non-full volume its full volumeSizeLimit: the master's default grow batch creates seven 1 GB volumes on the first write, reserving ~7.0 GB, and the manager's docker data root had 6.8 GB free. unclaimedSpaces goes non-positive, maxVolumeCount collapses to volumeCount, and the node reports zero free slots.
Those seven volumes belong to the default collection, so they can never serve collection 'nextcloud'. The master says it plainly - 'topo failed to pick 1 from 0 node candidates', 'create 7 volume, created 0' - and every S3 PUT returns 500 while seven empty volumes sit on disk. The volume server never left the topology: RestartCount 0, one unbroken log stream across the window.
That 500 is what starts the web-app-nextcloud 0 chain: occ maintenance:install fails eleven times, leaves a half-installed instance, and slot 1 then holds nextcloud-init-sync.lock for four minutes while every other replica dies on a blocking flock. Reproduced independently in variant 5 with collection 'peertube'; both compose jobs stayed green.
-volume.max makes isArgPassed true, so the entrypoint stops injecting the autosizing value. Slots become a fixed budget rather than a function of disk space; volumes stay sparse and unpreallocated, and -volume.minFreeSpacePercent still guards a genuinely full disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_any_bridge_partner_deployed matched a bridge against deployed role ids by equality or '-<bridge>' suffix. That reaches a partner whose entity carries the name (matrix -> web-app-matrix) but never one registered under a service key: 'sso' is provided by web-app-keycloak and 'ldap' by svc-db-openldap, neither of which ends in the bridge name. Those five declarations therefore evaluated to false in every run, so their addon flags were false everywhere and skipUnlessAddonEnabled skipped the specs in every variant - silently, because a gate that never opens looks exactly like a variant that does not apply.
The lookup now resolves a bridge through build_service_registry_from_applications first and keeps the entity-suffix rule as the fallback, so both spellings work. Existing bridge tests are unaffected: their applications stub yields an empty registry and takes the old path.
The new lint asserts that every bridges: value names a service some meta/services.yml declares. It fails on a typo or an invented name while staying agnostic about which run deploys the service, which is the lookup's question at runtime. Verified both ways: no findings against the real service set, thirty against an empty one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add basic_auth to the MCP auth schema so Jenkins can declare its
HTTP-Basic endpoint through the contract instead of around it.
Introduce the client-side renderers: mcp_authorization builds the
Authorization header per scheme, mcp_tool_server_connections builds the
Open WebUI tool-server entries and drops servers whose scheme it cannot
present, and mcp_renderable_servers filters a server list, since Jinja
select() needs a test and the repo ships no test plugins.
Replace the inline MCP_DISCOVERED_SERVERS Jinja with a mcp_servers
lookup, and carry auth and username per entry so basic_auth consumers can
build their credential.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A module that dies on an unhandled exception carries no msg: the AnsiballZ wrapper returns only failed and exception, whose event holds the sole account of what went wrong. The aggregation defaulted to the literal string package install failed and discarded it, so an AUR fetch that threw in run 30713666732 reported nothing at all and took a full investigation to place. Walk msg, exception.event.msg, exception, module_stderr, stderr instead, and tag every result with the module that produced it. This applies to apt, dnf, pacman and every other module the plugin drives, not only to the AUR.
The aur module fetches the RPC record and the snapshot tarball through bare open_url calls with no error handling, so one blip kills a deploy; distribution repositories need none of this because apt, dnf and pacman retry inside their own transport. Give the calls that reach an endpoint of their own a single shared policy. Only the AUR path is declared today; copr, ppa and build carry it against the day they are.
Split the result along its responsibilities so no file passes 250 lines: calls.py holds the vocabulary, sources.py one builder per acquisition path, plan.py only the dispatcher, outcome.py the reporting and facts.py the fact shapes. The action plugin is left with executing modules. Tests follow the code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve conflicts, resolved by intent rather than by side.
Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.
web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').
handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.
svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.
Six follow-on failures the merge produced, fixed at the root:
- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
the prefix lint forbids; the pair now lives in utils/github/variant/ as
bundles.py and bundle_limits.py, with the -m entry point, the ruff
per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
three bare timeout literals; both route through gotoOnion and resolveTimeout
now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
distros that ship docker without it, the loop device is already detached when
the probe failed before attaching, and modprobe zfs may fail because the
/dev/zfs check decides.
make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ruff`'s B905 wants every `zip()` to say whether it tolerates operands of
different length. Here it never sees one: both operands are the `packed`
form of an address from the same network, so both are four bytes for IPv4
and sixteen for IPv6. `strict=True` states that, and turns a silently
truncated - and therefore too permissive - regex into an error should the
invariant ever break.
The three other files carry `ruff format` reflows the same run produced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Subnet arithmetic lived in Jinja expressions - octet slicing for a JGroups bind
matcher, a regex_replace for a gateway address. Neither is reachable from a
test, and neither says anything useful when it meets a shape it cannot handle.
utils/networks/address.py carries the computation without an Ansible dependency,
so tests/unit reaches it directly.
subnet_address_regex works for any prefix length, aligned or not. A prefix mask
leaves a contiguous run of low bits free, so per octet the addresses span exactly
network_address to broadcast_address, and the cross product of those four ranges
is the network. That makes the octet decomposition exact without a case analysis:
/24 yields 192\.168\.24\.\d+ and /23 yields 192\.168\.(24|25)\.\d+ from the same
three lines.
subnet_gateway returns the address Docker assigns as the gateway, and raises for
a network too small to hold one.
Both lookups follow compose_networks: no positional terms, application_id from
context, networks.local.subnet resolved internally, ValueError turned into an
AnsibleError naming the role and the reason. The lookup form rather than a filter
is deliberate - the caller states its intent and the plugin owns where the value
comes from, instead of repeating the config key at every call site.
Verified exhaustively rather than by sampling: over all 65536 addresses of a /16,
for eight subnets, regex match equals network membership - about 524000
comparisons. Boundaries checked for every prefix length /8 to /32 including the
addresses immediately outside, gateways checked against ipaddress for /8 to /31,
and a cross check that each network's gateway is matched by its own regex.
The regex uses \d+ for a fully free octet, so it is exact over valid IPv4
addresses rather than over arbitrary strings; JGroups only ever matches addresses
of local interfaces. The test module docstring records that boundary.
No consumer at this commit; the call sites follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TOR_CONTAINER_DNS_HOST is empty in swarm mode by design, so no dnsmasq
listener appears on the docker bridge there. The lookup gated on the tor
group alone and wrote 172.17.0.1 into daemon.json anyway, handing every
nested container a resolver nothing answers on — nested image builds then
died resolving dl-cdn.alpinelinux.org.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The endpoint port resolver preferred ports.local and only reached for ports.internal when the block declared exposure: internal. Its single consumer builds a container-network URL (http://<service_key>:<port><path>), where a host-published port is always the wrong one. A server role that omits exposure would therefore have silently received the host port, which is the same class of silent-wrong-value the exposure handling was added to remove.
The order is now internal first, unconditionally, and the exposure argument is gone. All three MCP roles declare exposure: internal, so nothing rendered changes today. exposure stays part of the MCP schema, where it documents who may reach the endpoint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
