Fourteen real conflicts plus the 66 generated READMEs.
utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.
plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.
roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.
group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.
docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.
inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.
roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.
The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role's compose template names its service after the entity, which the rename turned into seaweedfs-console, while meta/volumes.yml still pointed the config mount at a role id that no longer exists and at the service key the engine consumer occupies. A mount whose service matches nothing is dropped without an error, so the console would have come up without its vhost config and still answered the HTTP probe from the nginx default page.
The remaining files are line joins the formatter produced once the shorter symbol name let the calls fit on one line; their syntax trees are identical to the previous revision.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Formatter output only: the parsed YAML and the Python AST of every touched file equal their previous versions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entity name keys a role's instance directory, its compose project and its container name, and was derived only by stripping the longest matching category prefix. Two roles under different categories whose names share a suffix therefore collapse onto one entity and overwrite each other's rendered stack, which no provides: can repair.
A literal entity_name in vars/main.yml now wins over the derivation. web-app-seaweedfs-console takes its shorter name back and declares the entity seaweedfs-console, which frees the seaweedfs key in its own services map for the engine consumer and removes the provides:, covers: and alias entries that stood in for it. A new lint pairs the declaration with the primary key it has to match, because a drift between them would drop the role from the provider set in silence.
Compared across all 288 roles, exactly one entity changes, and it restores what main already keeps on disk, so no instance directory moves.
The function and the filter lose their get_ prefix. Two lints read that prefix as the discriminator between the variable and the filter that fills it; both discriminate on the pipe now. Ansible's templar resolves a filter and a same-named variable independently, verified against ansible-core 2.20.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
required is the install-failure switch, not an enablement switch:
docs/requirements/026-unified-addon-syntax.md:33 and :111 define it that way,
it is implemented only as failed_when guards in web-app-wordpress and
web-app-mattermost, and no variant and no inventory pins it anywhere. ANDing
it into the gate therefore held the flag of every 'required: false' addon at
false in every variant and every mode since 70786aa126 created the suite, so
those Playwright specs reported as skipped with the plausible reason 'addon
disabled or not deployed' while asserting nothing.
Measured against this commit's parent: 32 addon specs declare
'required: false', 24 of them in the 11 roles that call the lookup, out of the
89 addon specs those roles ship. Those 24 move from always-skipped to
executing. The flag reaches no deploy configuration, so the worst case is a red
Playwright job rather than an open surface - the old behaviour was the silent
one.
Both test files stated the old contract and are corrected with it.
Residual: the repairs for several of the specs this wakes - discourse's akismet
assertions and its missing sso gates, moodle's two addons that no install path
covers - are not in this commit. A round over those roles is red until they
land.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
postgis/postgis publishes amd64 only, so every postgres consumer failed
on arm64. The official postgres image serves both architectures and
already ships the contrib extensions bloom, pg_trgm and unaccent.
PostGIS moves into the postgres_libraries registry and is installed from
PGDG (postgresql-18-postgis-3) only for a consumer that declares it as
an extension, which is mobilizon alone. apt update reads every source
list again, because the PostGIS package depends on GDAL from Debian
main, which a PGDG-only update left without an install candidate.
Postgres 18 keeps its cluster in /var/lib/postgresql/18/docker and its
entrypoint treats a volume at the former /var/lib/postgresql/data as an
old database, so the central and the dedicated data volume both mount at
/var/lib/postgresql. Existing 17 clusters are not migrated: their data
directory trips the entrypoint's old-data check and the container
refuses to start.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f78ebe46a4)
7b5833b37a assembled it in zammad's env template: urlencode both halves of the
credential by hand, patch them into the URL the lookup had already returned,
then append bucket and query. Three set-statements of string surgery on a value
the lookup builds from host and port in the first place.
The lookup now exposes credentialed_url next to url, built from the same
endpoint it already has, with urllib.parse.quote(safe='') on each half. The
template keeps only what is actually zammad's: the bucket path and the region /
force_path_style query.
This also removes the manual scheme concatenation that
test_onion_url_central_lookup refuses. That rule guards against a hand-built
app URL repeating the clearnet-vs-onion decision; the template was not doing
that, but it did name '://' and could therefore touch the scheme. Now it
cannot.
quote(x, safe='') is equivalent to the old
chain: checked over slashes, plus, equals, percent and space, no divergence.
Both branches of the lookup carry the key, the disabled one as the empty
string, so a consumer cannot hit a missing key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three failures of the same deploy, each uncovered by fixing the one before it.
web-app-seaweedfs-console read its two vhost domains through
lookup('config', ..., 'domains.canonical.*'), which resolves against the raw
applications tree. The onion substitution happens elsewhere -- in
get_merged_domains, where svc-net-tor's exclusive flag REPLACES the clearnet
value. On an onion node the role therefore rendered vhosts for clearnet names
while Keycloak's redirectUris, the CSP header and LOGOUT_DOMAINS all carried
the onion names, so the Filer and Master surfaces had no vhost at all and the
default server answered. Reading through lookup('domains', app).<key> is the
shape matrix and bluesky already use for named canonicals.
With the onion names in place the console's own nginx refused to start:
a server_name of ~81 characters overflows the stock bucket size of 64 and
nginx aborts with 'could not build server_names_hash'. The central openresty
has carried server_names_hash_bucket_size 128 since forever
(sys-svc-webserver-core/templates/nginx.conf.j2); the second nginx needs the
same.
With both surfaces served, an administrator still had to re-authenticate on
the Master host: cookie_domains was emitted only when the app had clearnet
domains, so an onion-exclusive node got no cookie scope and oauth2-proxy fell
back to a host-only cookie. whitelist_domains on the very next line already
handled both address families; cookie_domains now does too.
Both lists move out of the template into lookups of their own. The template
held six set-statements and two nested inline conditionals to build two JSON
arrays; the lookups take an application_id and return the list, which also
makes the behaviour testable without rendering the template. The template
structure assertion in test_cookie_scope.py went with them -- it measured
Jinja text, not behaviour -- replaced by thirteen cases over the two lookups:
single host, multi-host collapse, onion-only, dual-stack, no domains, no tor
node, and a missing DOMAIN_PRIMARY that now raises instead of silently
scoping nothing.
Evidence: make compose-playwright role=web-app-seaweedfs-console goes from
3 failed to 3 passed against the live stack. For the single-domain case the
change newly affects, web-app-prometheus keeps all four of its SSO specs
green with the scope applied.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-seaweedfs becomes two roles. web-svc-seaweedfs keeps the storage
engine: the container, the seaweedfs_data volume, the per-consumer S3
identities and the single canonical domain api. web-app-seaweedfs-console
takes the filer and master web UIs into its own compose stack behind the
admin-only oauth2 proxy, reaching the engine over the shared object-store
network. A node can now run the store without the consoles.
The console needs a distinct role name because get_entity_name strips the
category prefix: both halves would have resolved to the entity seaweedfs and
collided on the compose project, the instance directory and the container
name.
The filer and master ports move into the engine's services.seaweedfs.ports
.internal as their single owner; 01_init_consumer.yml consumes the master
port and would otherwise have resolved it only by role-var leakage from the
console, breaking bucket provisioning on exactly the console-less node the
split makes possible.
The engine's api vhost gains the prometheus locations. It never had them: the
override satisfied the cross-cutting-include lint only because the strings sat
in the SSO branch that the api domain never took.
Consumers, the objstore lookup and the catalogs follow the role id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`roles/user/meta/users.yml` reserved two names against the identity directory by
indexing the primary domain from the left: `sld` took label 0 and `tld` took
label 1. That is only the SLD and the TLD when the domain has exactly two
labels. On `label-a.tld.test` it reserved `label-a` and `tld` and let the real
TLD through, and on any deeper domain it drifts further.
The count is a runtime fact, so a static YAML file cannot express it. The two
entries are gone and `utils/cache/users` now emits one `domain_label_<n>` per
label:
infinito.nexus -> infinito, nexus (unchanged)
main.infinito.test -> main, infinito, test (was: main, infinito)
label-a.tld.test -> label-a, tld, test (was: label-a, tld)
Nothing read the `sld` or `tld` keys: a grep over roles, plugins, utils and cli
finds only the generator itself. The entries exist to carry `accounts: []` into
`reserved_usernames`, which `web-app-keycloak/tasks/02_init.yml:6` joins into the
username deny pattern, so the shape of the keys is free and only the set of
usernames matters.
Worth stating so the next reader does not overestimate it: the pattern at
`UserProfileProvider.json.j2:31` allows `[a-z0-9]+`, so a label carrying a hyphen
could never have been registered anyway. What this actually wins is the third
label. What it mainly wins is that the rule no longer has to be revisited when
the domain depth changes.
The `KEYCLOAK_DOMAIN` and `domain` fallbacks go with it. Both ran through a
helper that recovered the primary domain by keeping the last two labels of a
host like `auth.<DOMAIN_PRIMARY>`, which cannot work without already knowing the
depth it is trying to find. `DOMAIN_PRIMARY` and `SYSTEM_EMAIL_DOMAIN` remain,
and every deploy path supplies one of them. The test that pinned the removed
fallback is inverted rather than deleted, so the removal stays visible.
The rest is the refactor pass the change pulls in. The function no longer
materialises aliases and is renamed to `_materialize_domain_label_reservations`;
its candidate tuple carried one extractor after the fallbacks left and is a plain
key loop now; an `or [primary_domain]` guard that would have reserved a nonsense
username for an all-dots domain is dropped. `roles/user/vars/main.yml` loses four
comments that restated the line below them, and the README documents the
reservation registry, which no longer shows its full content on disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.
So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.
`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.
Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.
`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.
Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.
The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.
It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.
`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.
`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.
The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.
Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both roles mount their model cache as a `type: bind` volume, and swarm
rejects a task whose bind source is missing on the node it lands on. Neither
declares a `placement`, so the scheduler may put the container on any node,
while the directory was created by a single task that reached one host. CI
showed the consequence directly:
s1_s1.1 desired=Ready current=Rejected error="invalid mount config for
type "bind": bind source path does not exist:
/var/cache/infinito/core/cache/models/s1"
with the creating task logging exactly one result line, for the manager.
`nfs_prep.yml` already fans directories out across the cluster but gates on
`driver_opts is defined`, which a bind volume never carries, so these two
volumes were never covered by it.
The host list moves into a `node_hosts` lookup rather than an inline
expression, because two roles already spelled the same idea by hand and a
third and fourth were about to. It answers one question - the hosts a
node-local path must exist on - and keeps the swarm group name in one place
per plugin. The call sites use `query()`, since `lookup()` joins several
results into one string and a loop needs the list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lookup takes the role from its term and falls back to reading
application_id out of the play's variables. That read was raw, which is the
trap that took down three jobs from webserver_resource, so it now goes through
resolve_var like every other lookup.
Prevention rather than a fix. The fallback is the else of a term that the only
caller always passes, and where it would fire web-app-docs sets application_id
literally; the roles carrying a templated id reach it through sys-stk-full,
which web-app-docs does not. The consequence would also have differed: an
unrendered string never matches a peer, so the replay would have asked the
site for a role named after the template and failed on the page rather than in
the lookup.
Measured: with the resolver's body stripped back to the raw read this test and
the webserver_resource one both fail, and both pass with it restored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moving the effective-cpus expression into a lookup changed where the role id
is read from. Inside a Jinja expression the templar resolves a variable
recursively, so a value that is itself a template arrives fully rendered.
Reading it out of the play's variables does not. sys-stk-full sets
application_id from sys_stk_full_application_id, so the lookup asked the
applications dict for a role literally named
{{ sys_stk_full_application_id }}
and three jobs of one run died on it: dsk-gnt-claude, svc-ai-agent-broker and
svc-ai-searxng, every one of them a role that goes through that stack.
The lookup now templates application_id and service_name when they still
carry a template, and returns them untouched otherwise, so the ordinary role
never reaches the templar at all.
Measured: with the resolution removed the new test fails with the run's
message character for character, path_trace and config_path included, and
passes with it restored.
The same raw read survives as a fallback in guide_role, unreachable while its
only caller passes the id as a term.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A container cap had to be an absolute number, and docker refuses one above
the host's core count, so "use this machine" could only be written as a
number that breaks on a smaller machine. cpus now also takes a percentage,
read against the host's raw count: 100% is every CPU and lands exactly on the
ceiling docker accepts, 50% is half of it.
The share is read against that raw count rather than against the fair-share
default, because the two CPUs the resource lookup reserves exist to bound
what a service gets when it declares nothing. A service writing a percentage
overrides that default on purpose, and subtracting the reserve a second time
would count it twice and leave no way to name the whole machine.
A bare cpus: 0 is refused from here on. Docker reads a zero as uncapped
rather than as none, so the value most likely to be typed by mistake was also
the one that removed the limit; the error names the fix. No role declared
one: all 315 cpus values in the tree are plain numbers, the smallest 0.01.
Two lookups carry the resolution so that no caller repeats it or has to know
what a percentage is read against. lookup('cpus', app, service) replaces six
copies of lookup('config', ...) piped through the resolver; those six size
worker pools and thread counts from their own cpus, and left alone a
percentage would have given libretranslate a single thread and handed the
agent broker an uncapped container. lookup('webserver_resource', key)
replaces the two blocks that took the smaller of the proxy's value and the
app's, which were the same eight lines written twice for cpus and for
pids_limit.
resolve_cpus is no longer registered as a jinja filter. Nothing referenced it
from a template once the lookups existed, and keeping it would have left two
public ways to resolve one value.
Measured: 96 tests across the filter, both lookups and the container render
templates, including that a percentage resolves before the smaller of the two
is chosen, and that pids_limit stays an integer because nginx renders it
straight into worker_connections.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instructions test brings up a machine per role, so it can only ever
replay one guide. Which one it picked was a side effect of ordering: the
lookup returned the first invokable peer the round listed, and on a CI row
that is the role under test only because the row's apps happen to read
<role>,web-app-docs.
It now reads APP_ID, which the deploy workflow already sets to the row's app
and which the swarm routine already documents as the app under test. A round
that carries no APP_ID, or one whose app publishes no Production block, draws
an invokable peer at random instead, so a local round covers a different
guide each time rather than always the first one the inventory lists.
Ansible runs inside the infinito container rather than on the runner, and
that exec passes an explicit allowlist, so the name arrives only where it is
forwarded. Two seams carry it: _FORWARDED_ENV_KEYS, which covers CI and the
local initialize and reinstall verbs, and the update verb's own docker exec,
which assembles its arguments without that CLI.
The pin is tolerant rather than strict. APP_ID is set on every deploy row and
not only on documented ones, so refusing an app that publishes no guide would
let a generic variable abort a deploy.
Measured: the draw is observed over 200 rounds rather than assumed, and the
forwarding is negative-controlled, removing the key and watching the new test
fail on that key alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b64e4a0558 gave web-svc-mirror the shared monitoring include so it would
appear in the scrape set. That include ends in two locations proxying to
the role's own backend, and this role runs no container: it renders a
vhost and nothing else. Every deploy that reached its vhost then died in
resolve_upstream, twenty-one jobs of one CI run, in both the compose and
the swarm phrasing of the same message.
What followed was worse than the original gap. 514a1fe74b guarded the
probe inside healthz.conf.j2, which ninety-six roles render, so a
one-role want became a change to shared infrastructure. Three further
attempts sharpened that guard, none of them fixed the deploy, and the
renders I offered as proof were vacuous: the outer integration guard
short-circuited in the context I tested, so the block I was measuring had
never rendered.
main does not carry the include, cannot reach that code path, and does
not fail. The complaint that started this, that mirror carries no
app="web-svc-mirror" label, does not appear in the run preceding the
include either. It was not a regression and not urgent, and it did not
justify touching a template every monitored vhost depends on.
So the tree returns to main for both files and the lookup written to
support them is deleted. Mirror is unscraped again, which is how it has
always been, and now a decision to take deliberately rather than a defect
invented from a passing remark in a job log.
Measured: both templates are byte-identical to main, nothing references
the deleted lookup, and the ansible lint suite passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The monitoring block a vhost includes ends in two locations that proxy to
the application's own backend, and resolve_upstream raises rather than
returning empty when no port exists to build that upstream from. A role
that renders a vhost without running a container of its own has none, so
including the block aborted its play. b64e4a0558 made web-svc-mirror the
first such role, four months after the assumption was written down.
The probe halves are now conditional. /healthz/live stays: it answers from
openresty, which is what liveness means for a role whose only artifact is
a vhost. /healthz/ready and /_live_probe go, because both proxy to a
backend that is not there. Guarding only the second one left the first
referencing a variable the guard no longer set, which is its own failure
and the reason this took two passes.
The condition asks a lookup rather than rebuilding the rule in the
template. Hand-derived, it tested the local port alone, which is correct
outside swarm and wrong inside it, where the upstream comes from the
internal port; web-app-matrix declares only that one and would have lost
its probe silently. The lookup reads both scopes and the port a role
resolved for itself, which is what resolve_upstream consumes.
No working endpoint is removed. A vhost rendering the block today must
already carry a port, or the lookup would be raising for it as well, so
the condition can only skip what could never have rendered. Not yet
rendered: the deploy that proves the template resolves has not run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The entry carried two fields naming the same host by two routes.
canonical_url came from the tls lookup, which substitutes a role's onion
address on a Tor deployment; canonical_domain came straight from
domains.canonical, which does not. On clearnet the two agree and nobody
notices. Under Tor they part ways, and the field a consumer compares
against a rendered page is the one that stayed clearnet.
Every dashboard tile assertion failed on an onion swarm job for that
reason: the page offered
http://prometheus.toxlaup2sbprl6zklgcsofj7g4xim6uxjd4zjiiqtk3qzffykqwda2qd.onion
and the locator looked for a[href*="prometheus.main.infinito.test"]. The
matomo tracker spec failed the same way, comparing registered site URLs
against clearnet hosts. Deriving the domain from the URL leaves one
source instead of two that only agree off Tor.
The bug shipped with 6fa7bd2f57 in May, when canonical_url was added
beside the existing field without reconciling how each is resolved. It
stayed unreachable on swarm because the dashboard container failed to
start there at all, until 0e4ed57c3c let it start and the tiles rendered.
The stubbed tls in the tests answered with a host that no clearnet deploy
can produce, which is why the old expectations looked right. It now
answers with the role's own domain through the module's resolver, and
takes url_host to model a deployment whose URL host differs.
Consumers are the five playwright specs that read these fields; no task
reads them. Prometheus keeps matching on the role id, so its needle is
unaffected. css and logout carried the same latent failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff reflows the guide_role signature and its two call sites, the same for
three test modules, and the YAML formatter prepends the document marker to
the machine's compose file. These files were created one commit too late to
be caught by the previous formatting pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The replay of a role's "### Production" block moves out of the CI guide
scripts and into the documentation role itself. The block is read off the
rendered page over HTTP, so what runs is what a reader copies, and the role
that publishes the page is also the role that proves it.
scripts/github/guide/{one,compose_deploy,host_deploy}.sh and
cli/build/docs/readme/production.py are gone with the workflow step that
called them. The instructions marker now only adds web-app-docs to the row's
apps; test-e2e-cli discovers the role through its new templates/test.env.j2
and runs files/test/test.sh.
The block runs in a throwaway machine, never on the stack host. compose.yml
starts one service built from files/test/Dockerfile, whose base is
INFINITO_PARENT_IMAGE out of the mounted checkout's .env, so the machine is
the stack host's own distribution and the replay exercises the installation
instead of a prebuilt image. files/test/init.sh adds an init when the base
carries none and keeps the full -Syu that ef0f03ea98 established for arch.
scripts/tests/workspace/base/01_install.sh then prepares the machine before
the block runs.
Nothing is skipped any more. The previous host-install branch exited 0 and
passed silently over 65 of the 178 invokable roles; a block without a
"docker run" is now replayed against systemd in that same machine, with the
git clone dropped in favour of the mounted tree under test.
The target role comes from the new guide_role lookup, which reads
list_invokable_app_ids() rather than matching role-name prefixes, and falls
back to the asking role when the round deploys nothing else. deployment.py
grows running_apps() so both lookups share one definition of what the round
deploys. required_by_coverage.role_is_invokable drops its own walk over
categories.yml and delegates to the same SPOT; measured equal across all 280
roles.
Carried in the same index, because the refactor pass ran over this change
set: library.py splits into sites.py and build_queue.py mixins, builder.py
hands translated_languages to catalogs.py and folds three repeated log-tail
and state writes, and placeholders.py becomes a facade over spans.py,
repairs.py and damage.py. No test changed for those splits, and the two
ref-resolution dicts builder.py carried are replaced by _wanted_ref, which
also stops hashing the deployed tree on every tag build.
Verified: 7132 unit and lint tests pass over the whole tree, the guide_role
lookup renders through a real Templar, the base image resolves from .env,
and compose config validates the service. The replay itself has never been
executed; that needs a deploy of web-app-docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DEPLOYMENT_MODE was compared against a quoted literal 219 times across 110
files. IS_COMPOSE_MODE and IS_SWARM_MODE say the same thing once, and
test_deployment_mode_booleans keeps it that way: only the file that defines
them may still compare.
Both are real booleans under this ansible, measured before the swap because
the alternative is silent: a variable that templates to the string "False" is
truthy in Jinja, which is why IS_STACK_HOST is written `| bool` at every use.
These need no filter.
What had to learn the names:
* utils/annotations/task_gate.py, which several lints ask whether a task is
compose-only gated;
* four compose-template lints that recognise their gate by matching the
expression;
* the fixtures that stand in for the play's variables. They derive the
booleans from the mode now instead of carrying them, or a test that
overrides DEPLOYMENT_MODE keeps the old pair and asserts the wrong
branch.
Each replacement was checked rather than trusted: the 206 conditions that
changed in place were evaluated old against new through the Templar under
compose, swarm and host, with no disagreement, and no changed line turned out
to be anything but a condition swap. `!= 'swarm'` became `not IS_SWARM_MODE`
rather than the shorter IS_COMPOSE_MODE, because only the literal translation
stays correct should the mode ever be host.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-svc-compose writes compose.ca.override.yml for exactly lookup('ca_injected', application_id), which since 1254e27c2d also answers true for a service that declares ca_client and only calls the self-signed endpoint. compose_file_args rebuilt that predicate instead of asking it, and its copy still demanded a domain, so for a domainless ca_client the file was written and never reached --compose-file. Swarm takes its file list from that lookup; compose finds the file on disk by itself, which is why only the swarm jobs of run 36064589578 failed while the compose job passed.
The broker's container in that run carries no SSL_CERT_FILE and no CA mount, while openwebui, same job and same node, carries seven CA env vars and four binds. Every outbound call to Keycloak then fails the handshake, which the CLI test reports as 503 and Open WebUI relays as HTTP 400.
Two corrections in ca_injected itself. Its templar.template() call was dead: as_str strips the TrustedAsTemplate tag and ansible-core 2.19+ refuses to render an untrusted string, so the line could not resolve what it advertised, and tls.py and cert.py never template TLS_MODE either. And the ca_client branch short-circuited on the global TLS_MODE, skipping resolve_mode's per-app server.tls.mode override; it now applies only where there is no domain and so no per-app answer to get.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every desk- role becomes dsk-, and the three coding agents move to dsk-gnt- under a category of their own. The category carries a title, hanzi and icon but no invokable flag: an invokable child under an invokable parent is rejected outright, which is also why web itself is not invokable.
Three things a search and replace would have broken without a sound. get_entity_name now strips dsk-gnt-, so the primary entity keys agnt-claude, agnt-cursor and agnt-pi no longer resolved and all three roles dropped out of the CI plan. Eighteen run_once guards carried the old name against a convention its own test enforces. DESKTOP_ROLE_PREFIX in the addon spec lint is a literal, so the desktop exemption it grants would have matched nothing and the lint would have moved to the wrong rule in silence.
docs/requirements/035 is not a historical document but a register a lint holds to every role with an application_id, so its list carries the new ids. In 026 only the links move; the prose stays as a statement about the tree at the time it was written.
Measured against the plan with the rename normalised, the deployed role#variant set is unchanged. A stale id in an inventory is caught before any deploy starts: the pre-flight validator reports Invalid group 'desk-firefox' (not in application_ids).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ca_injected asked three questions, the first being whether the application
has a domain. That decides whether it is SERVED over the self-signed CA. An
application that only CALLS one has no domain of its own, so the predicate
answered false, sys-svc-compose-ca skipped its stack, and the container ran
with an empty trust store.
svc-ai-agent-broker is such a caller: it authorizes every request against
Keycloak group paths over the public HTTPS base. Without the CA every
outbound call died in the TLS handshake, the broker answered 503 "access
check unavailable", Open WebUI turned that into HTTP 400, and the spec
failed on a model it had just listed. Its container was the only one in the
rescue artifact without SSL_CERT_FILE, while litellm and open-webui carried
it.
A service may now declare ca_client: true, and the predicate then answers on
the deployment's TLS mode alone, because the domain question is meaningless
for a client. The three served-side conditions are untouched, so no
application that already received the CA stops receiving it.
The alternative of pointing the broker at an internal Keycloak URL was
rejected: the role declares no keycloak service, so the URL would render
without a route.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_passthrough_swarm quoted and escaped any value containing whitespace,
which contradicts the contract stated at the top of the same module:
docker stack deploy does not strip outer quotes, so the value reaches the
container with them.
Ansible's to_json emits the default ", " separators, so AGENT_PLATFORMS
always carried spaces and arrived as "{\"hermes\": ...}". json.loads then
returned a str rather than a mapping, and sorted(PLATFORMS) in the broker
iterated its characters: the broker advertised one model per character and
never hermes or openclaw, which is why Open WebUI's model list held neither
and the agent-broker specs failed before reaching a completion. The same
encoding mangled AGENT_CONSTRAINT, whose placement expression would have
failed once an agent was reachable.
The branch existed for the opposite consumer: test.env is sourced by bash,
where KEY=one two leaves the variable unset under set -u. Quoting is the
wrong instrument for that, because one filter cannot serve both contracts
keyed on DEPLOYMENT_MODE alone. Both values that join on a space now join
on a comma and their consumers split on one, so no rendered env value needs
quoting to survive the shell.
Two unit tests asserted the quoting while a sibling in the same class
asserted that swarm does not escape a double quote; the pair could not both
hold. They are replaced by one test that renders real JSON and checks the
container receives a mapping. The lint that flags a space-joined value no
longer accepts dotenv_quote as the remedy, since in swarm it no longer
quotes, and its message names the comma instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stage looped over every role of its group and skipped the ones this host
does not deploy, one task at a time, plus an unconditional handler flush behind
each. Measured on a one-app deploy on 2026-09-23: 132 of 139 entries skipped,
278 task deltas, 41.8 seconds of nothing.
The lookup now takes the `groups` magic variable and drops the entries no host
in the inventory carries, so they never reach the loop.
Which roles get deployed does not change, and that is provable rather than
hoped for: the body deploys a role exactly when application_allowed holds,
which requires the app to be in that host's group_names; if it is, that host is
in groups[app], so groups[app] is non-empty and the lookup keeps the entry. The
filter is therefore strictly weaker than the condition it precedes and can drop
nothing the body would have run.
The filter deliberately reads `groups` and not `group_names`. 8aea5a9b35
measured that a per-host condition above the loop makes ansible insert the
included blocks in host order instead of loop order, after which
svc-storage-nfs-client mounted an export svc-storage-nfs-server had not
deployed yet and the play died on 'mount.nfs4: Connection refused'
(run 31645153977). `groups` is the same mapping on every host, so every host
still walks one identical list, and tasks/utils/setup/role.yml keeps its `when`
exactly where that commit put it. Two of the new cases pin this: a role only
one host carries still enters the loop, and a group_names passed to the lookup
changes nothing.
First written the other way, with group_names in the lookup and the `when`
removed. A single-host deploy cannot show that defect, and it came back green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy refused at ok=771 failed=1 with 'services.litellm.remote_models names provider(s) mock; known providers are openai, anthropic, openrouter'. Declaring the mock among the remote models, which is what makes it an ordinary model rather than a config key of its own, runs into a guard that demands a key slot for every provider. A mock needs none: it answers from its own declared response.
The guard keeps its teeth for real providers and now names the exception and its reason, instead of tolerating it silently.
AI_MOCK_PROVIDER holds the name once. The guard, AI_REMOTE_ALIASES and AI_MOCK_ALIASES read it, and the template hands it to the route builder, which lost its own constant. Two literals spelling the same thing is how the LM Studio branch came to drop the context window its README still advertised; a parameter cannot drift.
Static checking never touched this line. It took a real deploy, 772 steps in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The proxy config grew a loop per model source, so the rule for how an alias is called lived in four places - and they had drifted: the LM Studio branch published no model_info.max_input_tokens although the role's README says a model entry carrying context publishes exactly that. Serving anything new meant a fifth branch.
The sources now meet in one filter that returns {alias, params, context}, and the template renders a single loop. Every value goes through to_json, so a response carrying a colon, a quote or a newline cannot break the YAML it is written into.
Publication order is a contract, not an accident of how the loops were stacked: LITELLM_CHAT_MODEL takes the first alias, so a locally served model has to outrank a declared one. That is now stated in the module and asserted, instead of being implied.
The gateway lint had to widen with it. Its canary checks that some gateway template still matches the provider-URL pattern, because without one the rule would pass vacuously on every consumer; the pattern demanded {{ … }} while the template now sets the URL in {% set … %}. Widening it to {{ or {% makes the rule stricter, since a provider URL in a set block is just as direct an address and would have slipped through anywhere else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gui, mig and the portfolio read core's role data and translations from one
place instead of cloning core themselves. web-svc-api serves them at
api.<domain> as a FastAPI service.
The container keeps a bare git mirror of core and, with forks: auto, of
every fork the GitHub forks API lists, fetched every fetch_interval seconds
under a file lock. The deployed working tree is committed as the ref
`deployed` from a snapshot baked into the image with fixed dates, so every
replica commits the same SHA. A ref is `deployed`, a SHA or prefix, a
branch or tag, or `owner:name` for a fork.
Endpoints: /v1/repositories, /v1/log, /v1/roles, /v1/roles/<role>,
/v1/categories, /v1/bundles, /v1/tree, /v1/file, /v1/todos, /v1/languages
and /v1/catalogs/core/<code>. Descriptions and category titles follow `lang`
or Accept-Language and the response names the language it used. Every
response carries Access-Control-Allow-Origin: *, an invalid ref or path is
a 400 and an unknown one a 404.
A role that enables services.api gets the API origin in its connect-src.
The Playwright spec covers every endpoint against the live stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A version bump that only changes a build argument never reached a running
container. In compose mode a changed compose.yml only notified compose-up,
and `compose up` reused the existing image under the unchanged tag. In swarm
mode registry_sync skipped every push whose tag already existed in the
registry, so a rebuilt image never reached the nodes.
Compose mode now reads the rendered compose file back and records the
docker_build_digest of its build sections (context, dockerfile, args) in
.build-inputs; only a change of that digest notifies compose-build, so a
port or env change still costs no build. registry_sync records the local
image ID it pushed for every locally built image in .pushed-images.json and
skips a push only when the tag is in the registry and the ID is unchanged.
External images keep the plain registry skip.
Comparing against the registry manifest instead would depend on the image
store: with the containerd store or BuildKit attestations the local ID is an
index digest, not the config digest the manifest carries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles_with_service drops every role without a canonical domain, which
suits consumers that need a URL but made LiteLLM skip headless
consumers: svc-ai-agent-broker never got its key registered and every
relayed call came back 401 "Unable to find token".
The lookup gains include_headless, which keeps such roles with an empty
canonical_domain and canonical_url, and LiteLLM's consumer key list
passes it. Callers that do not pass it behave as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databases.csv rows come only from consumers, through sys-svc-rdbms. A
database provider deployed without one, like the lean svc-db-postgres#1
variant (mirror, tor, backup and postgres, nothing else), therefore has no
row. baudolo copies its volume live and records it undumped, and the drill
failed with "postgres_data (postgres) ... no sql dump" in fork run
35731501845, on a cluster that holds only what the engine creates itself.
The new database_consumerless_volumes lookup names the data volume of every
deployed provider that no deployed role consumes as a shared database; a
dedicated database brings its own volume and does not count. test.env hands
the list to verify/backup.sh, which notes those volumes instead of failing.
Every other undumped database volume still fails, above all one whose
consumer forgot to seed its row, since that consumer is deployed.
Only the drill's verdict changes; what gets backed up does not, and the CLI
tests run on dev, act and github runtimes only. The one case it now lets
pass is a dev host still carrying the database of an app its inventory no
longer lists: that data sits undumped in the provider volume and reads as
NOTE instead of FAIL. A CI job always starts on a fresh host.
Counting the databases in the copied data directory was rejected: the
bootstrap set depends on the image, and postgis/postgis adds
template_postgis to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
postgis/postgis publishes amd64 only, so every postgres consumer failed
on arm64. The official postgres image serves both architectures and
already ships the contrib extensions bloom, pg_trgm and unaccent.
PostGIS moves into the postgres_libraries registry and is installed from
PGDG (postgresql-18-postgis-3) only for a consumer that declares it as
an extension, which is mobilizon alone. apt update reads every source
list again, because the PostGIS package depends on GDAL from Debian
main, which a PGDG-only update left without an install candidate.
Postgres 18 keeps its cluster in /var/lib/postgresql/18/docker and its
entrypoint treats a volume at the former /var/lib/postgresql/data as an
old database, so the central and the dedicated data volume both mount at
/var/lib/postgresql. Existing 17 clusters are not migrated: their data
directory trips the entrypoint's old-data check and the container
refuses to start.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stubs native_metrics_apps and asserts the intersection, including the case the
refactor exists for: a role that ships a fragment and is deployed but is not
scraped must not reach the rule file.
Brings in Kevin's AI-agent work, the svc-net-firewall role and the update
tooling. One conflict, in the MCP proxy audit: both sides appended a role to
privileged_control_plane, ours web-app-openbao and upstream svc-net-firewall.
Taken as a union at 25, which is what test_mcp_audit_completeness requires -
neither role declares meta/mcp.yml, so the audit is the only place they are
classified.
Nothing here was typed by hand. The docs generator rebuilt the MCP audit
table against pins that had already landed, so fider reads v0.36.0 instead
of stable, jenkins 2.568.3-lts instead of lts, pretix 2026.7.0 instead of
stable, and flowise 24-slim instead of 24-bookworm-slim. That is the table
catching up with da665ab69 and 10c5b35e8, reading the same declaration the
updater reads.
Autoformat wrapped three test files from the same series: one condition
pulled onto a single line, three assertEqual calls broken across lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mirror proxies /<origin>/<path> to https://<origin>/<path> and relayed the upstream Location header untouched. A root-relative Location then resolved against the mirror instead of the origin, the prefix that names the origin was lost, the follow-up request matched no location block and fell into the catch-all 404. Chromium discards a 404 served for a script request, so the page lost the asset with net::ERR_BLOCKED_BY_ORB.
MIG job 105068912392 shows both halves, 28 times each: GET /unpkg.com/3d-force-graph/dist/3d-force-graph.min.js answered 302, then GET /3d-force-graph@1.80.0/dist/3d-force-graph.min.js answered 404. unpkg redirects an unversioned path to the resolved version and expresses the target as an absolute path.
The forms move into a redirect_sources filter so the template states the intent once and the list is unit-tested. Measured against a two-container nginx fixture: an absolute https Location, an absolute http one and a root-relative one all come back as /<origin>/target. A Location naming a third host stays untouched, which is unchanged behaviour and still leaks past the mirror.
A network-path form was written first and then dropped: nginx normalises such a target in return, and routing one through add_header emits two Location headers of which the proxy relays only the first, so the case could not be produced and the rule stayed unproven.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
