resolve_term follows a redirect source to its target so an alias inherits the TLS settings of the application it points at. A redirect to a domain another host serves ends that chain on a domain this host does not know, and the tls and cert lookups failed: galaxyserver full-1 stopped in web-opt-rdr-domains on 'tls: domain project.infinito.nexus not found' for its project.veen.world -> project.infinito.nexus redirect.
When the chain ends outside the host's domains and web-opt-rdr-domains is deployed, the source now resolves to web-opt-rdr-domains with the source itself as primary domain, which is the role that serves its vhost and whose certificate bundle names it. Only the outermost call falls back, so the source asked for stays the primary domain across a longer chain; without the redirect role, or for a target this host serves, nothing changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The README template wrote COUNTRY=<your-country> unquoted since 558784e6ca. The web-app-docs CLI test checks the published Production commands with bash -n before it fills the placeholders, and bash reads <your-country> as a redirection: 'syntax error near unexpected token newline' for ERPNext's page (ERPNext#2, run 40131272595). It is quoted now like every other placeholder; the later sed substitution still matches inside the quotes.
test_generate also runs bash -n over every bash block of the generated Production section, so the next unquoted placeholder fails as a unit test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One string that every engine refused or mangled raised out of the batch route's gather, so the gateway answered the whole batch with a 422 and the client lost the other nineteen translations. Gateway.translate_each now owns the bounded fan-out; with partial=true it answers (None, None) for a string that ran into a GatewayError and keeps the rest. Non-gateway exceptions still raise, and a NoBackendError on every string still fails the batch as the outage it is.
The i18n client asks for partial batches and counts a null answer as a refused entry with its reason, so the re-ask and damage checks continue per string. A caller that does not send the flag gets the old behaviour.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PROTECTED masked identifiers with an underscore and bare numbers, so S3, OAuth2, POP3 and HTML5 went to the engines as a word plus a separately protected digit. Engines tore them ('Amazon S ilə uyğun ...' for 'S3-compatible') or glued the digit onto another word ('Distribuid3-compatible'); the masking audit over core found 159 such entries.
A letter-digit token is now one protected span, ahead of the bare-number alternative. The client masks it whole, and damage.reason() condemns a translation that lost or moved it, so prune and retranslation pick those entries up.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reconcile task retried every failure thirty times ten seconds apart, so a permission the key lacks surfaced only after five minutes: rdr-3 spent 331 s on a 403 that no retry could change. The reconciler now ends with exit code 3 when Flowise answers 401 or 403, and the task stops retrying on that code while its failed_when still fails the run under MCP_RECONCILE_STRICT.
Only 401 and 403 count as final: authorize may answer a provider that is still starting with another 4xx, and that wait is what the retries are for. Every exit that reports an HTTP status goes through fail(), which keeps the message on the exception, so the existing tests read it unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mcp_servers narrowed the MCP providers to application_closure(deployment.whitelist), which on a persistent host drops every provider an earlier run deployed and a partial run does not touch. sys-svc-mcp-reconcile then converges each client present on the host against that narrowed set: on echoserver the --id web-opt-rdr-domains run rdr-3 removed the Prometheus registration from Flowise.
deployment now reports ephemeral (runtime act or github), and the closure narrows only there, where a whitelist is all a run deploys; a persistent host narrows nothing, as a full deploy already did with its empty whitelist, so its clients keep every provider it serves. CI keeps its narrowing unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_mint_mcp_api_key.py computed PROJECT_ROOT itself from Path(__file__).parents[6], which test_project_root_import forbids outside an __init__.py. The files package now exports PROJECT_ROOT the way its reconcile subpackage does, and the test imports it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The reconciler prunes the managed fixture chatflows of providers it no longer probes, but the scoped API key it gets was minted without chatflows:delete, so the first prune ended in 403 Forbidden after thirty retries; on echoserver rdr-3 stopped on 'FAILED deleting stale infinito:fixture:web-app-prometheus'. Adding the permission alone would not have reached that host, because api_key() reused an existing key unchanged.
KEY_PERMISSIONS now carries chatflows:delete, and api_key() reuses the managed key only while its permissions equal KEY_PERMISSIONS; otherwise it deletes the key through DELETE /api/v1/apikey/<id> and mints a new one, which the reconciler receives in the same run. The script gains its mirrored unit test, so its mirrored-unit-test exemption goes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Asked by application id, lookup('tls') aligns the domain to the vhost in `domain`. sys-svc-cdn reloads the compose handlers inside every sibling of the vhost loop, so a handler flushed after the loop still saw the application's onion sibling there: on prometheus' second stack deploy (the MCP adapter rebuild) ca_injected answered False, the swarm CA override was skipped and compose_file_args dropped it from docker stack deploy. Blackbox lost the CA mount and the docs healthz probe failed with an unknown authority (API#1, Jenkins#1), and the spec flip recreated every task at once, so the sso-proxy reached central redis before its sidecar registered and the converge gate failed on NOAUTH (OnlyOffice#1, Nextcloud#1, GitLab#1, Mattermost#1; run 40131272595).
The CA mount belongs to the containers, not to a vhost, so the own-TLS check now asks with the canonical domain, which no alignment touches.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Host-wide redirects such as a legacy zone came from the top-level inventory variable redirect_domain_mappings, which only the role default declared. Once the role input moved to web_opt_rdr_domains_mappings nothing declared it any more and the inventory validation refused the deploy, because it does not see the lookup's vars_ read. The redirects now live where the role's other settings are: services.rdr-domains.redirects in meta/services.yml, empty by default and set per inventory under applications.web-opt-rdr-domains, and current_play_redirect_domains reads them through the config lookup. They still win over the application aliases, and a caller that includes the role without its own web_opt_rdr_domains_mappings still renders the merged CURRENT_PLAY_REDIRECT_DOMAINS.
Inventories move their redirect_domain_mappings list unchanged to applications.web-opt-rdr-domains.services.rdr-domains.redirects.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The VPN variants reset credentials with --mirror --backup in the mesh prologue, which copies the manager's host_vars onto the backup host again, NETWORK_MODE: multi included, after extend_inventory had stripped it. The backup deploy then failed in lookup('tls'): "NETWORK_MODE 'multi' needs svc-net-tor with a node onion on this node" (Meta#0, run 40131272595). The strip is now a helper that also runs right before the backup host deploys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every addon ran through the install, enable and configure stages on its own, and each stage asked occ for the plugin's state again. The role now reads occ app:list --output=json once, and the new nextcloud_plugin_plan filter replays the addon order (an enabled entry switches its incompatible_plugins off and itself on, a disabled entry switches itself off, the later entry wins) and splits the app ids into disable, install, enable and configure. One app:disable call takes every id that has to go off; the install, enable and configure includes loop only over the ids that need that stage, so an already enabled plugin skips straight to its configuration. A plugin shipped as a pinned archive still goes through install on every run.
On echoserver the plugin phase took 608 s in full-9 for 51 addons; a repeat deploy is derived at about 195 s instead of about 550 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
quality-high's autoformat pass rewrote three files that 9301220d05,
49aa920034 and c78d5f906c committed without the hooks. ruff drops three
`noqa: E402` markers as unused (RUF100): E402 does not fire on an import
that follows sys.path.insert. The formatter wraps the long
KEYCLOAK_RBAC_GROUP_LOGIC_DIGEST line and the OpenProject admin runner's
`sh -c` line; the double-quoted scalar and the `>-` block both fold the
break back into a space, and a parse against HEAD gives equal values.
On merge: feature/store-i18n 2bcf59109e carries the same four Keycloak
hunks byte for byte; they merge without a conflict.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff drops three noqa: E402 markers as unused (RUF100): E402 does not fire on an import that follows sys.path.insert. The formatter also wraps the long KEYCLOAK_RBAC_GROUP_LOGIC_DIGEST line; the double-quoted scalar folds the break back into a space.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The playwright env renders once per network pass. lookup('tls', app, 'url.base') aligns another application's URL to the pass network even where that application hands the network over, so a spec expected a host the browser is redirected away from (fediwall on its onion pass expected the Mastodon and Friendica onion siblings).
lookup('landing_url', app) returns the aligned URL, or the canonical one behind a handover. Every cross-application URL in the 79 playwright.env.j2 templates goes through it now, fediwall's inline handover expressions included, and a lint rejects both the raw tls form and the inline one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
network_redirects gave every network sibling of a source the mapping, and redirect-domain.conf.j2 falls back to the canonical target when the target has no sibling in the source's network. Since e4fd4552f0 schedules the clearnet apex entry without web-opt-rdr-domains in the round, its fan-out rendered a vhost for the bare node onion pointing at the clearnet homepage in every multi-network round without an onion dashboard: the CSP health check of API#1 and OnlyOffice#1 in run 40109131688 failed with '<node onion>: left .onion: ended at https://dashboard.<domain>/'.
The canonical sibling of a source keeps its mapping as before; a further sibling is fanned out only when the target is served in that sibling's network. A dropped onion source has no vhost and the default server answers it with 404 inside the onion instead of redirecting to clearnet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emailVerified update looped over every non-reserved user, uploaded a
shell script per user and ran kcadm with a JVM login each time, and it
always reported changed: 7 users took 23.8 s on echoserver.
email_verified.py uses the shared keycloak_admin client through
tasks/utils/rest_script.yml: it looks each user up by exact username,
skips users already verified, and writes emailVerified with a GET plus a
full-representation PUT, as kcadm update did. The task reports changed
only when a user was updated, and a failed lookup now fails the task
instead of being treated as a missing user. The old lookup piped
kcadm -q username=X into head -n1, which could have hit a longer username
sharing the prefix.
Evidence (local compose, slot 104): summary updated=0 unchanged=8
missing=0 failed=0 in 0.60 s, task ok, staging dir removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The per-app RBAC groups and LDAP group mappers were reconciled by four
kcadm shell scripts that started a JVM per call and re-synced every app
on every deploy: 582 s for the mappers plus 324 s for the orphan purge on
echoserver (39 apps).
rbac_reconcile.py now talks to the admin REST API from inside the
Keycloak network namespace (nsenter --net) with the admin credentials
read from /proc/<pid>/environ, so nothing secret reaches argv or output.
It drops the shared ldap-roles mapper, purges root groups of undeployed
apps, ensures each app group and mapper, and syncs a mapper only when the
group is new, the mapper config drifted, or the fingerprint stored on the
group (mapper payload, app rbac and domains, group-logic digest) differs.
The per-app payload no longer carries groups.path.group: Keycloak 26.8.1
does not store that key, so a payload with it never matched the stored
mapper and every deploy re-synced every app.
The admin client lives in keycloak_admin.py; tasks/utils/rest_script.yml
stages it with the script in a 0700 temp dir on KEYCLOAK_HOST_NODE, runs
it and removes the dir in an always block.
Evidence (local compose, slot 104): matrix iteration green in both
variants and passes; update runs show synced=1 in 1.1 s, then
synced=0 unchanged=1 in 0.81 s through the final delivery path, no
staging dir left behind.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
services_overrides_for_round applied the # variant-pin: markers when it resolved a round's include set, while plan_dev_inventory_matrix built round_variants, which the inventory bake, the deploy CLI and the swarm matrix read, by clamping the round index alone. Once docs variant 0 pinned web-app-dashboard#reciprocal, round 2 of a sweep resolved dashboard variant 1 (SSO off, no web-app-keycloak in the inventory) but baked variant 0 (SSO on): Keycloak came up without its own bake, defaulted totp_enabled to true, and every dashboard login stopped at CONFIGURE_TOTP while the Keycloak card was missing (11 jobs of run 40042690056).
active_variants_for_round holds the clamp-plus-pins logic once; services_overrides_for_round and the planner both call it, and the planner keeps the primary apps' indices on top.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a multi-network host one gateway serves the clearnet domain and its onion sibling, but allowedOrigins carried only the canonical URL. The Tor pass opens http://claw.ai.<onion>, so the gateway closed every Control UI websocket with 4008 before the pairing step and the UI reported origin-not-allowed (142 onion origins, 120 closes in the Funkwhale#1 rescue log of run 40042690056).
OPENCLAW_CONTROL_UI_ORIGINS takes lookup('network_urls', OPENCLAW_URL): one URL per network sibling with that sibling's scheme, canonical first, and the URL alone on a single-network host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A chat engine prefixed the Vietnamese catalog with its own preamble ("...Vietnamese:" followed by a blank line) and wrote Urdu in Devanagari. Neither tripped the damage predicate: vi is Latin script, so the script checks were blind, and untranslated only measures the Latin share, so a non-Latin language answered in another non-Latin script passed.
added_lines refuses a translation that breaks more lines than its source, and the gateway's _mangled carries the same check so such an answer falls through to the next engine instead of being cached. wrong_script refuses a non-Latin language when less than half of the letters it writes in another language's endonym script belong to its own; the script per language is derived from the endonym in meta/languages.yml, and letters are counted over runs of letters plus combining marks so Indic and vocalised Arabic words stay whole.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parallel-sessions test asserted that more than one of four sessions
appended something, which a loaded host where one worker drains the
shared iterator fails with nothing broken. The sessions now meet at a
barrier before their first append, so a serialised pool still fails the
assertion while a concurrent one always passes.
(cherry picked from commit 657015852a)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The repository-ref updater only checked semver refs and skipped commit SHAs, so a role pinned to a commit never received a bump. Discourse pins its launcher and plugins to commits on feature/updater (297f3f4a67) to stop a rebuild on every upstream push, and those pins must keep moving after the merge.
A full 40-hex ref next to a repository, in meta/services.yml or an addon's config, is now compared with the tip of the repository's default branch (git ls-remote <repository> HEAD, one query per repository and run) and rewritten through the same path as a semver ref, so it lands in the usual update/repository-refs PR. Short SHAs and branch names stay skipped, '# nocheck: repository-version' still freezes a pin, and a failed lookup leaves the pin alone. test_version_pins_are_watched counts such pins as watched, and test_semver_pinning no longer warns about them.
The jeff pin of svc-ai-s1 drops its unread 'semver-ref' marker, and bridgy-fed and ltengine drop the repository-version markers that were written to silence the non-semver warning, so all three follow their upstream from now on, as does taiga's oidc plugin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
current_play_redirect_domains gated both apex entries on web-opt-rdr-domains being in lookup('deployment').deployed whenever the host serves Tor. On a multi node that also dropped the clearnet DOMAIN_PRIMARY -> DOMAIN_HOMEPAGE entry for every isolated variant without that role, so the apex had no vhost and answered with a TLS handshake reject; the Matrix federation delegation on the apex depends on that vhost in swarm. A clearnet node, and main, always schedules it.
The clearnet entry is now scheduled whenever the node's resolved NETWORK_MODE serves clearnet, and the deployment gate keeps guarding only the onion-apex entry. A tor-only node keeps the old behaviour, and the operator's production inventory runs no Tor host, so production is unaffected. Tests cover multi without the role (clearnet apex only) and tor-only (no apex).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The documentation index renders README.md, whose badges load from img.shields.io, and one static build is served on every network the node runs, so the onion docs page requested a clearnet third-party host and network-leak.spec.js failed on the docs+tor pass (fork run 40042690056, jobs Moodle#1, Collabora#1, Listmonk#1 and the swarm API#1 job, log line 37249).
A Sphinx extension now downloads every remote image of an HTML build into the doctree cache, names it by the hash of its URL with the suffix its Content-Type names, and points the image at the local copy so Sphinx ships it under _images. An image that cannot be fetched gives way to its alt text and keeps its link, and the gettext build, which runs with -W, is left alone. Visitors' browsers no longer contact img.shields.io at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the asset and design services off (dashboard variant 1) the logo falls back to service_provider.platform.logo, which without web-svc-asset is the public https://file.infinito.nexus/assets. network_url leaves a host the node does not serve unchanged and port-ui embeds it as is, so the onion dashboard requested clearnet https://file.infinito.nexus/assets/img/logo.png and network-leak.spec.js failed in four jobs of fork run 40042690056 (Gitea#1, OpenProject#1, Mastodon#1, XWiki#1). Pure tor mode loads the same URL; its leak spec only runs with more than one pass.
csp.yml now lists the asset service URL under img-src. build_csp_header does not read img-src, so the header is unchanged; external_asset_origins does, which puts the host on the web-svc-mirror allowlist and into the onion body filter's rewrite to http://mirror.<onion>/file.infinito.nexus, the path every other third-party asset of an onion page takes. A deployed web-svc-asset resolves to the node's own file domain, which external_asset_origins skips. test_branding_mirror_origin.py covers both cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the +tor pass, web-svc-design 'loads shared CSS', web-svc-logout 'loads logout.js' and matomo 'tracker injected' expected the onion CDN and matomo hosts for every target, but three kinds of target render on clearnet there: the SSO issuer is never aligned to the pass network, a handover role's onion sibling only answers 302 to its canonical URL, and an IdP-gated role sends the anonymous visitor to the issuer (rescue proxy log of fork run 40042690056, job Pretix#0: onion 302, clearnet page, cdn.main.infinito.test). The dashboard tile test failed the same way for the four handover tiles.
roles_with_service takes an opt-in network= and keeps a role only when its canonical domain lies in that network and is not a handover sibling; the three env templates pass the network of the pass domain, so the clearnet pass still covers every target and each network's CSP is checked against its own hosts only. The design and matomo specs skip IdP-gated targets the way the logout spec already does. The dashboard exposes handover_domain and expects a handover tile to land on the canonical domain, while href and iframe checks stay on canonical_domain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sys-svc-objstore includes the provider role with application_id set to "{{ _provider_role }}", which is itself lookup('objstore', ...). For a consumer served only on onion the objstore lookup sets proxy_required and calls lookup('tls', consumer_id); without a consumer, align_domain_to_consumer renders variables['application_id'], which runs the objstore lookup again, and the chain repeats until Ansible stops with "Recursive loop detected" / "Maximum depth exceeded" (fork run 40042690056, jobs Moodle#2 and Nextcloud#12, log line 286323). Only web-app-matrix sets single_mode, so only its multi-mode onion deploys hit it.
The tls lookup takes a consumer= kwarg and hands it to align_domain_to_consumer, the way canonical_url already does, and the objstore lookup names its consumer. The default stays empty, so no other caller changes. Regression tests cover both lookups; an onion consumer now resolves to http://matrix.<onion>/objstore/matrix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0327114c44 made the provisioning CLI keep the keys of the last `disable=` run in host_vars under _INFINITO_SERVICES_DISABLED and read them back on the next run. The inventory validator from 285ce910d2 only collects keys that group_vars, role defaults or vars declare, that templates guard, or that plugins read by name, so a key read through a constant in cli/ counted as unread, and every local compose-deploy with disable= aborted with "Inventory validation failed" before the play (repro: make compose-deploy mode=reinstall apps=web-app-keycloak disable=matomo variant=0).
read_variables() now includes DISABLED_SERVICES_RECORD from the disabler module, so the name has one source. An inventory carrying the record validates; an unknown key next to it is still reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lookups render on the controller and read FILE_TOKENS there, while sys-token-store writes the store on STACK_HOST and mirrors it to the controller only in swarm. A compose deploy run from another machine therefore resolved every token empty: on echoserver Friendica rendered an empty SMTP_AUTH_PASS (msmtp: authentication method PLAIN needs a password) and Stalwart minted new bot app passwords on every run.
get_merged_users and the applications carrier key now take the sys_token_store_users fact, which the play reads from the stack host, over the controller file; write.yml refreshes the fact after each write, and 01_load.yml slurps from STACK_HOST so every host reads the store that is written. Without the fact, as in tests and ad-hoc calls, the controller file still applies, and where controller and stack host share the file the merged users are identical.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a fresh Fedora dev container lint-mermaid failed most files with
ConnectionClosedError: the browser aborted with
`FATAL:third_party/skia/src/ports/SkFontMgr_FontConfigInterface.cpp:163]
Not implemented.` in its font fallback, because the host had no font at
all. install-lint provisions the browser's libraries by soname on dnf,
yum and pacman, and a font is not a soname; apt-get gets fonts through
puppeteer's --install-deps.
It looked load-dependent because lint-packages installs rpmlint in the
same parallel phase, which pulls fontconfig and Noto Sans: renders
crashed until that transaction landed, and every later run was green.
With every font removed, lint-mermaid alone fails 274 of 289 files the
same way.
ensure() now installs the distribution's font provide when
/usr/share/fonts holds no file (`font(:lang=en)` on dnf and yum,
`ttf-font` on pacman), and browser_installed() counts the font, so the
install-lint stamp is not fresh on a font-less host.
Closes#1154
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
utils.cache.yaml registered its own !vault constructor that returned a plain str, and it overrides the VaultScalar constructor of utils.handler.vault whenever it registers last. Every writer that loads a host_vars file through the cached loader, changes one key and dumps it back then re-emitted each credential as an untagged quoted string, and Ansible handed the $ANSIBLE_VAULT block itself to the roles.
08680e933d and c88647b3a1 moved tor_node.py and network_mode.py to the round-trip helpers. The shared loader now returns a VaultScalar, so every yaml_io writer keeps the tag regardless of import order. VaultScalar subclasses str, so the other readers of the cached loader see the same value.
test_a_round_trip_keeps_the_vault_tag fails against the previous constructor: re-registering the plain-str constructor on CSafeLoader and SafeLoader drops '!vault |' from the dumped file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the Keycloak OIDC directory active, Stalwart refuses every account create with 'forbidden: Cannot set credentials for accounts in an external directory', and full-4 failed the new batch assert on all accounts. The create request now lists the accounts in the same call and takes each id from the listing of the provisioned domain. Under SSO a forbidden create is tolerated for people, who may not exist before their first login; every bot still needs an id because its app password is addressed by account.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#1180 gave web-app-matrix a canonical `jitsi` domain for its own Jitsi
and listed it under `services.jitsi.domains`. When Jitsi is shared or
disabled nothing serves that domain, and the blackbox healthz job of
web-app-prometheus probed every canonical domain, so the probe of
https://jitsi.matrix.<domain>/healthz/ready failed with "tls:
unrecognized name" and `prometheus scrape: every consumer role reports
up=1 and passes its probe` went red (main run 40017166046: Matrix#2,
Nextcloud#12).
web_health_expectations already skipped such keys. The rule now lives
in utils/domains/unserved.py: a key claimed by a service entity that is
disabled, or enabled but shared, is not served by that app. The health
filter uses it unchanged, and prometheus.yml.j2 filters its targets
through the new `served_domains` filter, which also drops the
`<key>_onion` mirror of a skipped key.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The reason appended to the nocheck marker pushed the line past ruff's limit, so ruff format wrapped the call and left the marker on the closing-parenthesis line, away from the rglob the lint matches. Keep the bare marker, the form every other project-walk opt-out in the repository uses.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_no_raw_project_walk flagged the rglob in DirectoryBatchTests.tearDown. It walks the test's own TemporaryDirectory to make every directory writable again before cleanup, after tests that leave an unwritable parent behind, so the cached project-file helpers do not apply.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Three per-user loops ran on every deploy: an ldap_search for whether the user existed (with failed_when: false, so an LDAP error read as 'missing'), a set_fact building a map from it, and community.general.ldap_passwd per user, 18 s for 7 users on echoserver. ldap_entries state=create now registers the DNs it created, and one ldap_passwords call sets the passwords over a single admin bind: a probe bind as the user decides, INVALID_CREDENTIALS leads to the password modify operation, check mode reports without writing.
openldap_user_passwords keeps the old policy: a user whose record leaves password_update on, or whom this run created, gets its password. Errors on single entries are collected so the others still converge, and the task then fails naming every DN.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Directory loops over ansible.builtin.file cost one SSH round trip per path: the openresty config tree (20 paths) took up to 12.9 s, the webserver core directories (16) up to 13.2 s, every app's compose directories (5) 2.5-5 s per app, the shared CDN roots 5.8 s and the OpenLDAP LDIF dirs 2.2 s on echoserver.
directory_batch takes entries (path plus its own mode, owner, group) or paths with one shared set, and handles each path like file with state: directory: symlinks are followed, missing parents get the same attributes, an attribute left out stays as found, and it never recurses. It reports changed_paths, supports check and diff mode, and fails on a relative path, an existing non-directory or a failed mkdir. It replaces the five loops and the earlier install -d of the CDN role directories, which now report changed honestly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every account ran about six JMAP calls of three Ansible tasks each through jmap_call.yml, 80 s for 7 accounts on echoserver. Each phase now sends one request for all accounts: one x:Account/set creates them all (ids from created or, on alreadyExists, from notCreated.objectId), one updates the passwords of the accounts that already existed, and the app-password phases (list, mint, retire, drop) carry one method call per bot, each addressing its bot through accountId, in chunks of STALWART_JMAP_MAX_METHOD_CALLS (Stalwart's default maxMethodCalls, 16).
jmap_call.yml retries and asserts on every method response of a batch, and a refusal names its owner through the call id. A create refused for a reason other than alreadyExists now fails the deploy instead of being skipped.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the role network's gateway on Stalwart's allowlist, every request OpenResty proxies arrives from an address that is never throttled or banned, so the anonymous HTTP limit and the scan and auth bans no longer applied to web clients at all. Stalwart can take the client address from a Forwarded header, but OpenResty passed a client-supplied one through, so trusting it would have let anyone pick the address that gets banned.
forwarded.conf.j2 gains an opt-in proxy_forwarded_client_address switch that sets Forwarded: for=$remote_addr on every location including the healthz ones, replacing any value the client sent; other vhosts render unchanged. The Stalwart vhost opts in, and the role sets useXForwarded through x:Http. The settings apply moved into security/apply.yml behind one include so the guarded block stays within the three-task limit.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Five conflicts, resolved by what each side is for rather than by taking a
side:
- sys-service/01_prune.yml and sys-service-terminator/02_unlock.yml take
the incoming single `systemctl` invocation over four module loops. The
unlock side also drops an unmask step: the list it runs on is built with
`reject('search', ' masked')`, so it never held a masked unit and the
step did nothing.
- sys-stk-front-proxy/tasks/main.yml keeps this branch's version. The block
the other side still carries there moved into vhost.yml, which runs it
once per network sibling; taking theirs would have duplicated it.
vhost.yml holds every task of that block, the global injection and the
TLS layer included.
- web-app-matomo/08_reinject_consumers.yml likewise keeps the per-vhost
loop, which re-renders the injection for every network sibling and passes
matomo_token_override.
- utils/cache/domains.py combines both: the incoming side rendered
DOMAIN_PRIMARY with the real templar and variables, which is right, but
added a second renderer beside the module's own. _render_plain now takes
that context optionally, so the other two call sites stay as they were.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apply_network_mode_from_env loaded the whole host_vars file through
utils.cache.yaml, set one scalar and wrote everything back. That loader's
!vault constructor returns a plain string, so every credential in the file
was re-emitted untagged and Ansible handed the raw ciphertext to the roles
instead of the secret. It only runs when the row sets `network`, which is
every multi-network round.
Run 40032069040 on the fork shows what that costs: the generated host_vars
of a failing job carries 0 `!vault` tags against 56 `$ANSIBLE_VAULT`
headers, mariadb then refuses root with 1045 on a freshly initialised
database, and the openldap bind fails with result 49.
08680e933d fixed the same class in tor_node.py, the sibling writer in this
package. This one writes through the round-trip helpers too.
The existing test did not cover the case, which is why the sibling fix did
not stop it: test_preserves_existing_vault_credentials now fails against
the previous writer with
`'!vault |' not found in "... root_password: '$ANSIBLE_VAULT;1.1;AES256`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
get_merged_domains rendered DOMAIN_PRIMARY with no templar and an empty variable scope. A primary declared as a template, such as echoserver's "{{ SOFTWARE_NAME | lower }}", therefore came out as 'none' inside the applications SPOT, while direct lookups saw the domain Ansible had already resolved. Every SPOT value built through it named file.none, and the dashboard waited for https://file.none/assets/img/logo.png until full-2 failed. The render now gets the templar and the play variables, which also covers services.tor.node.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
