- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two conflicts, both resolved by combining rather than by picking a side.
`roles/web-app-socialhome/tasks/02_design.yml`: designer restructured the pass,
moving the token request and the theme write into the new
`02_design/converge.yml` and adding a stat of the name-state file. HEAD still
carried the converge block inline. Kept designer's stat, dropped HEAD's inline
block: the merged body below the conflict already includes `converge.yml` and
reads `socialhome_name_marker`, which only designer defines, so keeping HEAD's
side would have duplicated the token request and left that variable undefined.
HEAD's `retries: 0` on the token request is deliberately not carried over; the
default of 30 retries covers a service that is not yet accepting connections,
which matters more here than failing fast on bad credentials.
`roles/web-app-fider/files/playwright/test-design.js`: the import line takes
both sides, since the merged body uses designer's `apiGetOnion` and HEAD's
`requireDotenvValue`. The env block keeps HEAD's `requireDotenvValue` form,
because designer's `|| ""` defaults are what
`tests/lint/ansible/.../test_unguarded_env_default.py` exists to reject, and
drops HEAD's module-scope `title` and `logoEnabled`: the merged body defines
`title` locally and reads `DESIGN_LOGO_ENABLED` directly, so both were dead and
would fail `no-unused-vars`.
Checked after resolution: no conflict markers (`git diff --check`), no `nocheck`
line dropped by the merge, and no duplicated YAML key in the seven merged files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`lookup('tls', ...)` aligns a cross-app domain to the consumer named by
`application_id` (utils/tls_common.py `align_domain_to_consumer`): a
clearnet-primary consumer gets the clearnet sibling of an onion-primary
target. The carrier key does not cover `application_id`, so a park that
renders inside a role hands that role's alignment to every later role
whose key matches.
The constructor and `tasks/utils/setup/role.yml` park at play scope,
where `application_id` is undefined. The re-park in
`roles/sys-token-store/tasks/write.yml` (67f6c8d7e8) and the nested-load
re-park render inside the caller: `svc-prx-openresty` at the preload
loader, `web-app-stalwart` or a provider role on the NFS host, the web
app itself under `sys-stk-front-base`. In a tor deploy that turned onion
targets into clearnet domains for onion-primary roles served from such a
park, and baked the caller's group into Nextcloud's sociallogin
`groupMapping`.
Drop `application_id` from the render's variables and from the
templar's for the duration of the render, so every park carries the
play-scope result the constructor parks have always carried. Rename the
nested re-park task so its log lines do not read as a role boundary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Job 116975298820 (Nextcloud#5, swarm, arch, btrfs, main 463df06202) hit
its budget (rc=124 after 20136 s). The app phase loads every service as a
nested `tasks/utils/load/app.yml` include under sys-service-loader, so
`tasks/utils/setup/role.yml` parks no carrier between `svc-virt-kata`
(L6073) and `update` (L38213). The only re-park in that stretch is the one
in `roles/sys-token-store/tasks/write.yml`, and Stalwart's user management
reaches it on the manager alone. After its last token write (L23574) the
manager carried the new key while both workers kept the old one and
rendered the full tree on every task: 85 non-wait tasks of 30 s or more, 63 min
from Keycloak (L25641) to the carry past `update` (L38237).
Park the carrier after every nested load on every host of the include.
A re-park whose key still matches serves the carried payload without
rendering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.
- 00_core.yml touches the store with modification_time and access_time
preserved, so it only creates a missing file. In swarm the controller
sees the store over NFS with attribute caching, and the touch alone
made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
mirror changed the store. Both run once, so every host that ran the
include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
so hosts that did not run the writing role recover too. Where the
key still matches this serves the carried payload; measured 0.1 s per
role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
memoised per inode, mtime and size, so a touch no longer changes the
key. A stat that still matches serves the memo (1.67 us against
1.41 us for the old stat signature), and a stale stat keeps key and
YAML cache on the same old content instead of pairing new content
with an old key.
- applications_carrier opens the tokens file before it computes the
key, so on the NFS-mounted controller the parked key reflects the
revalidated attributes rather than a cached stat that changes a
minute later.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The applications_carrier render that the designer merge brought in spelled the deployment mode out as DEPLOYMENT_MODE == 'swarm'. tests/lint/ansible/variables/test_deployment_mode_booleans.py names that line as its only hit, which is the real half of the red test-lint in the post-merge gate.
IS_SWARM_MODE is that comparison under its name, so the task still runs under swarm and still does not run under compose. Both mode lints pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six conflicts, all from git pairing store-i18n's renamed roles with their designer originals.
web-app-meta/meta/services.yml and web-app-sphinx/vars/main.yml: store-i18n renamed web-app-sphinx to web-app-meta and dropped its custom build, so the role keeps the prebuilt ghcr.io/infinito-nexus/meta image and the old sphinx role stays deleted.
svc-ai-tika/templates/compose.yml.j2: a bogus rename pairing dropped sphinx build lines into the tika service. Rejected; the template is byte-identical to its pre-merge state again.
web-app-docs/templates/Dockerfile.j2: the designer file renders sphinx under the design title and reads SPHINX_IMAGE, SPHINX_OUTPUT_DIR and SPHINX_EXEC_DIR, none of which exist in web-app-docs, which builds from files/Dockerfile. Rejected.
web-app-docs/templates/playwright.env.j2 and files/playwright/test-design.js: both sides add independently, so both are kept. Of the designer's three new env reads only designTitle survives; iconUrl and faviconUrl are shadowed inside the test by stricter requireDotenvValue reads and were dead at module scope.
plugins/lookup/email.py merges clean toward designer, which is intended rather than a loss: 08ef66d8de (2026-10-08) replaces the submission_via_relay port-25 rule of 233f15f3e0 (2026-07-01) with per-bot app passwords under SSO, and carries the matching 04_manage_user/app_password tasks.
Checked after resolving: no duplicate YAML keys across the 17 merged YAML files, and no nocheck line dropped above a renamed key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.
Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
The constructor parked no applications carrier between the merge block
and the token store, so every lookup in the svc-swarm-node, svc-storage,
svc-registry and svc-virt bootstrap rendered the SPOT cold, 25 to 30 s
each on a loaded host.
A carrier there was unsafe: in swarm, svc-storage-nfs-server mounts the
NFS export over the controller's DIR_VAR_LIB, so the tokens file the
render reads can appear in the middle of that section, and the cache key
did not notice. merged_applications_cache_key now includes the tokens
file's path, mtime and size, as the users cache key already did, so a
carrier rejects itself once the tokens change. A carrier parked after
the add_host tasks then serves the bootstrap roles. It is parked only in
swarm mode: in compose the section runs in about a second, so a carrier
there would add a render without saving one.
In a swarm drill of web-app-prometheus up to the known NFS probe-write
failure, "Resolve the NFS server backend flavor" went from 29.6 s to
0.07 s, "Probe-mount the NFS export" from 26.4 s to 0.4 s, "Mount
cluster-shared infinito state on controller" from 23.7 s to 0.5 s and
"Ensure Docker-In-Docker DNS works" from 30.0 s to 3.6 s; the run to
that point took 768 s instead of 1735 s.
Closes#925
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A role sync runs the whole role, several hundred tasks, although a
design carrier usually lives in one task file. Iterating on that file
cost minutes per try.
`make compose-role-sync role=<app> variant=<idx> task=<file>` now runs
only that task file of the role against the running stack. The sync
playbook gets a third scope `task` next to `design` and `role`: it
places the shared services like a role sync, loads the token store and
the proxy handlers, includes the role with `tasks_from` and flushes
its handlers.
The task file gets the variables of the role and nothing an earlier
task of the role would have set, so it has to stand alone. The script
accepts a task path of plain path characters only, because the value
travels into a command line inside the container.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make compose-role-sync marks every shared service as converged, so the
proxy role never runs ahead of the app role and its handlers stay
unknown until the app role includes its stack. A task that notifies
reload-openresty before that point aborted the sync with "The requested
handler 'reload-openresty' was not found" whenever it reported a
change; web-app-keycloak deploys its logout status panel that way.
The sync now loads the handlers of svc-prx-openresty first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make compose-role-sync runs only the role of one deployed application,
with its proxy vhost and injected snippets. It marks every shared
service of the stack as converged and skips the stack-wide service
stop. With pw= it stages the Playwright spec of the role again, renders
its env and runs it. make design-sync keeps re-rendering only the
shared CSS, the role stylesheet and the branding assets. Both run
through one playbook-sync.yml with a scope, which replaces
playbook-design.yml.
The host placement of the proxy and the shared services moves from the
constructor stage into tasks/utils/constructor/placement.yml, so the
role sync can include it.
base='<hex>' on both targets passes DESIGN_BASE_COLOR. The development
inventory references that variable for the design base color and keeps
#FFA500 as its default, so a deployed role can be rendered and checked
with another palette.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Narrowing the meshed drill to its backup half bought budget by giving
up the recovery legs over the mesh. The budget overrun is worth measuring
before it is designed around, so the pre-drill pass and the full teardown and
recover come back. Reverts a27b093c4.
The suite declares required_by for every web deploy, and the deploy
verifies afterwards that each such role actually executed. Holding it back
for the pre-drill pass therefore failed that pass outright:
[ERROR] required role(s) did not execute for deploy [...]:
- test-e2e-cli
Eleven meshed rounds died there, and every one of them was an application
that drills -- the only rounds that run a pass with the flag set. The lone
drilling round that survived was a database rather than a web application,
so the requirement never applied to it.
Only the browser suite stays behind the flag now. It is the expensive half
anyway: measured over one meshed round, Playwright is 13% of deploy task
time against 3.3% for the CLI suite, so holding the CLI suite back bought
almost nothing and cost the deploy its own completeness check.
3dca8aca93 replaced the bootstrap loop with a block to keep svc-storage-nfs-client
from running on a host that is only a swarm node. That broke a contract:
stage_groups excludes bootstrap categories because they run via explicit steps,
and test_invokable_roles_wiring therefore looks for them as a `- <item>` entry in
a stage loop. Without the loop the category svc-swarm had no reference left.
It only ever passed by accident: the pattern matched `- svc-swarm-node`, whose
hyphen is a word boundary, so the category was never named in its own right.
Restore the loop and express the gate as an item condition, which a loop
evaluates per element, so the NFS client is skipped while the swarm node is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3c07721303 put svc-swarm-node and svc-storage-nfs-client in one loop behind
"svc-swarm-node in group_names", so a host pulled the NFS client purely for
being a swarm node. storage.nfs.server defaults to the empty string, so on a
compose single host the client asserted itself out with "svc-storage-nfs-client
requires storage.nfs.server to be set in the inventory".
The swarm inventory places every svc-swarm-node host in svc-storage-nfs-client
as well, so requiring both leaves swarm deploys untouched and only drops the
run where the inventory never asked for a client. sys-svc-mail-msmtp and
sys-svc-compose already model it that way: both tolerate a host that is no NFS
client, and the mount is asserted only where the backend is nfs.
main carries this loop unchanged, so it is reachable there too: a role brings
the trigger in by declaring the node service consumer, and svc-registry-cache
and svc-registry-docker both do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nothing declares the flag and nothing sets it by default, so the runtime
check still decides whether the suites are meaningful and a deploy that does
not ask behaves exactly as before. Only the pass whose stack the recovery
drill is about to tear down sets it, on its own run.
web-svc-css becomes web-svc-design and owns the configuration that lived in group_vars/all/13_design.yml. Consumers switch their css service key to design, and sys-front-inj-css becomes sys-front-inj-design. javascript stays an independent key; a role's optional design.js.j2 is injected together with the design and CSP-hashed like the functional script.
An in-repo OKLCH lookup replaces colorscheme_generator. From design.colors.base alone it derives surfaces, text, borders, the primary and status fills with their on-colors, hover and active states and a dedicated dark palette; a unit test proves WCAG AA, and AAA for body text, across six base colors in both modes. Semantic --design-* tokens replace --color-NN-SS in every role stylesheet. The shared element styles sit in the cascade layer infinito-design, so an app's own design system always wins and takes the palette through a variable mapping in its role stylesheet or through its own theming mechanism.
design.logo and design.title resolve globally from web-svc-design or per role from its design entry, and false or 0 disables the replacement. The design_assets action renders the logo lockup per slot as SVG, PNG and ICO, and the legacy title keys move into the design entries.
Three roles carry the design end to end. The dashboard takes logo, favicon and title from the generator and replaces the hard-coded port-ui colors. Gitea maps its theme variables onto the tokens and mounts the generated logo and a branded start page. Nextcloud receives primary color, logo, name and slogan through occ theming and ships no stylesheet.
make design-sync re-renders the static design files of a deployed app, make design-gallery captures before and after screenshots in light, dark, desktop and mobile, and make design-queue orders the roles that are due for a design pass. The i8-designer skill and docs/agents/action/design.md describe that loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entity name keys a role's instance directory, its compose project and its container name, and was derived only by stripping the longest matching category prefix. Two roles under different categories whose names share a suffix therefore collapse onto one entity and overwrite each other's rendered stack, which no provides: can repair.
A literal entity_name in vars/main.yml now wins over the derivation. web-app-seaweedfs-console takes its shorter name back and declares the entity seaweedfs-console, which frees the seaweedfs key in its own services map for the engine consumer and removes the provides:, covers: and alias entries that stood in for it. A new lint pairs the declaration with the primary key it has to match, because a drift between them would drop the role from the provider set in silence.
Compared across all 288 roles, exactly one entity changes, and it restores what main already keeps on disk, so no instance directory moves.
The function and the filter lose their get_ prefix. Two lints read that prefix as the discriminator between the variable and the filter that fills it; both discriminate on the pipe now. Ansible's templar resolves a filter and a same-named variable independently, verified against ansible-core 2.20.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`WIREGUARD_ENABLED` is `DEPLOYMENT_MODE == 'swarm'`, so it is true on
both arms of the vpn axis and the account the mesh transport needs was
created on every swarm node whether or not a mesh was coming.
It now asks the same question the bootstrap below it asks -- whether this
host deploys svc-net-wireguard at all -- so the account appears only where
the second pass has to log in over the tunnel.
The merge block drops its carrier and the stage-wide one is parked after the
add_host tasks, so svc-net-wireguard bootstraps in the window between them
with no carrier at all. A block-scoped one spans just that role and is
dropped again, leaving every later stage to re-render as before.
The package cache was configured in six places at once: the nginx upstream
map, the consumer compose override, pip.conf, .npmrc, the apt sources and the
twenty ensure_proxy_repo calls that bootstrap Nexus each spelled the same
hostnames, repository names and suites by hand. A new upstream meant editing
five files and hoping none was missed, and the docs told contributors to do
exactly that.
A role now declares what it needs under cache: in its meta/networks.yml, and
everything else is derived from it. svc-cache-package holds the general
upstreams; web-svc-libretranslate holds its own. All nine derived files are
generated and gitignored, so a second spelling of a hostname cannot exist.
utils.cache.hosts reads the declarations without PyYAML, because the .env
generator reaches them before any dependency is installed; it raises rather
than truncate when a comment sits at column 0 inside the block, which would
otherwise drop every declaration below it in silence. utils.cache.records
owns the Nexus record format and the Ubuntu mirror resolution, so the dev
stack and the role provision from one rendering.
A build may only resolve a cached host it can verify: the plain-HTTP hosts
are hijacked for every image, and the TLS hosts only for an image whose
Dockerfile installs the frontend CA. Hijacking all of them would fail every
pip and npm build with CERTIFICATE_VERIFY_FAILED.
svc-cache-package deploys the cache and its TLS frontend by calling the same
shell scripts the dev stack calls, rather than reimplementing them as tasks.
The role is invokable through the new svc-cache category and inert until a
host carries it.
Verified: the twenty Nexus payloads, the fifteen nginx server blocks and all
seven package-manager configs are byte-identical to the hand-written files
they replace, and the new category adds one role to the deploy order while
preserving the relative order of the other 180.
The role has not been deployed yet.
Added on a hypothesis the next CI run disproved: the index was never stale,
the candidate version was identical before and after. It fixed nothing and
buys a refresh every deploy already gets where it needs one.
A worker carries no web-app group, so it resolves to no host-type stage and
never ran user-administrator; the account existed on the manager alone. Once
the transport moves, Ansible logs into every node as that user, and the
workers refused the key outright.
The deploy images delete /var/lib/apt/lists at build time and the package
registry installs without update_cache, so the first apt task of a deploy
resolved against an empty index and picked whatever version the stale .deb
metadata named. Refresh once, where apt is already made ready, so every role
benefits rather than each one carrying its own refresh.
A desktop role writes into the home of the workstation account, chowns to
its name, or becomes it, but nothing made that account exist. Job
108388860157 of run 36235368471 shows what that costs:
Module failed: chown failed: failed to look up user biber
roles/dsk-gnt-claude/tasks/01_gateway.yml:2 path: /home/biber
The round deployed dsk-gnt-claude, dsk-gnt-codex and dsk-code without
user-workstation, and the log had already warned `Unknown user 'biber'`.
tasks/stages/02_workstation.yml was the only guarantee, and it runs for
workstation hosts alone.
The premise is now a service edge. `dsk-base` provides `desktop-base` and
pulls in `user-workstation`; every other dsk role declares that key in its
own meta/services.yml. `_is_service_required` then marks the key required
from the consumer's own declaration and the loader runs both providers in
the constructor stage, before any desktop role. user-workstation becomes a
service at all here: it had no `application_id`, so it was in no registry,
which is what its `# nocheck: required-by-coverage — TODO` recorded.
Measured over every role, HEAD against this tree: exactly the 32 dsk roles
change, each gaining {dsk-base, user-workstation}; no other closure moves
and nothing is removed. For an inventory holding dsk-gnt-claude alone —
the CI constellation — both providers are required and both are in the
whitelist closure, so neither depends on group membership.
The flags stay literal with `# nocheck: dynamic-flag`. The documented
dynamic form is circular for a premise: the edge would only be true once
the provider is already in group_names, and the edge is what puts it there.
Two lints hold the pattern. `services-declared-load` fails an invokable
role that loads a service provider with include_role while declaring no
edge; it found svc-ai-agent-broker, which keeps a nocheck because the edge
would set kata.enabled on a role pinned to the manager, a combination
test_no_placement_manager_with_sandbox rejects as unschedulable.
`desktop-base-edge` fails a dsk role that declares no edge at all. Both
were negative-controlled; the first was vacuous on its first pass, because
iter_project_files_with_content yields absolute paths and the role was
derived from a relative one.
The MCP audit and the alias list follow from the new role and the earlier
i18n-retry target. Disposing of the two new roles is what pulled the audit
into this change, and the audit is why the MCP documents are split here
rather than separately: requirement 035 carried a user story, an
architecture, a per-role audit two lints parse, and 32 acceptance
criteria in one 593-line file. The decision half moves to
docs/architecture/decisions/0003-mcp-surface-per-provider.md, which holds
the forces, the contract, the audit and the consequences; 035 keeps the
user story, the criteria and the order of work. Both link to each other
and state the split: a disposition change belongs in the record, a newly
met criterion in the requirement. The three lints and three cross-
references follow the audit to its new home.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role was the first thing to install a package after 'update' dist-upgrades
every node, and apt could no longer satisfy the dependencies -- not for
wireguard-tools, and not for iproute2 or iputils-ping either. Nothing between
the two installs anything, so the role was simply first to hit it.
Ahead of the upgrade is also where the mesh belongs: it is the transport the
rest of the deploy is meant to run over, so it comes up before anything else
touches the host. It stays ahead of the swarm bootstrap for the reason it
always was.
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.
Seven conflicts:
- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
in e706170a0 were modified upstream. The deletion stands, which is the whole
premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
and this branch's _write_mesh call after it.
The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
A role read its application_id from its own vars/main.yml, which sits
below host facts in Ansible's precedence. test-e2e-playwright loads the
tested role's vars/main.yml with include_vars and no name:, so that
role's application_id and domain land as facts and outlive the loop. The
Playwright stage runs in 02_server, the workstation stage after it, so
every later role that resolves lookup('config', application_id, ...) from
role vars asked under the last tested role's id.
A local compose deploy died on it: desk-agnt-claude looked for
services.agnt-claude under applications['web-svc-mirror'] and aborted the
play, taking four more desk roles and the whole destructor stage with it.
Ten roles resolve a config path this way; a three-argument lookup in any
of them would have returned another application's value silently instead
of failing.
Binding application_id as an include param puts it above the leaked fact.
The value is not invented: role_entry.app is the application_id the role
itself declares, and this is the same immunity the web-app roles already
have by passing application_id on their own include_tasks.
A negative control reproduced the leak and then defeated it: a probe role
declaring application_id in its own vars saw the foreign value after an
include_vars, and saw its own once the include carried the param. All 183
roles that declare an application_id declare it identical to their
directory name, and this include site has exactly one caller, so the
binding changes nothing else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stage looped over every role of its group and skipped the ones this host
does not deploy, one task at a time, plus an unconditional handler flush behind
each. Measured on a one-app deploy on 2026-09-23: 132 of 139 entries skipped,
278 task deltas, 41.8 seconds of nothing.
The lookup now takes the `groups` magic variable and drops the entries no host
in the inventory carries, so they never reach the loop.
Which roles get deployed does not change, and that is provable rather than
hoped for: the body deploys a role exactly when application_allowed holds,
which requires the app to be in that host's group_names; if it is, that host is
in groups[app], so groups[app] is non-empty and the lookup keeps the entry. The
filter is therefore strictly weaker than the condition it precedes and can drop
nothing the body would have run.
The filter deliberately reads `groups` and not `group_names`. 8aea5a9b35
measured that a per-host condition above the loop makes ansible insert the
included blocks in host order instead of loop order, after which
svc-storage-nfs-client mounted an export svc-storage-nfs-server had not
deployed yet and the play died on 'mount.nfs4: Connection refused'
(run 31645153977). `groups` is the same mapping on every host, so every host
still walks one identical list, and tasks/utils/setup/role.yml keeps its `when`
exactly where that commit put it. Two of the new cases pin this: a role only
one host carries still enters the loop, and a group_names passed to the lookup
changes nothing.
First written the other way, with group_names in the lookup and the `when`
removed. A single-host deploy cannot show that defect, and it came back green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-swarm-node is staged explicitly in the constructor rather than reached
through the dependency walk, so the run_after edge declared against
svc-net-wireguard never ordered anything. Swarm initialised first, advertised
10.100.0.1, and docker refused: 'the address to advertise is not recognized as
a system address' -- the interface did not exist yet.
Bootstrapping WireGuard immediately before it puts the ordering where the
staging actually happens. The role keeps its own run-once guard, so the later
dependency-walked include is a no-op.
113113e553 parks the carrier after the token store, 165 lines below the merge
block, so each of the block's six tasks still renders the applications SPOT
cold in its own forked worker. Measured in CI run wp-swarm-tor2 they cost
323.51s in the PASS 1 main play, 180.39s in the backup play and 479.79s in
PASS 2.
The cost is the render, not the work. Two tasks in that block share the exact
construct `defaults_X | combine(X | default({}, true), recursive=True)`: design
takes 0.247s and service_provider 75.94s. The difference is that
defaults_service_provider is dense with lookup('config'/'users'/'domains'/'tls')
and every one of them lands in get_merged_applications.
The carrier cannot simply move up. get_merged_applications serves the carrier
fact whenever the key matches, and the carrier action goes through that same
path, so a second carrier task would be handed the first one's payload instead
of rendering. merged_applications_cache_key reads neither group_names nor the
token file, and with a real templar 71 of 265 applications differ by
group_names alone, so a carrier left standing across add_host or the token
store would be served where a fresh render belongs.
So the block opens its own carrier and drops it again at the end. Nothing
outside the block changes: every task from sys-version to the stage-wide
carrier renders exactly as before.
Placement follows measurement. A per-key perturbation test over the whole
265-application payload, with the credential regeneration noise floor
subtracted and every role in group_names, finds exactly two reads of the seven
facts the block sets:
.web-app-matomo.services.matomo.excluded_ips[0] <- networks
.web-app-wordpress.services.wordpress.title <- service_provider
networks is the block's first task and service_provider its last, which is why
the carrier opens after the one and is dropped before the other takes effect.
The block's tasks already render before service_provider is merged today, so
the frozen payload is what they would have rendered anyway.
The action gains a `clear` argument so the playbook never spells the private
fact name. Parking the carrier serves a payload in 0.000s against a 3.417s cold
render; a wrong key and a cleared fact both force the render again, which is
what makes the drop task work.
Worth roughly 806s of the 20199s job, against a floor of 673.6s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.
The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.
The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.
Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
68adb944ff made the constructor the sole opener of the preload context so
list_or_shoot would queue instead of shoot during the Bootstrap block, and the
loader's flush would run the queue once the shared services were up. Nothing
closes it again. After the flush the flag is still true, so every later
list_or_shoot appends to a queue nobody drains.
Where a shared service gets preloaded the leak is masked: preload_service.yml
clears the flag around each service, and the flush had items by then. It is
masked, not absent -- in three healthy jobs of run 33279942812 the last of the
43 mark/clear events is a mark, so the flag is true when the loader returns
there too, and the 43 to 59 appends that follow are discarded in silence. The
first of them is sys-ctl-alm-email's "Include mail dependency", which is why
the mail dependency never loads.
Where no shared service is preloaded there is nothing to mask it. The flush ran
once at 08:48:23 against an empty queue and all six appends came after it, so
eight sys-ctl roles never executed, the play still reported failed=0, and the
post-run check in cli/administration/deploy/dedicated/runner.py caught it with
exit 2. Twelve jobs across eight apps in that run failed exactly there.
The opener stays where it is. Two appends per healthy job land before the
loader opens its own first context, so preload_service.yml alone would not
cover them -- which is the case 68adb944ff was written for.
10f320df98 already assumed this flag flips: it pulled two roles out of their
once-gates so work would "re-evaluate after the preload-context flag flipped".
After the loader it never did. This is the condition that commit needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A per-host `when` on the looped include_tasks made ansible insert the included blocks in host order instead of loop order: whichever host reached an item first decided when that item ran. With the nfs-server host last in the inventory, svc-storage-nfs-client mounted the export before svc-storage-nfs-server had deployed it, and the play died on 'mount.nfs4: Connection refused' (run 31645153977, job 94280126636).
Gating on the include_role instead keeps every host on the same loop, so one block per item is inserted in loop order. This is what cli/build/include emitted before 351b64e2c: the when on the include_role, the flush unconditional behind it.
Reproduced with a four-host probe -- server host last inverts the order, gating one level down restores it. In run 31543714113 'setup svc-storage-nfs-server' ran 3 times; in 31645153977 it never ran.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
