Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.
- 00_core.yml touches the store with modification_time and access_time
preserved, so it only creates a missing file. In swarm the controller
sees the store over NFS with attribute caching, and the touch alone
made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
mirror changed the store. Both run once, so every host that ran the
include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
so hosts that did not run the writing role recover too. Where the
key still matches this serves the carried payload; measured 0.1 s per
role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
memoised per inode, mtime and size, so a touch no longer changes the
key. A stat that still matches serves the memo (1.67 us against
1.41 us for the old stat signature), and a stale stat keeps key and
YAML cache on the same old content instead of pairing new content
with an old key.
- applications_carrier opens the tokens file before it computes the
key, so on the NFS-mounted controller the parked key reflects the
revalidated attributes rather than a cached stat that changes a
minute later.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A role read its application_id from its own vars/main.yml, which sits
below host facts in Ansible's precedence. test-e2e-playwright loads the
tested role's vars/main.yml with include_vars and no name:, so that
role's application_id and domain land as facts and outlive the loop. The
Playwright stage runs in 02_server, the workstation stage after it, so
every later role that resolves lookup('config', application_id, ...) from
role vars asked under the last tested role's id.
A local compose deploy died on it: desk-agnt-claude looked for
services.agnt-claude under applications['web-svc-mirror'] and aborted the
play, taking four more desk roles and the whole destructor stage with it.
Ten roles resolve a config path this way; a three-argument lookup in any
of them would have returned another application's value silently instead
of failing.
Binding application_id as an include param puts it above the leaked fact.
The value is not invented: role_entry.app is the application_id the role
itself declares, and this is the same immunity the web-app roles already
have by passing application_id on their own include_tasks.
A negative control reproduced the leak and then defeated it: a probe role
declaring application_id in its own vars saw the foreign value after an
include_vars, and saw its own once the include carried the param. All 183
roles that declare an application_id declare it identical to their
directory name, and this include site has exactly one caller, so the
binding changes nothing else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stage looped over every role of its group and skipped the ones this host
does not deploy, one task at a time, plus an unconditional handler flush behind
each. Measured on a one-app deploy on 2026-09-23: 132 of 139 entries skipped,
278 task deltas, 41.8 seconds of nothing.
The lookup now takes the `groups` magic variable and drops the entries no host
in the inventory carries, so they never reach the loop.
Which roles get deployed does not change, and that is provable rather than
hoped for: the body deploys a role exactly when application_allowed holds,
which requires the app to be in that host's group_names; if it is, that host is
in groups[app], so groups[app] is non-empty and the lookup keeps the entry. The
filter is therefore strictly weaker than the condition it precedes and can drop
nothing the body would have run.
The filter deliberately reads `groups` and not `group_names`. 8aea5a9b35
measured that a per-host condition above the loop makes ansible insert the
included blocks in host order instead of loop order, after which
svc-storage-nfs-client mounted an export svc-storage-nfs-server had not
deployed yet and the play died on 'mount.nfs4: Connection refused'
(run 31645153977). `groups` is the same mapping on every host, so every host
still walks one identical list, and tasks/utils/setup/role.yml keeps its `when`
exactly where that commit put it. Two of the new cases pin this: a role only
one host carries still enters the loop, and a group_names passed to the lookup
changes nothing.
First written the other way, with group_names in the lookup and the `when`
removed. A single-host deploy cannot show that defect, and it came back green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A per-host `when` on the looped include_tasks made ansible insert the included blocks in host order instead of loop order: whichever host reached an item first decided when that item ran. With the nfs-server host last in the inventory, svc-storage-nfs-client mounted the export before svc-storage-nfs-server had deployed it, and the play died on 'mount.nfs4: Connection refused' (run 31645153977, job 94280126636).
Gating on the include_role instead keeps every host on the same loop, so one block per item is inserted in loop order. This is what cli/build/include emitted before 351b64e2c: the when on the include_role, the flush unconditional behind it.
Reproduced with a four-host probe -- server host last inverts the order, gating one level down restores it. In run 31543714113 'setup svc-storage-nfs-server' ran 3 times; in 31645153977 it never ran.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cli build include` generated one tasks/groups/<group>-roles.yml per invokable category during setup, and the stage plays included those files. They were written by the in-container UID, which is why a dedicated clean-container-owned helper had to delete them from inside the container.
The stage plays now loop over lookup('group_roles', <group>) and include tasks/utils/setup/{group,role}.yml directly. ordered_roles() in utils/roles/order.py holds the run_after topological sort the generator used to perform at build time.
utils/roles/stage.py:_group_names() no longer globs the generated files; it reads get_invokable_paths(), the same categories.yml source those files were derived from.
Measured: handlers still flush after every role (ansible-core 2.20.2, three loop items, three separate handler runs), and resolving all 18 groups costs 0.60s cold against 1.4s for the generator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbach
