A whitelist-scoped update preloaded and fully executed every provider role
the host is a member of, because the gate only asked whether the role was in
group_names. On this inventory that meant a single-app update built compose
contexts, rendered configs and vhosts for 17 unrelated applications.
reachable_roles() is lifted out of applications_if_group_and_all_deps() so the
same traversal can start from an arbitrary seed set instead of group_names.
The old function now calls it and filters as before, so every existing caller
keeps its result. A new application_closure lookup exposes it to Ansible, and
sys-service-loader gates each preload on the whitelist's closure. An empty
whitelist yields an empty closure, which the gate reads as the full inventory
scope, so a no-apps deploy is unchanged.
Measured on mode=update apps=web-app-snipe-it: 42:49 before, 17:18 after, with
ok tasks dropping from 4903 to 2030. Nothing outside the closure ran, and
snipe-it's real dependencies (mariadb, redis, openldap, postgres, keycloak,
mailu, prometheus, seaweedfs, cdn, logout) all still did.
The closure also gates the required branch, deliberately. `required` reads as
a global property but plugins/lookup/service.py derives it from group_names,
so it means "some provisioned app needs this" and admits nearly every service.
Exempting it was measured at 42:39 against 17:18 for the same deploy, which is
the entire saving. A service whose provider lies outside the closure is by
definition required by an application outside the deploy scope.
Out of scope: seeding applications_current_play from the closure. That shrinks
CURRENT_PLAY_DOMAINS_ALL, which cert SANs, openresty vhosts, DNS records and
the webserver health check all consume at full inventory scope.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flatten a single list term in the service lookup so query(service, [ids]) does one applications load instead of one per id. Precompute the required service ids in sys-service-loader and gate the preload by set membership, collapsing about 30 applications loads into one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sys-service-loader iterates the role-wide static service order, so it
asks the service lookup about keys (e.g. the css alias of cdn) that the
play-local registry built from applications does not contain. The lookup
raised and killed the bootstrap play on every distro.
Fall back to the registry read from roles_dir, the same source the
loader iterates, so every loader-visible key resolves with its provider
role and play-computed flags (false when no consumer is deployed).
Canonical aliases aggregate their primary key's consumer flags. A term
unknown to both registries still raises.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A host-dependent when on the include loop made Ansible register the
included blocks in result-arrival order: workers answering before the
manager pulled their subset (the app's own service, ldap) ahead of the
manager-only engine providers, so consumer stacks deployed before the
engines they converge-wait on (gitea before redis). The condition now
lives inside load_service.yml, every host reports an identical include
set and execution follows the registry order. Hosts that load nothing
also leave the loop with service_loader_preloading reset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pre-flush made the preload block exceed the 3-child lint limit; move the block body (mark, pre-flush, load, post-flush) into preload_service.yml so the block keeps only the include plus the always-clear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A shared-service preload switches application_id mid-consumer-pass; the preloaded service's front-proxy pipeline flushes handlers (sys-stk-front-proxy:113), firing the consumer's pending compose-up notifies under the service's binding whose instance dir may not exist. Drain them under the still-correct consumer binding first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
list_or_shoot.yml was previously a no-op during the constructor's
"Bootstrap registry + NFS clients" block because nothing had activated
the preload context yet, so transitive deps such as sys-ctl-hlth-msmtp
got inline-shot before web-app-mailu existed and crashed every swarm
deploy. The constructor now opens the preload context (`service_loader_preloading: true` plus an empty `service_loader_post_run`) right before the Bootstrap block, so list_or_shoot routes everything it touches into the post-load queue and the loader's existing flush phase runs them after the preload pass has brought up web-app-mailu.
sys-service-loader's own queue-init task is gone (the constructor is
the sole opener of the context), list_or_shoot's append falls back
to `default([])` for standalone callers, and the two facts lose
their leading underscore because they are now public play-level state
owned by the constructor rather than a role-internal convention.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The lint `test_all_roles_have_run_once_tag` flags every `include_role:
name: sys-service-loader` (used with `tasks_from: list_or_shoot.yml`)
because main.yml lacks a `run_once_sys_service_loader` gate.
The loader is the orchestrator entry point and intentionally
re-runnable; per-service idempotency is enforced inside
`load_service.yml` via each loaded role's own `run_once_<role>` flag.
Mark main.yml with `# nocheck: run-once` to opt out of the lint with
the reason inline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames the shared-services loader role from sys-utils-service-loader to
sys-service-loader (shorter, no orphaned "utils" prefix; entity name in
meta/services.yml realigns to service-loader per get_entity_name).
Introduces an explicit two-phase contract in the loader: after the
ordered preload pass, flush a `_service_loader_post_run` queue that
service roles register into during preload via the new helper
`list_or_shoot.yml` (list during preload, shoot directly outside it).
Replaces the `when: not _service_loader_preloading` guards in
sys-svc-compose and sys-svc-webserver-core with helper calls. Closes
the gap where non-web deploys would never load these post-load
dependencies; in normal web deploys behavior stays the same thanks to
each sub-role's `run_once_<role>` gate.
Docs, comments, test docstring, and constructor-stage invocation
follow the rename. Historical changelogs (CHANGELOG.md, packaging/*)
intentionally left at the original name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5