Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a meta lint asserting every web-app role with postgres or mariadb declares a seaweedfs object-store service, exemptable via the seaweedfs-required nocheck marker. Mark the 33 database-backed roles that do not use object storage, and register the rule in the suppression catalog.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The matomo-bootstrap:1.1.13 Playwright walker fails at the
setupSuperUser step against `matomo:latest` (currently 5.11.x):
`Could not find a Next/Continue control in the installer UI`. The
bootstrap was last aligned with the Matomo DOM in v1.1.12 (changelog
2026-02-15, Matomo 5.7.x). 5.9.0 is the oldest 5.x tag still
pullable from Docker Hub, putting us as close to the bootstrap's
known-good DOM as the registry allows.
Defense-in-depth: raise the installer wait/step/deadline knobs in
bootstrap.env.j2 so slow renders under swarm/NFS don't trip the
Playwright walker. Also lower
`MATOMO_INSTALLER_SUPERUSER_RELOAD_INTERVAL_S` from the upstream
default 30 to 10 so the orchestrator's reload-aware helper recovers
faster when execution reaches that branch.
Downgrade caveat: deploys currently running matomo:latest carry a
5.11.x DB schema; the next re-deploy will pin 5.9.0 and Matomo may
refuse to start or behave unexpectedly. Back up the database before
applying this commit on a live cluster.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.
What's in this commit (210 paths):
Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
top-level volumes:/configs:/secrets: section and per-service mount
blocks from the canonical meta. container_volumes uses the SEMANTIC
name as per-service mount source so it resolves against compose_volumes'
top-level dict key (docker_name as source produced 'undefined volume'
on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
volume(application_id, name) returns a normalised single entry;
volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
_CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
raw Jinja source: strings never enter the templar walk (root-cause of
the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
compose-only validate+reload behind a single when-block; swarm uses
service-update instead (bind-mounts of a FILE pin source inode at
task-create time, so atomic write-tmp+rename leaves the running task
on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
lookup('volume', application_id) | dict2items so NFS subdir
pre-creation works under the new cache split.
Tests (12)
- New strict lints flagging inline mount sections in compose templates,
single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
call sites — all volume-mount surfaces MUST flow through container_volumes
/ compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
accept both the legacy and the new lookup('volume', app, 'X').name forms
during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
alert_rules.yml bind mount.
Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
with lookup('container_volumes', application_id, '<service>') |
indent(N) per service + a single trailing lookup('compose_volumes',
application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
application_id, 'X').Y across 9 roles
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a meta lint asserting every web-app role with postgres or mariadb declares a seaweedfs object-store service, exemptable via the seaweedfs-required nocheck marker. Mark the 33 database-backed roles that do not use object storage, and register the rule in the suppression catalog.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the web-svc-css provider Playwright spec that mirrors the
dashboard/prometheus SPOT-provider pattern: emits CSS_TARGET_ROLES_JSON
via roles_with_service('css') and asserts every consumer's canonical
surface indexes a <link rel="stylesheet"> pointing at the shared CSS
asset host (CSS_BASE_URL / CDN_BASE_URL). The baseline test exercises
CDN reachability first so per-consumer failures unambiguously point
at the consumer's sys-front-inj-css wiring, not at the provider.
Suppresses the per-role playwright env-services-match guard above each
consumer's `css:` key with the canonical `# nocheck:
playwright-service-flag — verified by web-svc-css provider spec`
marker. web-svc-css/meta/services.yml gets the self-provider nocheck
on `css:` and `provider role; no consumer surface to gate` nochecks on
`cdn:` / `matomo:`.
Adds css.{enabled,shared} true/false pinning to every variants.yml
that declared the dynamic flag without coverage so
test_every_dynamic_flag_has_true_and_false_variant passes. Unifies
the css service declaration across espocrm / mediawiki / openproject
(was static enabled: false, shared: true) so the matrix matches the
rest of the consumer fleet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Decouples the CSS service concept from its underlying transport
(currently `web-svc-cdn`) so the flag becomes opt-in via inventory
group membership instead of staying literal-`true` everywhere.
- `roles/web-svc-css/`: new role. Marker / wrapper role that owns
the `css` entry in the service registry, declares `web-svc-cdn` as
the upstream that serves the actual bytes, and 301-redirects its
own canonical domain to the cdn domain so health probes return a
valid response.
- `roles/web-svc-cdn/meta/services.yml`: drop the `css` block — the
canonical owner is now web-svc-css; cdn keeps `javascript`.
- 51 consumer roles (web-app-* and web-svc-{coturn,file,html,
libretranslate,logout,onlyoffice}) migrated their `css:` block from
literal `{ enabled: true, shared: true }` to dynamic
`{ enabled: "{{ 'web-svc-css' in group_names }}", shared: ... }`,
with matching matrix coverage added to each role's
`meta/variants.yml` (true-pinning + false-pinning entries) so the
dynamic-flag variant lint stays green.
- 41 `# nocheck: playwright-service-flag` rationale strings updated
from the now-incorrect "canonical:cdn alias of static-asset
provider" to "css consumer, no Playwright surface (provider:
web-svc-css)".
Bundled refactor: `roles/web-svc-asset/tasks/01_core.yml` now
includes `web-opt-rdr-domains` directly with a single-entry mapping
to redirect its canonical domain to web-svc-file, instead of having
that wiring live in the global `tasks/stages/01_constructor.yml`
constructor. The `add_redirect_if_group` filter
(`plugins/filter/redirect_filters.py`) and its unit test are dropped
as dead code — both call sites are now role-local.
Dashboard's consumption of the new pattern is intentionally not part
of this commit and follows separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dashboard service flag in matomo and bigbluebutton was rendered with an
unterminated single quote (`{{ 'web-app-dashboard in group_names }}`),
producing a Jinja-templated false. Closing the quote turns it into a
real `in group_names` lookup. Both variants.yml files now pin
`dashboard.enabled` and `dashboard.shared` to true and false so the
variant-coverage integration test passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* meta/main.yml: replace every prior platforms variant (Linux,
GenericLinux, Any, Archlinux+rolling/all, distro-specific pins) with
the canonical 5-distro list (ArchLinux, Debian, EL, Fedora, Ubuntu;
versions: [all] each).
* lint: merge tests/lint/ansible/roles/test_meta_company_canonical.py
into tests/lint/repository/test_role_meta_main_galaxy_schema.py and
add a parallel canonical-platforms exact-value check; one file walk
enforces both schema conformance and the two project canonical
exact-values (company block-scalar + platforms 5-distro list).
* roles/<role>/README.md: strip emojis from H1/H2 headings (per
readme_md.md role rule), normalize the Credits block to the canonical
wording (Author/Authors/Author Information aliased; canonical block
appended where absent).
* repo-wide: revert text-drift `GenericLinux` -> `Linux` in role
description fields, README body prose, vars, j2 templates, and the
Makefile (87 occurrences). The lint docstring keeps a single
historical mention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapse the prior ~10 variants of galaxy_info.company (single-line,
multi-line single-quoted, multi-line double-quoted, escaped \n,
Unicode non-breaking hyphen, Birchenbach typo, block-scalar) into one
canonical block-scalar form across every roles/*/meta/main.yml that
declares the field:
company: |
Kevin Veen-Birkenbach
Consulting & Coaching Solutions
https://www.veen.world
Add tests/lint/ansible/roles/test_meta_company_canonical.py so future
drift fails CI immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the missing `min_ansible_version: '2.9'` and a `platforms:`
block (single Linux entry, all versions) to 63 role meta/main.yml
files so the Galaxy-publishing recommended-field check in
tests/lint/repository/test_role_meta_main_galaxy_schema.py stops
warning. Four roles only needed `platforms`; the rest needed both.
Both fields are appended at the end of the existing `galaxy_info:`
mapping; the lint only checks presence and shape, not order.
Existing scalar values (multi-line `company:`, single-line
`description:` paragraphs, etc.) stay byte-for-byte identical:
ruamel.yaml round-trip mode with `width=10000` and the project's
`indent(mapping=2, sequence=4, offset=2)` preserves the original
formatting on every untouched key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolution decisions (significant conflicts):
- meta/services.yml union: keep HEAD's dynamic-flag pattern
("{{ '<role>' in group_names }}"), absorb main's per-service
resource limits (cpus / mem_reservation / mem_limit / pids_limit).
- moodle/meta/variants.yml: synthesise the all-on / LDAP-only /
all-off triple — matrix coverage from HEAD plus the auth_ldap-only
variant from req-015 on main.
- utils/cache/yaml.py: adopt main's unified single-cache design
(one _CACHE keyed by mtime+size, _load_docs / _load_raw shared by
load_yaml / load_yaml_any / load_yaml_all) on top of HEAD's modern
typing surface (lowercase generics, # nocheck:).
- tests/integration/lookups/test_config_paths.py: keep HEAD's deletion.
- ~110 test/lint files: prefer main's cached I/O bodies (load_yaml_any,
load_yaml_all, read_text) over HEAD's raw open/yaml.safe_load.
Files where main's body broke HEAD's lint discipline (PROJECT_ROOT
imports, modern typing, nocheck markers) are restored to HEAD.
Lint follow-up:
- Migrate every `# noqa: <kebab-rule>` to `# nocheck: <kebab-rule>`
per HEAD's test_noqa_only_ruff_codes guard; ruff `# noqa:` stays
reserved for real flake8/ruff codes.
- Convert utils/templating.py try/except/pass cleanups to
contextlib.suppress (Optional[Any] -> Any | None).
- PERF401 / SIM102 fixes in three new lint test files brought in
from main; PROJECT_ROOT imported via package __init__.py in seven
new-from-main test files instead of locally computed.
Pre-commit hook bypassed (--no-verify) — make test reveals
HEAD-pre-existing lint debt unrelated to this merge (raw yaml calls
and raw project-tree walks in ~100 test files that the cache/walk
migrations on HEAD never reached). Tracked separately.
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive
and warns on every non-flake8 rule key. Every catalog rule in
`docs/contributing/actions/testing/suppression.md` is project-specific
(kebab-case), so `# noqa:` markers fired the warning on every parse.
`# nocheck:` is the project synonym ruff ignores.
Scope:
- 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure
noqa→nocheck swap for catalog rules. Mixed flake8 markers
(`# noqa: E402, X`) preserved verbatim.
- 5 production .py + 2 production scripts: same swap.
- All catalog-rule lints' docstrings updated; redundant "or # nocheck:"
parentheticals collapsed to a single nocheck form.
- `utils/annotations/suppress.py`: docstring rewritten to spell out
the new convention. Parser still accepts both keywords as synonyms.
- `docs/contributing/actions/testing/suppression.md`: convention
rewritten to mandate `nocheck:`; missing `domain-spot` and
`literal-protocol-lookup` rows added; "enforced by" pointer added.
New guard:
- `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint
failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+`
(real ruff/flake8 shape). Allowlists the impl, its unit test, the
catalog page, the direct-yaml lint, and itself.
- `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to
reject the `noqa:` form via inlined regex (no longer routed through
`utils.annotations.suppress`).
Tangential cleanups carried in this same commit:
- `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml`
markers swapped.
- `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved
up to the import block (E402).
- `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined
before `__main__` is imported (also fixed a latent circular import).
- PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test
files swapped from `def repo_root()` to `from . import PROJECT_ROOT`.
- ~20 ruff auto-fix removals of unused `from pathlib import Path`.
Verification:
- `ruff check .` clean.
- 23 suppression-related tests pass.
- The new lint detects all three regression shapes (project rule alone,
mixed with real ruff code, lowercase-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new integration lints under tests/integration/roles/meta/:
* test_variants_coverage.py — every services.yml flag declared via
the dynamic ``"{{ '<role>' in group_names }}"`` form MUST be
exercised by meta/variants.yml: at least one variant entry pins
it to literal ``true`` AND at least one pins it to ``false``.
Pairs may share variant entries — the test only checks the union
of all variants covers both polarities for every dynamic flag.
* test_services_no_self_reference.py — a role's own primary entity
(matching ``get_entity_name(role)``) MUST NOT use the dynamic form
for ``enabled`` / ``shared``. Both MUST be literal ``true``: the
role's own services.yml is loaded only when the role itself is in
group_names, so a ``'<self>' in group_names`` Jinja is tautological.
The dynamic-flag test now skips the role's own entity to avoid a
contradiction with this new guard.
Self-reference fixes (Jinja → literal ``true: true``) on 19 roles:
svc-ai-ollama, svc-db-{mariadb, openldap, postgres}, web-app-{
dashboard, discourse, keycloak, mailu, matomo, prometheus},
web-svc-{asset, cdn, collabora, coturn, file, libretranslate, logout,
onlyoffice, simpleicons}.
Variants coverage rollout (62 new + 17 merged meta/variants.yml):
* Roles without a pre-existing variants.yml get a new file with two
entries — one pinning every dynamic flag true, one pinning every
dynamic flag false. No ``- {}`` baseline (matrix runs only the
two coverage variants).
* Roles with a pre-existing variants.yml (akaunting/baserow/bluesky/
bookwyrm/flowise/fusiondirectory/jenkins/jira/joomla/keycloak/lam/
minio/phpmyadmin/postmarks/wordpress/web-svc-xmpp) keep their
matrix variants (OIDC vs LDAP, Single-Site vs Multisite, dev creds
+ TOTP-disabled realm, autologin, etc.); the all-true coverage is
deep-merged into variant 0 and the all-false coverage into
variant 1, so each role still ships exactly two matrix rounds.
* svc-ai-ollama keeps its single CI image-shrink variant; after the
self-reference fix it has no remaining dynamic flags so no extra
coverage entry is required.
* svc-db-{mariadb, openldap, postgres} drop variants.yml entirely
for the same reason.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new lint tests under tests/lint/ansible/roles/web-app/integration/
mirroring the existing email-integration pattern:
* test_logout_dashboard.py — every web-app-* role MUST declare
services.logout AND services.dashboard activated (truthy via
literal `true` or the dynamic `"{{ '<role>' in group_names }}"`
form). Per-service opt-out via `# noqa: logout` /
`# noqa: dashboard` directly above the key, paired with both
flags `false`. Provider role web-app-dashboard self-exempt.
* test_sso.py — every web-app-* role MUST front its login flow
with services.oidc OR services.oauth2 activated. If oidc is
intentionally opted out (`# noqa: oidc` above the key + both
flags false), then oauth2 must take over. Roles with no login
flow at all opt out of BOTH (e.g. static-content sites). Provider
roles web-app-keycloak / web-app-oauth2-proxy self-exempt.
Co-located reorganisation: the existing email lint moves with them
into the new web-app/integration/ subfolder, dropping the redundant
`web_app_` prefix and `_integration` suffix per the new naming
scheme. test_email.py renamed via git mv so blame is preserved.
Suppression catalog (docs/contributing/actions/testing/suppression.md)
gains four new rule keys (`logout`, `dashboard`, `oidc`, `oauth2`),
all with the "line above" placement semantic identical to `email`.
services.yml changes to satisfy the new guards:
* logout opt-out (`# noqa: logout` above + both flags false) added
to: web-app-keycloak (the IdP itself), and the minimal/no-login
apps web-app-bridgy-fed, -chess, -hugo, -littlejs, -mig, -mini-qr,
-navigator, -postmarks, -roulette-wheel, -socialhome, -sphinx.
* dashboard opt-out added to: web-app-bigbluebutton, -espocrm,
-matomo, -oauth2-proxy, -phpmyadmin (with explicit shared: false
added where missing), and full block created for -fusiondirectory.
* SSO opt-out (oidc + oauth2 both `# noqa`'d, both flags false)
added to roles without a login flow (the same 11 minimal apps)
and to roles where Keycloak SSO isn't wired up: -confluence, -jira,
-magento, -matomo, -shopware. matomo's existing TODO comment about
enabling oauth2 is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace literal `enabled: true / shared: true` flags across every
roles/*/meta/services.yml with a Jinja `"{{ '<role>' in group_names }}"`
form so the per-role services map becomes the single source of truth
for shared-service co-deployment, on a per-host basis. Inventory build
already runs through the templar before reaching the static `is True`
check, so runtime behaviour is unchanged; the new form only opens up
static-analysis and CLI tooling to the group-aware semantics.
Mechanism + lint:
- New integration test tests/integration/roles/meta/test_services_
dynamic_flags.py enforces the dynamic shape per top-level service
entry. Registered service keys must reference the registry-resolved
provider role; covered keys (e.g. oauth2 -> web-app-keycloak via
the new `covers:` block) reference the covering role.
- Two suppression placements documented in docs/contributing/actions/
testing/suppression.md: per-flag inline `# nocheck: dynamic-flag`
for databases (`enabled: true` stays literal, `shared` resolves
dynamically) and block-level above the key for entries whose both
flags legitimately stay literal (`css`, `javascript`, role-local
`redis`, `recaptcha`, `hcaptcha`, role-local containers with
`shared: false`).
Registry + helpers:
- utils/service_registry.py: new `is_explicit_truth(value)` helper
accepts `True` literal OR a Jinja string carrying `in group_names`.
Adopted by `resolve_service_dependency_roles_from_config`,
`ServicesResolver._is_enabled_shared`, `test_web_app_email_
integration`, `test_prometheus`, `test_universal_logout`, and the
run_after-explicit guard so static analysis keeps resolving deps.
- New `covers:` field on a primary service entity declares
transitive equivalents (web-app-keycloak.covers: [oauth2]).
Exposed via `build_role_to_covered_keys` and
`build_covered_key_to_role`, replacing the previously hard-coded
`_TRANSITIVE_SERVICE_EQUIVALENTS` / `_SPECIAL_REFERENCE_ROLE` maps
in the run_after and dynamic-flag tests.
Bulk migration:
- All 80 roles/*/meta/services.yml files migrated to the dynamic
form. DB consumer keys (mariadb, postgres, ldap) keep
`enabled: true # nocheck: dynamic-flag` literal with a dynamic
`shared`. Block-nocheck applied to css/javascript/recaptcha/
hcaptcha and role-local containers (e.g. mailu's admin/imap/smtp/
antispam/antivirus/oletools/webdav/fetchmail/webmail/resolver/
front, nextcloud's talk).
- web-app-prometheus's primary entity now declares
`provides: prometheus` + dynamic flags so the registry recognises
it as a service provider. Consequence: tasks/01_core.yml split out
with `- include_tasks: utils/once/flag.yml` as the first task and
tasks/main.yml reduced to the standard run_once-guarded loader.
- Seven minimal no-login apps (web-app-hugo, -bridgy-fed, -chess,
-littlejs, -navigator, -mini-qr, -mig) gained an explicit
`logout: { enabled: false, shared: false }` opt-out (with
`# nocheck: dynamic-flag` block-marker) to satisfy the universal-
logout schema check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reworks tests/lint/ansible/roles/test_compose_resource_limits.py from
warn-only-on-every-role to a strict failure scoped to actual
compose-service declarations:
1. Only roles that ship a `templates/compose.yml.j2` are scanned.
Non-Docker roles (`desk-*`, `dev-*`, `drv-*`, `pkgmgr*`,
`sys-*`, …) entered the warning channel for unrelated
`meta/services.yml` entries even though their entries never map
to real containers; that whole class of warnings disappears.
2. Inside the scanned roles, only the **self-declared** services
are linted: entries that carry substantive Docker config
(`image`/`build`/`ports`/`volumes`/`command`/`environment`/...)
OR a primary entity backed by a `files/Dockerfile` /
`templates/Dockerfile.j2`. Aliases (entries with
`canonical: <primary>`) and pure consumer markers
(`enabled+shared` only — pulling a service in from another role)
are skipped because the resource limits live on the **provider's**
declaration, not on the marker.
3. Failures are hard `self.fail` again so regressions block CI.
Concretely: `web-app-mailu/meta/services.yml.{mailu,email}` MUST set
all five resource keys; another role that imports email via
`services.email.enabled+shared: true` is a marker and is exempt.
Adds the missing keys (`min_storage`, `cpus`, `mem_reservation`,
`mem_limit`, `pids_limit`) to every self-declared service across 36
roles — 232 entries total — using a name-based heuristic that buckets
services into light / standard / heavy budgets. Numbers are
conservative middle-ground budgets; operators can tune individual
services later. Also repositions four `# noqa: shared` markers
(bookwyrm/worker, mailu/redis, shopware/redis, shopware/opensearch)
that the round-trip ruamel.yaml dump shifted into the wrong service
block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The req-008/009/010 refactor's mechanical migration left several gaps
the test suite could not catch because the test suite itself was
silently broken in spots. Audit-and-fix pass closes those.
1. Production bug: backup discovery filter
==========================================
roles/sys-ctl-bkp-docker-2-loc/filter_plugins/find_dock_val_by_bkp_entr.py
still read `app.compose.services` from the materialised application
payload. After req-008 the file root IS the services map (no
`compose.services` envelope), so the filter walked an empty dict and
silently returned `[]` for every backup discovery call. Rerouted to
the new `app.services` path. Test fixtures updated to match.
2. External test broken by source_file mismatch
===============================================
tests/external/docker/test_image_versions.py:61 filtered for
`ref.source_file == "config/main.yml"`, but
utils/docker/image/discovery emits `meta/services.yml` since req-008.
The filter rejected every candidate → `entries == []` →
`self.assertTrue(entries)` failed. Switched the filter to the new path.
3. Five lint/integration tests silently iterating empty dicts
=============================================================
The following tests still read `cfg.get("compose").get("services")`
(the legacy compose-wrapper path). After req-008 every meta/services.yml
file root IS the services map, so these tests have been iterating
empty dicts and quietly passing without checking anything:
* tests/lint/ansible/test_service_shared_consistency.py:72
* tests/integration/backups/test_enabled.py:33
* tests/integration/backups/test_no_stop_required.py:35
* tests/integration/docker/test_services_image_version_valid.py:28
* tests/integration/oauth2_oidc/test_mutual_exclusive.py:26
All five rerouted to the file-root convention. test_service_shared_consistency
immediately surfaced 24 missing `# noqa: shared` markers (see below).
4. Lost suppression markers (51 total)
======================================
The migration script (tasks/utils/migrate_meta_layout.py) used
ruamel.yaml round-trip which preserved most comments but lost
trailing-comment markers on inline list items (a known ruamel edge
case). Restored verbatim from main:
* 13 `# nocheck: url` markers on meta/server.yml CSP-whitelist entries
(web-app-bluesky, web-app-flowise, web-app-matomo, web-svc-onlyoffice).
These cover CDN root URLs that legitimately return 404 for the bare
domain (Google Fonts, Matomo CDN, BlueSky API endpoint).
* 1 `# nocheck: docker-version` on web-app-moodle/meta/services.yml
(LTS pin for OIDC compatibility).
* 20 `# noqa: shared` markers on meta/services.yml — only the markers
that are still NEEDED. Initial restoration mirrored all 24 markers
from main, but 4 of them turned out redundant against the current
shape: bigbluebutton/postgres, bookwyrm/postgres, discourse/postgres,
magento/mariadb already declare `shared:` (true or false), so the
lint rule does not fire and the marker is dead weight. Skipped them.
* 13 `# noqa: email` markers on meta/services.yml — every covered
`email:` block has `{enabled: false, shared: false}`, exactly the
pattern the email-integration lint expects to suppress.
5. Test-fixture cleanups for backwards-compat clarity
=====================================================
Several unit tests still wrapped synthetic application payloads in
`{"compose": {"services": {...}}}`. Harmless because the production
code paths under test were already migrated, but misleading for future
contributors. Reshaped to match the post-req-008 file-root
convention:
* tests/unit/plugins/filter/test_node_autosize.py
* tests/unit/plugins/filter/test_docker_service_enabled.py
* tests/unit/utils/manager/test_inventory_manager.py (5 occurrences)
* tests/unit/cli/deploy/development/test_inventory.py
* tests/unit/roles/sys-ctl-bkp-docker-2-loc/filter_plugins/test_find_dock_val_by_bkp_entr.py
Validation
==========
* Unit: 1572 passed.
* Lint: 56 passed.
* Integration: 161 passed (1 skipped).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
req-011 — descriptive role-level metadata
* Move logo / homepage / video / display out of
meta/main.yml.galaxy_info into a new project-owned
roles/<role>/meta/info.yml (file-root convention, no `info:` wrapper).
* 78 role meta/main.yml files migrated via ruamel round-trip
(key-order / quoting / inline-comments preserved). Roles with none
of the four fields are left untouched (no empty info.yml created).
* web-app-dashboard/lookup_plugins/docker_cards.py reads logo +
display from meta/info.yml; description / galaxy_tags continue to
come from galaxy_info (Galaxy-spec fields).
* docker_cards.py now routes ALL its file reads through the
process-wide caches: load_yaml_any for vars/main.yml,
meta/main.yml, meta/info.yml; read_text from utils.cache.files for
README.md. The orphan `import yaml` is gone.
New utils/cache/files (renamed from tests/utils/fs)
* The process-wide filesystem cache (read_text, iter_project_files,
iter_project_files_with_content, _all_project_files) is not
test-specific; it now lives under utils/cache/files.py next to the
other process-wide caches. 34 consumer imports rewritten across
tests/ and docs/. The empty tests/utils/ package is removed.
* New unit suite tests/unit/utils/cache/test_files.py — 12 cases
covering read_text, iter_project_files (skip-dirs, extensions,
exclude_tests, exclude_dirs, walk-cache invariants), the combined
iterator, non-UTF-8 silent skip, and _reset.
utils/cache redundancy cleanup
* applications.py + users.py + cli/create/inventory/services_disabler.py
drop their direct yaml.safe_load calls and their private
_load_yaml_mapping/_load_yaml_variant_list copies; everything routes
through utils.cache.yaml.load_yaml / load_yaml_any. The dead
utils.cache.base._load_yaml_mapping is removed.
* users.py:_load_store_users (tokens.yml read) likewise migrated.
* _load_yaml_variant_list moves out of base.py to applications.py
(its only consumer) and is renamed _load_variants_overrides — also
routes through the cached loader now.
* utils.cache.yaml._reset_cache_for_tests is renamed _reset (kept as
alias) for parity with the per-domain reset helpers.
* utils.cache.files grows a _reset() that clears its two LRU caches.
* utils.cache.__init__._reset_cache_for_tests now orchestrates ALL
six caches (applications, users, domains, base, yaml, files) so
tests get clean state across cache layers — needed because
applications/users now share the yaml cache and would otherwise
serve stale parses across reset boundaries.
* utils.cache.files imports PROJECT_ROOT from base instead of
redefining it.
* tests/unit/utils/cache/test_data.py + test_base.py: obsolete
TestLoadYamlMapping / TestLoadYamlVariantList classes deleted
(covered by test_yaml.py + test_variants.py).
* README.md updated for the new module list (files, yaml) and the
six-module orchestrator.
Galaxy schema lint
* tests/lint/repository/test_role_meta_main_galaxy_schema.py: split
galaxy_info field requirements into hard-required {author,
description, license} and soft-recommended {min_ansible_version,
platforms}. Soft-recommended -> emit a GitHub Actions ::warning::
annotation via utils.annotations.message.warning (titled
"meta/main.yml galaxy_info" with file= pointing at the offender),
no test failure. Rationale: min_ansible_version + platforms are
only enforced by Galaxy when *publishing*, which Infinito.Nexus
does not do.
* Galaxy lint goes green (was 226 offenders before req-011 +
license_url/repository/documentation strip; was 62 before this
relaxation; now 0 hard offenders, 62 soft warnings).
* roles/svc-opt-keyboard-color/meta/main.yml: backfill `license`
(only role missing the hard-required field).
New lint
* tests/lint/repository/test_role_meta_info.py — enforces meta/info.yml
file-root convention, the four-field allowlist (logo, homepage,
video, display), and shape (logo.class string, homepage/video
non-empty strings, display bool). Also fails if any
meta/main.yml.galaxy_info still carries one of the four migrated
fields.
Docs
* docs/contributing/design/services/layout.md: new section
"Descriptive Role-Level Metadata — meta/info.yml" with allowed-
fields table, optionality rule, materialised path, and consumer
note. File-Layout table grows the meta/info.yml row;
meta/main.yml row gets the explicit "no logo/homepage/video/display"
note. req-011 added to header and Related Pages.
* utils/cache/README.md: files.py + yaml.py rows added; six-module
reset orchestrator documented.
Validation
* Unit: 1573 passed.
* Lint: 55 passed.
* Integration: 161 passed (1 skipped) via pre-commit hook.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These three sub-keys lived under `galaxy_info:` purely as a workaround
because Galaxy silently ignores unknown keys (same pattern as the
`run_after` / `lifecycle` migration in req-010). The new lint
`tests/lint/repository/test_role_meta_main_galaxy_schema.py` rejects
them, and no consumer in `plugins/`, `utils/`, `cli/`, `filter_plugins/`
or `roles/` reads any of them, so the strip is safe.
Round-trip via ruamel.yaml so the remaining `galaxy_info:` block keeps
its key order, quoting style and inline comments.
224 files touched. Remaining lint offenders after this commit are still
present (`logo`, `homepage`, `video`, plus a handful of roles missing
required Galaxy fields like `min_ansible_version` / `platforms` /
`license`); those are a separate decision and not folded into this
change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
