Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7.9 KiB
Service Management 🧱
This page is the SPOT for how shared services are declared, discovered, ordered, loaded, and injected.
What Is a Service? 🧩
A service is a reusable dependency that an application enables via
services.<service_key>.
Examples:
web-svc-cdnprovides the primary service keycdnweb-app-keycloakprovidesoidcsvc-db-mariadbprovidesmariadb
Each service entry lives in the provider role's own
meta/services.yml — the file root IS the services map
keyed by <entity_name> (no compose: and no services: wrapper). See
layout.md for the full per-role meta layout.
Role-Local Service Metadata 🏷️
Service providers are self-describing. The provider role owns:
enabledshared- optional
provides - optional
canonical
Example (file root of meta/services.yml):
keycloak:
enabled: false
shared: true
provides: oidc
Canonical aliases are also role-local:
cdn:
enabled: false
shared: true
css:
enabled: true
shared: true
canonical: cdn
javascript:
enabled: true
shared: true
canonical: cdn
Rules:
- The primary service entry is the role entity name returned by
get_entity_name. provides:is only used when the public service name differs from the entity name.canonical:is only used on alias entries that resolve back to the primary service key.frontendvs.backendis derived from the role name prefix, not stored in config.
Service Discovery 🔍
Service discovery is built from role configs, not from a central registry file.
Primary implementation files:
The discovery layer:
- scans role configs from roles/
- discovers provider entries from
services - derives deploy type and loader bucket from the role name
- resolves
provides:andcanonical: - validates and applies
run_after:frommeta/services.yml.<primary_entity>(per req-010)
Load Order 🧮
main.yml is the single loader SPOT for all shared services.
It runs from 01_constructor.yml before the normal application stage.
Global bucket order:
universalworkstationserverweb-svcweb-app
Within the same bucket, ordering is refined by run_after: declared on the
provider role's primary entity in
meta/services.yml (i.e. services.<primary_entity>.run_after,
per req-010).
Rules:
run_after:entries are role names, not service keys.- Cross-type
run_after:is invalid and fails hard. - Later-bucket dependencies are invalid and fail hard.
- Cross-type service dependencies do not need
run_after:because constructor-stage loading already brings backend services up before normal app deployment.
Loading vs Injection 🔀
Service loading and frontend injection are separate mechanisms.
Loading 📥
Loading decides whether the provider role is deployed at all.
- queries the ordered discovered service list
- checks
lookup('service', service_key).required - skips roles already protected by
run_once_* - loads services through load_app.yml
Frontend service probe/load helper:
Injection 🔌
Injection decides whether a deployed app gets extra nginx integration such as dashboard, logout, CSS, or JavaScript hooks.
This stays in:
Injection still reads the current app's services.<feature>.enabled flags.
It does not load provider roles.
Lookup Plugins 🔎
service 🧷
File: service.py
Examples:
lookup('service', 'matomo')
lookup('service', 'oidc')
lookup('service', 'web-svc-cdn')
Returns:
idroleenabledsharedrequired— True when some deployed app has the service with bothenabled: trueANDshared: true, directly or transitively through its own enabled service dependencies. This is the flag consumers should gate on when deciding whether to load, integrate with, or configure the service — a plainenabledflag is not enough on its own, because a service that is enabled but never shared with another app is not actually contractually required by anyone.
Behavior:
- accepts either a service key or a provider role name
- resolves aliases through
canonical - resolves provider roles through discovered primary service keys
- computes
requiredtransitively from enabled shared services
service_registry 📚
File: service_registry.py
Examples:
query('service_registry') | first
query('service_registry', 'ordered') | first
Modes:
- default: full discovered registry mapping
ordered: ordered primary service entries for the service loader
applications_current_play 🧭
File: applications_current_play.py
Builds the current-play application set including:
- group-selected roles
- transitive shared service dependencies
- meta dependencies
Database Services 🗄️
Relational databases are regular services now:
svc-db-mariadbprovidesmariadbsvc-db-postgresprovidespostgres
Applications express database choice directly via the file root of
meta/services.yml:
mariadb:
enabled: true
shared: true
or:
postgres:
enabled: true
shared: false
The lookup('database', ...) API remains as the convenience accessor for database
connection values, but it now resolves the active direct database service from those
role-local keys instead of services.database.type.
Related Files 📁
| File | Purpose |
|---|---|
| service_registry.py | Service discovery, provides, canonical, bucket detection, run_after ordering |
| service_registry.py lookup | Exposes the discovered registry and ordered provider list to Ansible |
| service.py | Resolves service flags and transitive need |
| applications_current_play.py | Builds the current play app graph with shared service deps |
| main.yml | Single shared-service loader SPOT |
| load_service.yml | Per-service load helper used by the central service loader |
| 01_constructor.yml | Calls the service loader during constructor |
| load_app.yml | Run-once role loader |
| test_service_registry.py | Unit tests for discovery, buckets, and run_after ordering |
| test_service.py | Unit tests for lookup('service', ...) |
| test_resolvable.py | Integration checks for discovered service resolution |
| test_canonical.py | Canonical alias consistency checks |
| test_transitive_dependencies.py | Integration coverage for transitive dependency resolution |