Files
computer-playbook/plugins/lookup/service.py
T
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00

230 lines
7.1 KiB
Python

from __future__ import annotations
from typing import Any, Dict, List, Optional, Set, Tuple
from ansible.errors import AnsibleError
from ansible.plugins.lookup import LookupBase
from utils.applications.config import get
from utils.cache.applications import get_merged_applications
from utils.service_registry import (
build_role_to_primary_service_key,
build_service_registry_from_applications,
canonical_service_key,
equivalent_service_keys,
)
def _get_service_flag(
applications: Dict[str, Any],
app_id: str,
service_keys: List[str],
flag: str,
) -> bool:
return any(
bool(
get(
applications=applications,
application_id=app_id,
config_path=f"services.{service_key}.{flag}",
strict=False,
default=False,
skip_missing_app=True,
)
)
for service_key in service_keys
)
def _get_enabled_service_keys(
applications: Dict[str, Any],
app_id: str,
) -> List[str]:
services = get(
applications=applications,
application_id=app_id,
config_path="services",
strict=False,
default={},
skip_missing_app=True,
)
if not isinstance(services, dict):
return []
return [
k
for k, v in services.items()
if isinstance(v, dict) and v.get("enabled", False)
]
def _resolve_service_provider_app_id(
applications: Dict[str, Any],
service_registry: Dict[str, Any],
service_key: str,
) -> Optional[str]:
entry = service_registry.get(service_key)
if not isinstance(entry, dict):
return None
role = entry.get("role")
if isinstance(role, str) and role:
return role
return None
def _is_service_required(
applications: Dict[str, Any],
service_registry: Dict[str, Any],
app_id: str,
service_key: str,
visited: Set[str],
) -> bool:
"""
Return True if app_id directly or transitively (via its enabled services)
has service_key with both enabled: true AND shared: true.
Uses visited to prevent infinite loops.
Exposed via the ``required`` flag on the ``service`` lookup result.
"""
if app_id in visited:
return False
visited.add(app_id)
service_keys = equivalent_service_keys(service_registry, service_key)
enabled = _get_service_flag(applications, app_id, service_keys, "enabled")
shared = _get_service_flag(applications, app_id, service_keys, "shared")
if enabled and shared:
return True
for svc in _get_enabled_service_keys(applications, app_id):
if svc == service_key:
continue
dep_app_id = _resolve_service_provider_app_id(
applications, service_registry, svc
)
if dep_app_id and dep_app_id in applications:
if _is_service_required(
applications, service_registry, dep_app_id, service_key, visited
):
return True
return False
def _build_role_to_key(service_registry: Dict[str, Any]) -> Dict[str, str]:
return build_role_to_primary_service_key(service_registry)
def _resolve_term(
term: str,
service_registry: Dict[str, Any],
role_to_key: Dict[str, str],
) -> Tuple[str, str]:
"""
Resolve a term (service key or role name) to (service_key, role).
Raises AnsibleError if the term is not a known key or role.
"""
if term in service_registry:
entry = service_registry[term]
role = entry.get("role", "")
return term, str(role)
if term in role_to_key:
key = role_to_key[term]
entry = service_registry[key]
role = entry.get("role", "")
return key, str(role)
raise AnsibleError(
f"service: '{term}' is neither a known service key nor a known role name. "
f"Known keys: {sorted(service_registry)}. "
f"Known roles: {sorted(role_to_key)}."
)
def _compute_flags(
applications: Dict[str, Any],
group_names: List[str],
service_registry: Dict[str, Any],
service_key: str,
) -> Dict[str, bool]:
deployed = [app_id for app_id in group_names if app_id in applications]
equivalent_keys = equivalent_service_keys(service_registry, service_key)
any_enabled = any(
_get_service_flag(applications, app_id, equivalent_keys, "enabled")
for app_id in deployed
)
any_shared = any(
_get_service_flag(applications, app_id, equivalent_keys, "shared")
for app_id in deployed
)
primary_key = canonical_service_key(service_registry, service_key)
any_required = any(
_is_service_required(applications, service_registry, app_id, primary_key, set())
for app_id in deployed
)
return {"enabled": any_enabled, "shared": any_shared, "required": any_required}
class LookupModule(LookupBase):
"""
Resolve a service by key or role name and return its aggregated deployment flags.
Usage:
lookup('service', 'matomo')
lookup('service', 'web-app-matomo') # resolved via reverse mapping
Reads 'applications' and 'group_names' from Ansible variables and discovers
service providers from the role-local services metadata.
Returns a dict per term:
id — canonical service key (e.g. 'matomo')
role — provider role name (e.g. 'web-app-matomo')
enabled — True if any deployed app has services.<key>.enabled: true
shared — True if any deployed app has services.<key>.shared: true
required — True if any deployed app has both enabled AND shared (direct
or transitively via its own enabled service dependencies).
"Required" was chosen over "needed" to express that the
service is contractually required by a real consumer, not
merely convenient.
"""
def run(
self,
terms: List[Any],
variables: Optional[Dict[str, Any]] = None,
**kwargs: Any,
) -> List[Dict[str, Any]]:
if not terms:
return []
vars_ = variables or getattr(self._templar, "available_variables", {}) or {}
applications = get_merged_applications(
variables=vars_,
roles_dir=kwargs.get("roles_dir"),
templar=getattr(self, "_templar", None),
)
group_names = vars_.get("group_names", [])
if not isinstance(group_names, list):
raise AnsibleError(
"service: required variable 'group_names' must be a list"
)
service_registry = build_service_registry_from_applications(applications)
role_to_key = _build_role_to_key(service_registry)
results: List[Dict[str, Any]] = []
for term in terms:
term_str = str(term).strip()
if not term_str:
raise AnsibleError("service: service key/role must not be empty")
service_key, role = _resolve_term(term_str, service_registry, role_to_key)
flags = _compute_flags(
applications, group_names, service_registry, service_key
)
results.append({"id": service_key, "role": role, **flags})
return results