mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-11 13:17:07 +00:00
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
220 lines
7.5 KiB
Python
220 lines
7.5 KiB
Python
from __future__ import annotations
|
|
|
|
from typing import Any, Dict, Optional
|
|
|
|
from ansible.errors import AnsibleError
|
|
from ansible.plugins.lookup import LookupBase
|
|
|
|
from plugins.lookup.applications import LookupModule as ApplicationsLookup
|
|
from plugins.lookup.domain import LookupModule as DomainLookup
|
|
from plugins.lookup.users import LookupModule as UsersLookup
|
|
|
|
|
|
SYSTEM_EMAIL_PREFIX = "SYSTEM_EMAIL_"
|
|
|
|
# Declared in resolution order so each computed default sees already-resolved
|
|
# predecessors (for example ``port`` depends on ``tls``).
|
|
RESOLUTION_ORDER = (
|
|
"enabled",
|
|
"timeout",
|
|
"external",
|
|
"environment",
|
|
"domain",
|
|
"tls",
|
|
"port",
|
|
"host",
|
|
"auth",
|
|
"start_tls",
|
|
"smtp",
|
|
"from",
|
|
"username",
|
|
"password",
|
|
)
|
|
|
|
_TRUE_STRINGS = frozenset({"true", "yes", "1", "on"})
|
|
_FALSE_STRINGS = frozenset({"false", "no", "0", "off"})
|
|
|
|
|
|
def _short_to_var(short_key: str) -> str:
|
|
return SYSTEM_EMAIL_PREFIX + short_key.upper()
|
|
|
|
|
|
def _as_bool(value: Any) -> bool:
|
|
if isinstance(value, bool):
|
|
return value
|
|
if isinstance(value, (int, float)):
|
|
return bool(value)
|
|
if isinstance(value, str):
|
|
lowered = value.strip().lower()
|
|
if lowered in _TRUE_STRINGS:
|
|
return True
|
|
if lowered in _FALSE_STRINGS:
|
|
return False
|
|
return bool(value)
|
|
|
|
|
|
def _render(value: Any, templar: Optional[Any]) -> Any:
|
|
if templar is None:
|
|
return value
|
|
if not isinstance(value, str) or "{{" not in value:
|
|
return value
|
|
try:
|
|
return templar.template(value, fail_on_undefined=False)
|
|
except Exception:
|
|
return value
|
|
|
|
|
|
class LookupModule(LookupBase):
|
|
def run(
|
|
self,
|
|
terms: Optional[list[Any]],
|
|
variables: Optional[Dict[str, Any]] = None,
|
|
**kwargs: Any,
|
|
) -> list[Any]:
|
|
terms = terms or []
|
|
if len(terms) > 1:
|
|
raise AnsibleError(
|
|
"email: expected 0 or 1 terms: lookup('email'[, application_id])"
|
|
)
|
|
|
|
variables = variables or getattr(self._templar, "available_variables", {}) or {}
|
|
self._kwargs = kwargs
|
|
|
|
templar = getattr(self, "_templar", None)
|
|
resolved: Dict[str, Any] = {}
|
|
for short_key in RESOLUTION_ORDER:
|
|
var_name = _short_to_var(short_key)
|
|
raw = variables.get(var_name)
|
|
if raw is None or raw == "":
|
|
resolved[short_key] = _render(
|
|
self._compute(short_key, resolved, variables), templar
|
|
)
|
|
else:
|
|
resolved[short_key] = _render(raw, templar)
|
|
|
|
if len(terms) == 0:
|
|
return [resolved]
|
|
|
|
application_id = str(terms[0]).strip()
|
|
overrides = self._app_email_overrides(application_id, variables)
|
|
merged = dict(resolved)
|
|
for key, value in overrides.items():
|
|
merged[str(key).lower()] = _render(value, templar)
|
|
return [merged]
|
|
|
|
def _compute(
|
|
self,
|
|
short_key: str,
|
|
resolved: Dict[str, Any],
|
|
variables: Dict[str, Any],
|
|
) -> Any:
|
|
if short_key == "enabled":
|
|
return True
|
|
if short_key == "timeout":
|
|
return "30"
|
|
if short_key == "external":
|
|
group_names = variables.get("group_names") or []
|
|
return bool("web-app-mailu" in group_names)
|
|
if short_key == "environment":
|
|
external = _as_bool(resolved.get("external"))
|
|
tls_enabled = _as_bool(variables.get("TLS_ENABLED"))
|
|
docker_in_container = _as_bool(variables.get("DOCKER_IN_CONTAINER"))
|
|
base = "external" if (external or tls_enabled) else "localhost"
|
|
suffix = "_container" if (docker_in_container and not tls_enabled) else ""
|
|
return base + suffix
|
|
if short_key == "domain":
|
|
return variables.get("DOMAIN_PRIMARY")
|
|
if short_key == "tls":
|
|
external = _as_bool(resolved.get("external"))
|
|
if not external:
|
|
return False
|
|
return _as_bool(variables.get("TLS_ENABLED"))
|
|
if short_key == "port":
|
|
external = _as_bool(resolved.get("external"))
|
|
tls = _as_bool(resolved.get("tls"))
|
|
return 465 if (external and tls) else 25
|
|
if short_key == "host":
|
|
env = resolved.get("environment")
|
|
if env in ("external_container", "localhost", "localhost_container"):
|
|
return "localhost"
|
|
return self._lookup_mailu_domain(variables)
|
|
if short_key == "auth":
|
|
env = resolved.get("environment")
|
|
if env in ("external_container", "localhost"):
|
|
return False
|
|
return _as_bool(resolved.get("tls"))
|
|
if short_key == "start_tls":
|
|
return False
|
|
if short_key == "smtp":
|
|
return True
|
|
if short_key == "from":
|
|
external = _as_bool(resolved.get("external"))
|
|
if external:
|
|
no_reply = self._lookup_no_reply_user(variables)
|
|
if isinstance(no_reply, dict):
|
|
email = no_reply.get("email")
|
|
if email:
|
|
return email
|
|
inventory_hostname = variables.get("inventory_hostname") or "localhost"
|
|
return f"root@{inventory_hostname}.localdomain"
|
|
if short_key == "username":
|
|
return resolved.get("from")
|
|
if short_key == "password":
|
|
no_reply = self._lookup_no_reply_user(variables)
|
|
if isinstance(no_reply, dict):
|
|
tokens = no_reply.get("tokens") or {}
|
|
if isinstance(tokens, dict):
|
|
return tokens.get("web-app-mailu", "") or ""
|
|
return ""
|
|
raise AnsibleError(f"email: unknown key {short_key!r}")
|
|
|
|
def _lookup_mailu_domain(self, variables: Dict[str, Any]) -> Any:
|
|
domain_lookup = DomainLookup()
|
|
domain_lookup._templar = getattr(self, "_templar", None)
|
|
try:
|
|
return domain_lookup.run(["web-app-mailu"], variables=variables)[0]
|
|
except Exception:
|
|
return "localhost"
|
|
|
|
def _lookup_no_reply_user(self, variables: Dict[str, Any]) -> Dict[str, Any]:
|
|
users_lookup = UsersLookup()
|
|
users_lookup._templar = getattr(self, "_templar", None)
|
|
forwarded = {
|
|
k: v for k, v in getattr(self, "_kwargs", {}).items() if k == "roles_dir"
|
|
}
|
|
try:
|
|
entry = users_lookup.run(
|
|
["no-reply", {}], variables=variables, **forwarded
|
|
)[0]
|
|
except AnsibleError:
|
|
return {}
|
|
return entry if isinstance(entry, dict) else {}
|
|
|
|
def _app_email_overrides(
|
|
self,
|
|
application_id: str,
|
|
variables: Dict[str, Any],
|
|
) -> Dict[str, Any]:
|
|
apps = ApplicationsLookup()
|
|
apps._templar = getattr(self, "_templar", None)
|
|
forwarded = {
|
|
k: v for k, v in getattr(self, "_kwargs", {}).items() if k == "roles_dir"
|
|
}
|
|
try:
|
|
entry = apps.run(
|
|
[application_id, {}],
|
|
variables=variables,
|
|
**forwarded,
|
|
)[0]
|
|
except AnsibleError:
|
|
return {}
|
|
if not isinstance(entry, dict):
|
|
return {}
|
|
# Per req-008 the materialised payload exposes services under the
|
|
# bare ``services`` key (no ``compose.services`` envelope).
|
|
services = entry.get("services") or {}
|
|
email = services.get("email") or {} if isinstance(services, dict) else {}
|
|
if not isinstance(email, dict):
|
|
return {}
|
|
return email
|