Files
computer-playbook/plugins/lookup/nginx.py
T
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00

205 lines
7.2 KiB
Python

# lookup_plugins/nginx.py
#
# Resolve nginx path configuration (lowercase keys) and (optionally) a domain-specific
# server config path placed under:
#
# <servers_dir>/<protocol>/<domain>.conf
#
# New API (STRICT):
# lookup('nginx', want_path [, domain ])
#
# Examples:
# lookup('nginx', 'files.configuration')
# lookup('nginx', 'files.domain', 'example.com')
# lookup('nginx', 'files.domain', 'example.com', protocol='http')
#
# Notes:
# - want-path is ALWAYS the first positional argument
# - domain is optional and only affects domain-specific keys
# - want= kwarg is intentionally ignored
from __future__ import annotations
import os
from typing import Any, Dict, List, Optional
from ansible.errors import AnsibleError
from ansible.plugins.lookup import LookupBase
from ansible.plugins.loader import lookup_loader
from utils.applications.config import get
from utils.cache.applications import get_merged_applications
from utils.tls_common import as_str, want_get
def _join(*parts: Any) -> str:
cleaned = [str(p).strip() for p in parts if str(p).strip()]
return os.path.join(*cleaned) if cleaned else ""
def _ensure_trailing_slash(p: str) -> str:
p = p.strip()
return p if not p or p.endswith("/") else p + "/"
def _normalize_protocol(value: str) -> str:
v = as_str(value).strip().lower()
if v in ("http", "https"):
return v
raise AnsibleError(
f"nginx: invalid protocol override '{value}' (expected http|https)"
)
def _dir_spec(path: str, mode: str) -> Dict[str, str]:
path = as_str(path).strip()
mode = as_str(mode).strip()
if not path:
raise AnsibleError("nginx: empty path in directories.ensure")
if mode not in ("0700", "0755"):
raise AnsibleError(
f"nginx: invalid mode '{mode}' in directories.ensure (expected 0700|0755)"
)
return {"path": path, "mode": mode}
def _resolve_protocol_via_tls(
*, domain: str, variables: dict, loader: Any, templar: Any
) -> str:
try:
tls_lookup = lookup_loader.get("tls", loader=loader, templar=templar)
except Exception as exc:
raise AnsibleError(f"nginx: failed to load tls lookup: {exc}") from exc
protocol = tls_lookup.run([domain, "protocols.web"], variables=variables)[0]
protocol_s = as_str(protocol).strip().lower()
if protocol_s not in ("http", "https"):
raise AnsibleError(
f"nginx: unexpected protocol '{protocol_s}' for domain '{domain}'"
)
return protocol_s
class LookupModule(LookupBase):
def run(self, terms, variables: Optional[dict] = None, **kwargs):
variables = variables or {}
terms = terms or []
# STRICT API: want-path is mandatory
if len(terms) not in (1, 2):
raise AnsibleError("nginx: requires want_path [, domain]")
want = as_str(terms[0]).strip()
if not want:
raise AnsibleError("nginx: want_path is empty")
domain = as_str(terms[1]).strip() if len(terms) == 2 else ""
applications = get_merged_applications(
variables=variables,
roles_dir=kwargs.get("roles_dir"),
templar=getattr(self, "_templar", None),
)
proxy_app_id = as_str(kwargs.get("proxy_app_id", "svc-prx-openresty")).strip()
if not proxy_app_id:
raise AnsibleError("nginx: proxy_app_id is empty")
www_dir = get(applications, proxy_app_id, "volumes.www", strict=True)
nginx_dir = get(applications, proxy_app_id, "volumes.nginx", strict=True)
www_dir = _ensure_trailing_slash(as_str(www_dir))
nginx_dir = _ensure_trailing_slash(as_str(nginx_dir))
conf_dir = _ensure_trailing_slash(_join(nginx_dir, "conf.d"))
global_dir = _ensure_trailing_slash(_join(conf_dir, "global"))
servers_dir = _ensure_trailing_slash(_join(conf_dir, "servers"))
servers_http_dir = _ensure_trailing_slash(_join(servers_dir, "http"))
servers_https_dir = _ensure_trailing_slash(_join(servers_dir, "https"))
maps_dir = _ensure_trailing_slash(_join(conf_dir, "maps"))
streams_dir = _ensure_trailing_slash(_join(conf_dir, "streams"))
data_html_dir = _ensure_trailing_slash(_join(www_dir, "public_html"))
data_files_dir = _ensure_trailing_slash(_join(www_dir, "public_files"))
data_cdn_dir = _ensure_trailing_slash(_join(www_dir, "public_cdn"))
data_global_dir = _ensure_trailing_slash(_join(www_dir, "global"))
cache_general_dir = "/tmp/cache_nginx_general/"
cache_image_dir = "/tmp/cache_nginx_image/"
ensure: List[Dict[str, str]] = [
_dir_spec(nginx_dir, "0755"),
_dir_spec(conf_dir, "0755"),
_dir_spec(global_dir, "0755"),
_dir_spec(servers_dir, "0755"),
_dir_spec(servers_http_dir, "0755"),
_dir_spec(servers_https_dir, "0755"),
_dir_spec(maps_dir, "0755"),
_dir_spec(streams_dir, "0755"),
_dir_spec(www_dir, "0755"),
_dir_spec(data_html_dir, "0755"),
_dir_spec(data_files_dir, "0755"),
_dir_spec(data_cdn_dir, "0755"),
_dir_spec(data_global_dir, "0755"),
_dir_spec(cache_general_dir, "0700"),
_dir_spec(cache_image_dir, "0700"),
]
resolved: Dict[str, Any] = {
"files": {
"configuration": _join(nginx_dir, "nginx.conf"),
},
"directories": {
"configuration": {
"base": conf_dir,
"global": global_dir,
"servers": servers_dir,
"maps": maps_dir,
"streams": streams_dir,
"http_includes": [
global_dir,
maps_dir,
servers_http_dir,
servers_https_dir,
],
},
"data": {
"www": www_dir,
"well_known": "/usr/share/nginx/well-known/",
"html": data_html_dir,
"files": data_files_dir,
"cdn": data_cdn_dir,
"global": data_global_dir,
},
"cache": {
"general": cache_general_dir,
"image": cache_image_dir,
},
"ensure": ensure,
"ensure_paths": [d["path"] for d in ensure],
},
}
if domain:
protocol_override = kwargs.get("protocol", None)
protocol = (
_resolve_protocol_via_tls(
domain=domain,
variables=variables,
loader=getattr(self, "_loader", None),
templar=getattr(self, "_templar", None),
)
if not protocol_override
else _normalize_protocol(protocol_override)
)
resolved["domain"] = {
"name": domain,
"protocol": protocol,
"protocol_overridden": bool(protocol_override),
}
resolved["files"]["domain"] = _join(servers_dir, protocol, f"{domain}.conf")
return [want_get(resolved, want)]