Files
computer-playbook/tests/unit/cli/build/test_graph.py
T
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00

249 lines
7.2 KiB
Python

import os
import json
import shutil
import tempfile
import unittest
from io import StringIO
from contextlib import redirect_stdout
from cli.build.graph import (
load_meta,
load_tasks,
build_mappings,
output_graph,
ALL_KEYS,
)
class TestGraphHelpers(unittest.TestCase):
def setUp(self) -> None:
self.tmpdir = tempfile.mkdtemp()
self.addCleanup(lambda: shutil.rmtree(self.tmpdir, ignore_errors=True))
def _write_file(self, rel_path: str, content: str) -> str:
path = os.path.join(self.tmpdir, rel_path)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(content)
return path
def test_load_meta_parses_run_after_and_dependencies(self):
meta_path = self._write_file(
"roles/role_a/meta/main.yml",
"""
galaxy_info:
author: Test Author
dependencies:
- role_d
- role_e
""",
)
# Per req-010 run_after lives in meta/services.yml under the role's
# primary entity; for "role_a" (no category prefix) the entity name
# is the role name itself.
self._write_file(
"roles/role_a/meta/services.yml",
"""
role_a:
run_after:
- role_b
- role_c
""",
)
meta = load_meta(meta_path)
self.assertIn("galaxy_info", meta)
self.assertEqual(meta["galaxy_info"]["author"], "Test Author")
self.assertEqual(meta["run_after"], ["role_b", "role_c"])
self.assertEqual(meta["dependencies"], ["role_d", "role_e"])
def test_load_tasks_filters_out_jinja_and_reads_names(self):
tasks_path = self._write_file(
"roles/role_a/tasks/main.yml",
"""
- name: include plain file
include_tasks: "subtasks.yml"
- name: include with dict
include_tasks:
name: "other.yml"
- name: include jinja, should be ignored
include_tasks: "{{ dynamic_file }}"
- name: import plain file
import_tasks: "legacy.yml"
- name: import with dict
import_tasks:
name: "more.yml"
- name: import jinja, should be ignored
import_tasks: "{{ legacy_file }}"
""",
)
include_files = load_tasks(tasks_path, "include_tasks")
import_files = load_tasks(tasks_path, "import_tasks")
self.assertEqual(sorted(include_files), ["other.yml", "subtasks.yml"])
self.assertEqual(sorted(import_files), ["legacy.yml", "more.yml"])
class TestBuildMappings(unittest.TestCase):
def setUp(self) -> None:
self.roles_dir = tempfile.mkdtemp()
self.addCleanup(lambda: shutil.rmtree(self.roles_dir, ignore_errors=True))
def _write_file(self, rel_path: str, content: str) -> str:
path = os.path.join(self.roles_dir, rel_path)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(content)
return path
def _create_minimal_role(self, name: str, with_meta: bool = False) -> None:
os.makedirs(os.path.join(self.roles_dir, name), exist_ok=True)
if with_meta:
self._write_file(
f"{name}/meta/main.yml",
"""
galaxy_info:
author: Minimal
""",
)
def test_build_mappings_collects_all_dependency_types(self):
# Create roles directory structure
self._create_minimal_role("role_b")
self._create_minimal_role("role_c")
self._create_minimal_role("role_d")
self._create_minimal_role("role_e")
# Role A with meta (dependencies); run_after now lives in services.yml
self._write_file(
"role_a/meta/main.yml",
"""
galaxy_info:
author: Role A Author
dependencies:
- role_c
""",
)
self._write_file(
"role_a/meta/services.yml",
"""
role_a:
run_after:
- role_b
""",
)
# Role A tasks with include_role, import_role, include_tasks, import_tasks
self._write_file(
"role_a/tasks/main.yml",
"""
- name: use docker style role
include_role:
name: role_d
- name: use import role
import_role:
name: role_e
- name: include static tasks file
include_tasks: "subtasks.yml"
- name: import static tasks file
import_tasks:
name: "legacy.yml"
""",
)
# Dummy tasks/meta for other roles not required, but create dirs so they
# are recognized as roles.
self._create_minimal_role("role_a") # dirs already exist but harmless
graphs = build_mappings("role_a", self.roles_dir, max_depth=2)
# Ensure we got all expected graph keys
for key in ALL_KEYS:
self.assertIn(key, graphs, msg=f"Missing graph key {key!r} in result")
# Helper to find links in a graph
def links_of(key: str):
return graphs[key]["links"]
# run_after_to: role_a -> role_b
run_after_links = links_of("run_after_to")
self.assertIn(
{"source": "role_a", "target": "role_b", "type": "run_after"},
run_after_links,
)
# dependencies_to: role_a -> role_c
dep_links = links_of("dependencies_to")
self.assertIn(
{"source": "role_a", "target": "role_c", "type": "dependencies"},
dep_links,
)
# include_role_to: role_a -> role_d
inc_role_links = links_of("include_role_to")
self.assertIn(
{"source": "role_a", "target": "role_d", "type": "include_role"},
inc_role_links,
)
# import_role_to: role_a -> role_e
imp_role_links = links_of("import_role_to")
self.assertIn(
{"source": "role_a", "target": "role_e", "type": "import_role"},
imp_role_links,
)
# include_tasks_to: role_a -> "subtasks.yml"
inc_tasks_links = links_of("include_tasks_to")
self.assertIn(
{"source": "role_a", "target": "subtasks.yml", "type": "include_tasks"},
inc_tasks_links,
)
# import_tasks_to: role_a -> "legacy.yml"
imp_tasks_links = links_of("import_tasks_to")
self.assertIn(
{"source": "role_a", "target": "legacy.yml", "type": "import_tasks"},
imp_tasks_links,
)
def test_output_graph_console_prints_header_and_yaml(self):
graph_data = {"nodes": [{"id": "role_a"}], "links": []}
buf = StringIO()
with redirect_stdout(buf):
output_graph(graph_data, "console", "role_a", "include_role_to")
out = buf.getvalue()
self.assertIn("--- role_a_include_role_to ---", out)
self.assertIn("nodes:", out)
self.assertIn("role_a", out)
def test_output_graph_writes_json_file(self):
graph_data = {"nodes": [{"id": "role_a"}], "links": []}
# Use current working directory; file is small and cleaned manually.
fname = "role_a_include_role_to.json"
try:
output_graph(graph_data, "json", "role_a", "include_role_to")
self.assertTrue(os.path.exists(fname))
with open(fname, "r", encoding="utf-8") as f:
loaded = json.load(f)
self.assertEqual(graph_data, loaded)
finally:
if os.path.exists(fname):
os.remove(fname)
if __name__ == "__main__":
unittest.main()