Files
computer-playbook/tests/unit/utils/cache/test_applications.py
T
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00

222 lines
7.8 KiB
Python

"""Focused unit tests for ``utils.cache.applications``.
Pins the public API (`get_application_defaults`, `get_variants`,
`get_merged_applications`) plus the cache invariants (per-roles_dir
keying, deep-copy on read), and the strict ansible-free import-time
contract that the GitHub Actions runner-host CLI path depends on.
The matrix-deploy variants loader is exhaustively covered in
test_variants.py; this file owns the module-API contract.
"""
from __future__ import annotations
import sys
import tempfile
import textwrap
import unittest
from pathlib import Path
from utils.cache import _reset_cache_for_tests
from utils.cache import applications as cache_apps
def _write(path: Path, content: str) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(textwrap.dedent(content), encoding="utf-8")
def _seed_minimal_roles(tmp: Path) -> Path:
"""Create a minimal `<tmp>/roles/web-app-foo/meta/...` tree (post req-008)
so applications/variants can resolve a real role.
"""
roles = tmp / "roles"
role = roles / "web-app-foo"
_write(
role / "meta" / "services.yml",
"""
foo:
image: foo
version: latest
""",
)
_write(
role / "meta" / "users.yml",
"""
administrator: {}
""",
)
return roles
class TestGetApplicationDefaults(unittest.TestCase):
def setUp(self) -> None:
_reset_cache_for_tests()
def test_returns_dict_per_role(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
defaults = cache_apps.get_application_defaults(roles_dir=roles)
self.assertIn("web-app-foo", defaults)
self.assertIn("services", defaults["web-app-foo"])
def test_caches_per_roles_dir(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
first = cache_apps.get_application_defaults(roles_dir=roles)
# Mutate the returned copy; the cache MUST hand back a fresh
# deep copy on the next call.
first["web-app-foo"]["mutated"] = True
second = cache_apps.get_application_defaults(roles_dir=roles)
self.assertNotIn("mutated", second["web-app-foo"])
def test_users_block_rewritten_to_lookup_jinja(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
defaults = cache_apps.get_application_defaults(roles_dir=roles)
users = defaults["web-app-foo"]["users"]
self.assertEqual(
users["administrator"],
"{{ lookup('users', 'administrator') }}",
)
class TestGetVariants(unittest.TestCase):
def setUp(self) -> None:
_reset_cache_for_tests()
def test_returns_single_variant_when_no_meta_variants_yml(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
variants = cache_apps.get_variants(roles_dir=roles)
self.assertEqual(len(variants["web-app-foo"]), 1)
def test_returns_multiple_variants_when_meta_variants_yml_lists_them(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
_write(
roles / "web-app-foo" / "meta" / "variants.yml",
"""
- {}
- services:
foo:
image: foo-alt
""",
)
variants = cache_apps.get_variants(roles_dir=roles)
self.assertEqual(len(variants["web-app-foo"]), 2)
# Variant 0 (default) keeps the canonical image; variant 1 has the
# override applied.
self.assertEqual(
variants["web-app-foo"][0]["services"]["foo"]["image"],
"foo",
)
self.assertEqual(
variants["web-app-foo"][1]["services"]["foo"]["image"],
"foo-alt",
)
def test_caches_per_roles_dir(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
first = cache_apps.get_variants(roles_dir=roles)
first["web-app-foo"][0]["mutated"] = True
second = cache_apps.get_variants(roles_dir=roles)
self.assertNotIn("mutated", second["web-app-foo"][0])
class TestBuildRoleBaseConfig(unittest.TestCase):
def setUp(self) -> None:
_reset_cache_for_tests()
def test_empty_meta_yields_empty_dict(self):
with tempfile.TemporaryDirectory() as tmp:
role = Path(tmp) / "roles" / "web-app-empty"
(role / "meta").mkdir(parents=True)
self.assertEqual(
cache_apps._build_role_base_config(role, role.parent),
{},
)
class TestGetMergedApplicationsRespectsOverrides(unittest.TestCase):
"""Smoke test that `get_merged_applications` deep-merges runtime
`applications` overrides on top of the role defaults. The full
rendering pipeline (with templar) is covered in test_data.py and
integration suites; this file pins the contract from the
applications module's perspective.
"""
def setUp(self) -> None:
_reset_cache_for_tests()
def test_runtime_override_wins_over_default(self):
with tempfile.TemporaryDirectory() as tmp:
roles = _seed_minimal_roles(Path(tmp))
merged = cache_apps.get_merged_applications(
variables={
"applications": {
"web-app-foo": {
"services": {"foo": {"image": "override"}},
}
}
},
roles_dir=roles,
templar=None,
)
self.assertEqual(
merged["web-app-foo"]["services"]["foo"]["image"],
"override",
)
class TestApplicationsImportableWithoutAnsible(unittest.TestCase):
"""The CI runner-host CLI path
(`cli.deploy.development.init` -> `plan_dev_inventory_matrix` ->
`get_variants`) MUST stay ansible-free. CI run 24935979190 broke
because `_build_variants` instantiated `ApplicationGidLookup`,
pulling `ansible.plugins.lookup.LookupBase` at call time. This
test pins both the import-time AND call-time invariants by
spawning a fresh subprocess so namespace-package / sys.modules
state cannot leak across tests.
"""
def test_module_imports_and_get_variants_callable_without_ansible(self):
import subprocess
from pathlib import Path
repo_root = Path(__file__).resolve().parents[4]
snippet = (
"import sys\n"
"sys.path.insert(0, %r)\n"
"class _Block:\n"
" def find_spec(self, name, path=None, target=None):\n"
" if name == 'ansible' or name.startswith('ansible.'):\n"
" raise ImportError(f'blocked: {name}')\n"
" return None\n"
"sys.meta_path.insert(0, _Block())\n"
"from utils.cache.applications import get_variants\n"
"assert callable(get_variants)\n"
"from utils.cache.base import ROLES_DIR\n"
"v = get_variants(roles_dir=ROLES_DIR)\n"
"assert len(v) > 0\n"
"print('OK', len(v))\n"
) % str(repo_root)
result = subprocess.run(
[sys.executable, "-c", snippet],
capture_output=True,
text=True,
cwd=str(repo_root),
timeout=60,
)
self.assertEqual(
result.returncode,
0,
msg=f"stderr=\n{result.stderr}\nstdout=\n{result.stdout}",
)
self.assertIn("OK", result.stdout)
if __name__ == "__main__":
unittest.main()