mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-10 04:36:40 +00:00
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
186 lines
6.7 KiB
Python
186 lines
6.7 KiB
Python
# tests/unit/cli/test_inventory_manager.py
|
|
|
|
import unittest
|
|
import tempfile
|
|
import shutil
|
|
from pathlib import Path
|
|
from unittest.mock import patch
|
|
|
|
from utils.handler.yaml import YamlHandler
|
|
from utils.handler.vault import VaultHandler, VaultScalar
|
|
from utils.manager.inventory import InventoryManager
|
|
from utils.manager.value_generator import ValueGenerator
|
|
|
|
|
|
class TestInventoryManager(unittest.TestCase):
|
|
def setUp(self):
|
|
# Create a temporary directory for role and inventory files
|
|
self.tmpdir = Path(tempfile.mkdtemp())
|
|
|
|
# Patch YamlHandler.load_yaml
|
|
self.load_yaml_patcher = patch.object(
|
|
YamlHandler, "load_yaml", side_effect=self.fake_load_yaml
|
|
)
|
|
self.load_yaml_patcher.start()
|
|
|
|
# Patch VaultHandler.encrypt_string with correct signature
|
|
self.encrypt_patcher = patch.object(
|
|
VaultHandler,
|
|
"encrypt_string",
|
|
new=lambda self, plain, key: f"{key}: !vault |\n encrypted_{plain}",
|
|
)
|
|
self.encrypt_patcher.start()
|
|
|
|
def tearDown(self):
|
|
patch.stopall()
|
|
shutil.rmtree(self.tmpdir)
|
|
|
|
def fake_load_yaml(self, path):
|
|
path = Path(path)
|
|
|
|
# Return schema for meta/schema.yml
|
|
if path.match("*/meta/schema.yml"):
|
|
return {
|
|
"credentials": {
|
|
"plain_cred": {
|
|
"description": "desc",
|
|
"algorithm": "plain",
|
|
"validation": {},
|
|
},
|
|
"nested": {
|
|
"inner": {
|
|
"description": "desc2",
|
|
"algorithm": "sha256",
|
|
"validation": {},
|
|
}
|
|
},
|
|
}
|
|
}
|
|
|
|
# Return application_id for vars/main.yml
|
|
if path.match("*/vars/main.yml"):
|
|
return {"application_id": "testapp"}
|
|
|
|
# Return docker service flags for meta/services.yml. Per req-008 the
|
|
# file root IS the services map (no `compose.services` wrapper).
|
|
if path.match("*/meta/services.yml"):
|
|
return {
|
|
"mariadb": {"enabled": True, "shared": True},
|
|
}
|
|
|
|
# Return empty inventory for inventory.yml
|
|
if path.name == "inventory.yml":
|
|
return {}
|
|
raise FileNotFoundError(f"Unexpected load_yaml path: {path}")
|
|
|
|
def test_load_application_id_missing(self):
|
|
"""Loading application_id without it should raise SystemExit."""
|
|
role_dir = self.tmpdir / "role"
|
|
(role_dir / "vars").mkdir(parents=True)
|
|
(role_dir / "vars" / "main.yml").write_text("{}", encoding="utf-8")
|
|
|
|
with patch.object(YamlHandler, "load_yaml", return_value={}):
|
|
with self.assertRaises(SystemExit):
|
|
InventoryManager(
|
|
role_dir, self.tmpdir / "inventory.yml", "pw", {}
|
|
).load_application_id(role_dir)
|
|
|
|
def test_generate_value_algorithms(self):
|
|
"""
|
|
Verify ValueGenerator.generate_value produces outputs of the expected form
|
|
and contains no dollar signs (bcrypt is escaped).
|
|
"""
|
|
vg = ValueGenerator()
|
|
|
|
# random_hex → 64 bytes hex = 128 chars
|
|
hex_val = vg.generate_value("random_hex")
|
|
self.assertEqual(len(hex_val), 128)
|
|
self.assertTrue(all(c in "0123456789abcdef" for c in hex_val))
|
|
self.assertNotIn("$", hex_val)
|
|
|
|
# sha256 → 64 hex chars
|
|
sha256_val = vg.generate_value("sha256")
|
|
self.assertEqual(len(sha256_val), 64)
|
|
self.assertNotIn("$", sha256_val)
|
|
|
|
# sha1 → 40 hex chars
|
|
sha1_val = vg.generate_value("sha1")
|
|
self.assertEqual(len(sha1_val), 40)
|
|
self.assertNotIn("$", sha1_val)
|
|
|
|
# bcrypt → contains no '$' after escaping
|
|
bcrypt_val = vg.generate_value("bcrypt")
|
|
self.assertNotIn("$", bcrypt_val)
|
|
|
|
# alphanumeric → 64 chars
|
|
alnum = vg.generate_value("alphanumeric")
|
|
self.assertEqual(len(alnum), 64)
|
|
self.assertTrue(alnum.isalnum())
|
|
self.assertNotIn("$", alnum)
|
|
|
|
# base64_prefixed_32 → starts with "base64:"
|
|
b64 = vg.generate_value("base64_prefixed_32")
|
|
self.assertTrue(b64.startswith("base64:"))
|
|
self.assertNotIn("$", b64)
|
|
|
|
# random_hex_16 → 32 hex chars
|
|
hex16 = vg.generate_value("random_hex_16")
|
|
self.assertEqual(len(hex16), 32)
|
|
self.assertTrue(all(c in "0123456789abcdef" for c in hex16))
|
|
self.assertNotIn("$", hex16)
|
|
|
|
def test_apply_schema_and_recurse(self):
|
|
"""
|
|
apply_schema should inject database password and vault plain_cred.
|
|
"""
|
|
# Setup role directory (post-req-008: only meta/ + vars/).
|
|
role_dir = self.tmpdir / "role"
|
|
(role_dir / "meta").mkdir(parents=True, exist_ok=True)
|
|
(role_dir / "vars").mkdir(parents=True, exist_ok=True)
|
|
|
|
# IMPORTANT: files must exist because InventoryManager checks .exists()
|
|
(role_dir / "meta" / "schema.yml").write_text("{}", encoding="utf-8")
|
|
(role_dir / "meta" / "services.yml").write_text("{}", encoding="utf-8")
|
|
(role_dir / "vars" / "main.yml").write_text("{}", encoding="utf-8")
|
|
|
|
# Create empty inventory.yml
|
|
inv_file = self.tmpdir / "inventory.yml"
|
|
inv_file.write_text(" ", encoding="utf-8")
|
|
|
|
# Provide override for plain_cred to avoid SystemExit
|
|
overrides = {"credentials.plain_cred": "OVERRIDE_PLAIN"}
|
|
|
|
# Instantiate manager with overrides
|
|
mgr = InventoryManager(role_dir, inv_file, "pw", overrides=overrides)
|
|
|
|
# IMPORTANT:
|
|
# This unit test is NOT about transitive shared-provider resolution.
|
|
with patch.object(
|
|
InventoryManager, "resolve_schema_includes_recursive", return_value=[]
|
|
):
|
|
# Patch ValueGenerator for predictable outputs
|
|
with patch.object(
|
|
ValueGenerator, "generate_value", side_effect=lambda alg: f"GEN_{alg}"
|
|
):
|
|
result = mgr.apply_schema()
|
|
|
|
apps = result["applications"]["testapp"]
|
|
|
|
# credentials must exist now
|
|
self.assertIn("credentials", apps)
|
|
creds = apps["credentials"]
|
|
|
|
# database_password comes from ValueGenerator.generate_value("alphanumeric")
|
|
self.assertEqual(creds["database_password"], "GEN_alphanumeric")
|
|
|
|
# plain_cred vaulted from override
|
|
self.assertIsInstance(creds["plain_cred"], VaultScalar)
|
|
|
|
# Per req-008 nested credential keys are supported and walked
|
|
# recursively, so nested.inner with `algorithm: sha256` IS vaulted.
|
|
self.assertIsInstance(creds["nested"]["inner"], VaultScalar)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|