mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-10 04:36:40 +00:00
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
306 lines
11 KiB
Python
306 lines
11 KiB
Python
"""Focused unit tests for ``utils.cache.users``.
|
|
|
|
Pins the public API (`get_user_defaults`, `get_merged_users`) plus the
|
|
internal helpers that only this module owns: `_load_user_defs`,
|
|
`_build_users`, `_compute_reserved_usernames`, `_load_store_users`,
|
|
`_resolve_tokens_file`, `_hydrate_users_tokens`, `_merge_users`,
|
|
`_materialize_builtin_user_aliases`. Module name is `test_users_module`
|
|
to disambiguate from the existing `tests/unit/plugins/lookup/test_users.py`
|
|
which targets the lookup plugin.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import tempfile
|
|
import textwrap
|
|
import unittest
|
|
from collections import OrderedDict
|
|
from pathlib import Path
|
|
|
|
import yaml
|
|
|
|
from utils.cache import _reset_cache_for_tests
|
|
from utils.cache import base as cache_base
|
|
from utils.cache import users as cache_users
|
|
|
|
|
|
def _write(path: Path, content: str) -> None:
|
|
path.parent.mkdir(parents=True, exist_ok=True)
|
|
path.write_text(textwrap.dedent(content), encoding="utf-8")
|
|
|
|
|
|
def _seed_minimal_user_role(tmp: Path, role_name: str = "web-app-foo") -> Path:
|
|
# Per req-008 the file root of meta/users.yml IS the users map
|
|
# (no `users:` wrapper).
|
|
roles = tmp / "roles"
|
|
role = roles / role_name
|
|
_write(role / "meta" / "services.yml", "{}\n")
|
|
_write(
|
|
role / "meta" / "users.yml",
|
|
f"""
|
|
{role_name.split("-")[-1]}:
|
|
description: "test-only user"
|
|
""",
|
|
)
|
|
return roles
|
|
|
|
|
|
class TestComputeReservedUsernames(unittest.TestCase):
|
|
def test_extracts_lowercase_alnum_suffix(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = Path(tmp)
|
|
(roles / "web-app-postgres").mkdir(parents=True)
|
|
(roles / "web-app-mailu").mkdir(parents=True)
|
|
(roles / "svc-db-mariadb").mkdir(parents=True)
|
|
(roles / "not-a-role.txt").write_text("", encoding="utf-8")
|
|
self.assertEqual(
|
|
cache_users._compute_reserved_usernames(roles),
|
|
["mailu", "mariadb", "postgres"],
|
|
)
|
|
|
|
|
|
class TestLoadUserDefs(unittest.TestCase):
|
|
def test_aggregates_users_across_roles(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = _seed_minimal_user_role(Path(tmp), "web-app-alpha")
|
|
_write(
|
|
roles / "web-app-beta" / "meta" / "users.yml",
|
|
"""
|
|
beta:
|
|
description: "beta user"
|
|
""",
|
|
)
|
|
defs = cache_users._load_user_defs(roles)
|
|
self.assertIn("alpha", defs)
|
|
self.assertIn("beta", defs)
|
|
|
|
def test_conflicting_definitions_raise(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = Path(tmp) / "roles"
|
|
_write(
|
|
roles / "web-app-a" / "meta" / "users.yml",
|
|
"""
|
|
shared: {description: from-a}
|
|
""",
|
|
)
|
|
_write(
|
|
roles / "web-app-b" / "meta" / "users.yml",
|
|
"""
|
|
shared: {description: from-b}
|
|
""",
|
|
)
|
|
with self.assertRaisesRegex(ValueError, "Conflict for user 'shared'"):
|
|
cache_users._load_user_defs(roles)
|
|
|
|
|
|
class TestBuildUsers(unittest.TestCase):
|
|
def test_allocates_distinct_uids_for_users_without_explicit_uid(self):
|
|
defs: OrderedDict[str, dict] = OrderedDict(
|
|
[
|
|
("alice", {}),
|
|
("bob", {}),
|
|
]
|
|
)
|
|
users = cache_users._build_users(
|
|
defs, primary_domain="x.example", start_id=1001, become_pwd="pw"
|
|
)
|
|
self.assertEqual(users["alice"]["uid"], 1001)
|
|
self.assertEqual(users["bob"]["uid"], 1002)
|
|
|
|
def test_explicit_uid_is_honoured(self):
|
|
defs = OrderedDict(
|
|
[
|
|
("alice", {"uid": 5000}),
|
|
("bob", {}),
|
|
]
|
|
)
|
|
users = cache_users._build_users(
|
|
defs, primary_domain="x.example", start_id=1001, become_pwd="pw"
|
|
)
|
|
self.assertEqual(users["alice"]["uid"], 5000)
|
|
self.assertEqual(users["bob"]["uid"], 1001)
|
|
|
|
def test_duplicate_explicit_uid_raises(self):
|
|
defs = OrderedDict(
|
|
[
|
|
("alice", {"uid": 1500}),
|
|
("bob", {"uid": 1500}),
|
|
]
|
|
)
|
|
with self.assertRaisesRegex(ValueError, "Duplicate uid 1500"):
|
|
cache_users._build_users(
|
|
defs, primary_domain="x.example", start_id=1001, become_pwd="pw"
|
|
)
|
|
|
|
def test_duplicate_username_after_overrides_raises(self):
|
|
defs = OrderedDict(
|
|
[
|
|
("alice", {"username": "shared"}),
|
|
("bob", {"username": "shared"}),
|
|
]
|
|
)
|
|
with self.assertRaisesRegex(ValueError, "Duplicate username 'shared'"):
|
|
cache_users._build_users(
|
|
defs, primary_domain="x.example", start_id=1001, become_pwd="pw"
|
|
)
|
|
|
|
def test_default_email_uses_username_at_primary_domain(self):
|
|
users = cache_users._build_users(
|
|
OrderedDict([("alice", {})]),
|
|
primary_domain="example.org",
|
|
start_id=1001,
|
|
become_pwd="pw",
|
|
)
|
|
self.assertEqual(users["alice"]["email"], "alice@example.org")
|
|
|
|
|
|
class TestMergeUsers(unittest.TestCase):
|
|
def test_overrides_extend_defaults(self):
|
|
defaults = {"alice": {"description": "from-defaults"}}
|
|
overrides = {"alice": {"username": "alice-override"}, "bob": {}}
|
|
merged = cache_users._merge_users(defaults, overrides)
|
|
self.assertEqual(merged["alice"]["username"], "alice-override")
|
|
# The defaults description survives when not overridden.
|
|
self.assertEqual(merged["alice"]["description"], "from-defaults")
|
|
self.assertIn("bob", merged)
|
|
|
|
|
|
class TestHydrateUsersTokens(unittest.TestCase):
|
|
def test_store_tokens_fill_empty_in_users_only(self):
|
|
users = {
|
|
"alice": {"tokens": {"web-app-x": ""}},
|
|
"bob": {"tokens": {"web-app-y": "from-user"}},
|
|
}
|
|
store = {
|
|
"alice": {"tokens": {"web-app-x": "from-store"}},
|
|
"bob": {"tokens": {"web-app-y": "store-attempt"}},
|
|
}
|
|
merged = cache_users._hydrate_users_tokens(users, store)
|
|
self.assertEqual(merged["alice"]["tokens"]["web-app-x"], "from-store")
|
|
# User-supplied non-empty token MUST win.
|
|
self.assertEqual(merged["bob"]["tokens"]["web-app-y"], "from-user")
|
|
|
|
def test_returns_deep_copy_independent_of_input(self):
|
|
users = {"alice": {"tokens": {}}}
|
|
store = {"alice": {"tokens": {"x": "y"}}}
|
|
merged = cache_users._hydrate_users_tokens(users, store)
|
|
merged["alice"]["tokens"]["x"] = "mutated"
|
|
# Original `store` is untouched.
|
|
self.assertEqual(store["alice"]["tokens"]["x"], "y")
|
|
|
|
|
|
class TestResolveTokensFile(unittest.TestCase):
|
|
def test_falls_back_to_default_when_no_overrides(self):
|
|
# `_resolve_tokens_file` reads `base.DEFAULT_TOKENS_FILE` at call
|
|
# time, so a test that patches the constant takes effect on the
|
|
# very next call.
|
|
previous = cache_base.DEFAULT_TOKENS_FILE
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
sentinel = Path(tmp) / "sentinel.yml"
|
|
sentinel.write_text("users: {}\n", encoding="utf-8")
|
|
cache_base.DEFAULT_TOKENS_FILE = sentinel
|
|
try:
|
|
result = cache_users._resolve_tokens_file(variables={})
|
|
self.assertEqual(result, sentinel)
|
|
finally:
|
|
cache_base.DEFAULT_TOKENS_FILE = previous
|
|
|
|
def test_explicit_FILE_TOKENS_wins(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
explicit = Path(tmp) / "tokens.yml"
|
|
explicit.write_text("users: {}\n", encoding="utf-8")
|
|
result = cache_users._resolve_tokens_file(
|
|
variables={"FILE_TOKENS": str(explicit)}
|
|
)
|
|
self.assertEqual(result, explicit)
|
|
|
|
|
|
class TestLoadStoreUsers(unittest.TestCase):
|
|
def test_missing_path_returns_empty(self):
|
|
self.assertEqual(cache_users._load_store_users(None), {})
|
|
self.assertEqual(cache_users._load_store_users(""), {})
|
|
|
|
def test_yaml_users_block_passes_through(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
path = Path(tmp) / "tokens.yml"
|
|
path.write_text(
|
|
yaml.safe_dump({"users": {"alice": {"tokens": {"x": "y"}}}}),
|
|
encoding="utf-8",
|
|
)
|
|
result = cache_users._load_store_users(path)
|
|
self.assertEqual(result, {"alice": {"tokens": {"x": "y"}}})
|
|
|
|
|
|
class TestMaterializeBuiltinUserAliases(unittest.TestCase):
|
|
def test_no_primary_domain_returns_users_unchanged(self):
|
|
users = {"alice": {"username": "alice"}}
|
|
out = cache_users._materialize_builtin_user_aliases(
|
|
users, variables={}, templar=None
|
|
)
|
|
self.assertEqual(out, users)
|
|
|
|
def test_sld_alias_username_resolved_from_DOMAIN_PRIMARY(self):
|
|
users = {
|
|
"sld": {"username": "{{ DOMAIN_PRIMARY.split('.') | first }}"},
|
|
}
|
|
out = cache_users._materialize_builtin_user_aliases(
|
|
users,
|
|
variables={"DOMAIN_PRIMARY": "infinito.example"},
|
|
templar=None,
|
|
)
|
|
self.assertEqual(out["sld"]["username"], "infinito")
|
|
|
|
|
|
class TestGetUserDefaults(unittest.TestCase):
|
|
def setUp(self) -> None:
|
|
_reset_cache_for_tests()
|
|
|
|
def test_returns_users_per_role(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = _seed_minimal_user_role(Path(tmp), "web-app-foo")
|
|
defaults = cache_users.get_user_defaults(roles_dir=roles)
|
|
self.assertIn("foo", defaults)
|
|
self.assertEqual(defaults["foo"]["description"], "test-only user")
|
|
|
|
def test_caches_per_roles_dir(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = _seed_minimal_user_role(Path(tmp), "web-app-foo")
|
|
first = cache_users.get_user_defaults(roles_dir=roles)
|
|
first["foo"]["mutated"] = True
|
|
second = cache_users.get_user_defaults(roles_dir=roles)
|
|
self.assertNotIn("mutated", second["foo"])
|
|
|
|
def test_reserved_usernames_added_when_missing(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
# Seed a role whose `meta/users.yml` does NOT declare the
|
|
# role-suffix as a user. Then create an additional bare
|
|
# role directory whose suffix MUST be auto-reserved.
|
|
roles = _seed_minimal_user_role(Path(tmp), "web-app-foo")
|
|
(roles / "svc-db-mariadb").mkdir(parents=True)
|
|
defaults = cache_users.get_user_defaults(roles_dir=roles)
|
|
self.assertIn("mariadb", defaults)
|
|
self.assertTrue(defaults["mariadb"]["reserved"])
|
|
|
|
|
|
class TestGetMergedUsersWithOverrides(unittest.TestCase):
|
|
def setUp(self) -> None:
|
|
_reset_cache_for_tests()
|
|
|
|
def test_runtime_override_wins_for_username(self):
|
|
with tempfile.TemporaryDirectory() as tmp:
|
|
roles = _seed_minimal_user_role(Path(tmp), "web-app-foo")
|
|
merged = cache_users.get_merged_users(
|
|
variables={
|
|
"users": {
|
|
"foo": {"username": "override-user"},
|
|
}
|
|
},
|
|
roles_dir=roles,
|
|
templar=None,
|
|
)
|
|
self.assertEqual(merged["foo"]["username"], "override-user")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|