refactor(tests): reorganise meta tests + add multi-doc yaml cache

Two intertwined changes that share the same touched files:

1. Move every meta-related integration test under
   `tests/integration/roles/meta/` into one of three sibling
   sub-clusters and shorten the redundant filename prefixes:

   - `services/` — `test_services_dynamic_flags.py` →
     `test_dynamic_flags.py`, `test_services_no_self_reference.py` →
     `test_no_self_reference.py`. The `run_after` cluster moves
     under it (it is a `meta/services.yml` concern, not a
     stand-alone dependency one): `test_services_explicit.py` keeps
     its name, the rest drop the redundant prefix.
   - `variants/` — `test_variants_coverage.py` → `test_coverage.py`,
     plus the two new lints `test_auth_coverage.py` (LDAP / OIDC /
     OAuth2 matrix completeness) and `test_services_match.py`
     (every variant override key MUST exist in `services.yml`).
   - `dependencies/` — relocates the legacy
     `tests/integration/roles/dependencies/` cluster so all
     `meta/`-shaped tests live under one parent.

   Path-index constants (`Path(__file__).resolve().parents[N]`,
   `os.pardir` chains) updated for the deeper nesting.

2. Route every yaml read in the migrated tests through
   `utils.cache.yaml`. The single-doc helper `load_yaml_any` covers
   most of them; the new `load_yaml_all` mirrors `load_yaml_any` for
   the multi-doc case (`yaml.safe_load_all`) and is keyed off the
   same `(path, mtime, size)` signature, with parallel `_drop_path`
   / `_reset` housekeeping. `tests/lint/ansible/variables/
   test_vars_usage_in_yaml.py` is the first user.

READMEs cleaned up to match the new directory structure (path
crosslinks, level-up counts) and four new READMEs added for the
new sub-clusters under the `docs/` README guidelines (RFC 2119,
emojis after headlines, no em dashes, no SPOT self-labels).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

tests/integration/lookups/README.md

@@ -4,9 +4,6 @@ Integration tests for the custom Ansible lookup plugins under `plugins/lookup/`:
 
 Tests in this directory MUST only cover lookup-plugin behavior (schema validation of `lookup('config', …)` paths, caching semantics, performance smoke). Pure unit tests for a single lookup plugin's return values MUST live under `tests/unit/plugins/lookup/`.
 
-`config/` holds the `lookup('config', …)` path-validation suite, split SRP-style:
-`test_literal_paths.py`, `test_variable_paths.py`, `test_wildcard_paths.py`,
-`test_role_local_paths.py`. All four share a single project scan via
-`_scan.get_scan` (`functools.lru_cache`); validation helpers live in `_validate.py`.
+[`config/`](config/) holds the `lookup('config', …)` path-validation suite. The four classifier tests share a single cached project scan exposed by `_scan.iter_matches` and `_scan.get_context`; validation helpers live in `_validate.py`.
 
 For framework, directory layout, and `make test-integration` usage see [integration.md](../../../docs/contributing/actions/testing/integration.md).

tests/integration/lookups/config/README.md

@@ -0,0 +1,9 @@
+# `lookup('config', …)` Path Validation 🔎
+
+Integration tests that statically validate every `lookup('config', application_id, …)` (and quoted-app variants) found in role YAML and Jinja templates against the role's effective configuration.
+
+Tests in this directory MUST only cover the four classifications of a single project-wide match stream: literal-app calls (`test_literal_paths.py`), variable-app calls with an existing literal path (`test_variable_paths.py`), `~`-concatenated path expressions (`test_wildcard_paths.py`), and the strict per-role pass for `lookup('config', application_id, …)` calls inside `roles/<role>/...` (`test_role_local_paths.py`). Pure unit coverage of the lookup plugin's return values MUST live under `tests/unit/plugins/lookup/`.
+
+The match stream is built once per process by `_scan.iter_matches` and shared across the four test classes; the repo-derived context (application defaults, user defaults, per-role schemas, `application_id`-declaring role set) comes from `_scan.get_context`. Validation helpers live in `_validate.py`. Suppression is wired through the project's `noqa` / `nocheck` grammar; the rule key is `lookup-config-path` (see [suppression.md](../../../../docs/contributing/actions/testing/suppression.md)).
+
+For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/README.md

@@ -2,6 +2,6 @@
 
 Integration tests that enforce invariants of each role's `meta/main.yml`: the file MUST exist for every role under `roles/`, and `galaxy_info` MUST carry the fields the Sphinx generator expects (non-empty `description`, no `None` values).
 
-Tests in this directory MUST only cover `meta/main.yml` presence and field shape. Tests about role dependencies declared in that file (`dependencies:`, `galaxy_info.run_after`) MUST live under [`../dependencies/`](../dependencies/).
+Tests in this directory MUST only cover `meta/main.yml` presence and field shape. Tests about role dependencies declared in that file (`dependencies:`) MUST live under [`dependencies/`](dependencies/); checks tied to `meta/services.yml` MUST live under [`services/`](services/) (with the ordering-only `galaxy_info.run_after` checks under [`services/run_after/`](services/run_after/)); checks against `meta/variants.yml` MUST live under [`variants/`](variants/).
 
 For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/dependencies/README.md

@@ -2,6 +2,6 @@
 
 Integration tests that enforce invariants of the role dependency graph expressed via `meta/main.yml`: declared `dependencies:` targets MUST exist, the `dependencies:` graph MUST be acyclic, and meta dependencies MUST NOT be declared when a guarded `include_role` / `import_role` would suffice.
 
-Tests in this directory MUST only cover `meta/main.yml`'s `dependencies:` key. Checks tied specifically to `galaxy_info.run_after` (self-references, reference existence, run-after cycles) MUST live under [`run_after/`](run_after/). Generic structural role checks (folder names, presence of `meta/main.yml`, include target existence) MUST live one level up under `tests/integration/roles/`.
+Tests in this directory MUST only cover `meta/main.yml`'s `dependencies:` key. Checks tied specifically to `galaxy_info.run_after` (self-references, reference existence, run-after cycles) MUST live under [`../services/run_after/`](../services/run_after/). Generic structural role checks (folder names, presence of `meta/main.yml`, include target existence) MUST live two levels up under `tests/integration/roles/`.
 
-For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../docs/contributing/actions/testing/integration.md).
+For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/dependencies/test_circular_meta.py

@@ -1,12 +1,12 @@
 import unittest
 import os
-import yaml
+
+from utils.cache.yaml import load_yaml_any
 
 
 def load_yaml_file(file_path):
-    """Load a YAML file and return its content."""
-    with open(file_path, "r") as file:
-        return yaml.safe_load(file) or {}
+    """Load a YAML file via the cached helper."""
+    return load_yaml_any(file_path, default_if_missing={}) or {}
 
 
 def get_meta_info(role_path):

tests/integration/roles/meta/dependencies/test_meta_resolve.py

@@ -1,7 +1,8 @@
 import unittest
 import os
 import glob
-import yaml
+
+from utils.cache.yaml import load_yaml_any
 
 
 class TestRoleDependencies(unittest.TestCase):
@@ -9,7 +10,7 @@ class TestRoleDependencies(unittest.TestCase):
         # Determine the path to the roles directory relative to this test file
         tests_dir = os.path.dirname(__file__)
         project_root = os.path.abspath(
-            os.path.join(tests_dir, os.pardir, os.pardir, os.pardir, os.pardir)
+            os.path.join(tests_dir, os.pardir, os.pardir, os.pardir, os.pardir, os.pardir)
         )
         roles_dir = os.path.join(project_root, "roles")
 
@@ -24,9 +25,8 @@ class TestRoleDependencies(unittest.TestCase):
             role_dir = os.path.dirname(os.path.dirname(meta_file))
             role_name = os.path.basename(role_dir)
             with self.subTest(role=role_name):
-                # Load the YAML metadata
-                with open(meta_file, "r") as f:
-                    meta = yaml.safe_load(f) or {}
+                # Load the YAML metadata via the cached helper.
+                meta = load_yaml_any(meta_file, default_if_missing={}) or {}
 
                 # Extract dependencies list
                 dependencies = meta.get("dependencies", [])

tests/integration/roles/meta/dependencies/test_unnecessary.py

@@ -2,18 +2,18 @@
 import os
 import re
 import glob
-import yaml
 import unittest
 from typing import Dict, Set, List, Optional
 
 from utils.cache.files import read_text as _read_text_cached
+from utils.cache.yaml import load_yaml_any
 
 # ---------------- Utilities ----------------
 
 
 def safe_load_yaml(path: str):
     try:
-        return yaml.safe_load(_read_text_cached(path)) or {}
+        return load_yaml_any(path, default_if_missing={}) or {}
     except Exception:
         return {}
 
@@ -208,7 +208,7 @@ class TestUnnecessaryRoleDependencies(unittest.TestCase):
     def setUp(self):
         # project root = two levels up from this test file
         self.project_root = os.path.abspath(
-            os.path.join(os.path.dirname(__file__), "..", "..", "..", "..")
+            os.path.join(os.path.dirname(__file__), "..", "..", "..", "..", "..")
         )
         self.roles = iter_role_dirs(self.project_root)
 

tests/integration/roles/meta/services/README.md

@@ -0,0 +1,7 @@
+# Role `meta/services.yml` Tests 🛎️
+
+Integration tests that enforce invariants of each role's `meta/services.yml`: the dynamic-flag form (`"{{ '<role>' in group_names }}"`) MUST be used wherever a service entry depends on a sibling role's deployment, and a role's `services.yml` MUST NOT reference itself as a sibling provider.
+
+Tests in this directory MUST only cover `meta/services.yml` shape and content. The ordering-only `galaxy_info.run_after` cluster lives in the [`run_after/`](run_after/) sub-cluster; coverage of `meta/variants.yml` lives one level up in [`../variants/`](../variants/); coverage of `meta/main.yml`'s `dependencies:` key lives in [`../dependencies/`](../dependencies/).
+
+For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/services/run_after/README.md

@@ -2,6 +2,6 @@
 
 Integration tests that enforce invariants of `galaxy_info.run_after` in each role's `meta/main.yml`: every referenced name MUST resolve to an existing role directory, no role MUST list itself in `run_after`, and the resulting ordering graph MUST be acyclic.
 
-Tests in this directory MUST only cover `run_after` semantics. Checks that target the `dependencies:` key MUST live in the parent [`dependencies/`](../) cluster.
+Tests in this directory MUST only cover `run_after` semantics. Checks that target the `dependencies:` key MUST live in the sibling [`../../dependencies/`](../../dependencies/) cluster; broader `meta/services.yml` shape checks MUST live one level up in [`../`](../).
 
-For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../../docs/contributing/actions/testing/integration.md).
+For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/services/run_after/test_circular.py

@@ -15,7 +15,7 @@ class TestCircularDependencies(unittest.TestCase):
     def setUpClass(cls):
         # Determine the path to the repo root and the roles directory
         here = os.path.abspath(os.path.dirname(__file__))
-        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", ".."))
+        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", "..", ".."))
         cls.roles_dir = os.path.join(repo_root, "roles")
 
     def test_no_circular_dependencies(self):

tests/integration/roles/meta/services/run_after/test_references.py

@@ -1,6 +1,7 @@
 import os
 import unittest
-import yaml
+
+from utils.cache.yaml import load_yaml_any
 
 
 class TestRunAfterReferences(unittest.TestCase):
@@ -13,7 +14,7 @@ class TestRunAfterReferences(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
         here = os.path.abspath(os.path.dirname(__file__))
-        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", ".."))
+        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", "..", ".."))
         cls.roles_dir = os.path.join(repo_root, "roles")
         # collect all role names (folder names) in roles/
         cls.existing_roles = {
@@ -30,8 +31,7 @@ class TestRunAfterReferences(unittest.TestCase):
                 # skip roles without a meta/main.yml
                 continue
 
-            with open(meta_path, "r") as f:
-                data = yaml.safe_load(f) or {}
+            data = load_yaml_any(meta_path, default_if_missing={}) or {}
 
             run_after = data.get("galaxy_info", {}).get("run_after", [])
             for dep in run_after:

tests/integration/roles/meta/services/run_after/test_self.py

@@ -1,6 +1,7 @@
 import os
 import unittest
-import yaml
+
+from utils.cache.yaml import load_yaml_any
 
 
 class TestSelfDependency(unittest.TestCase):
@@ -12,7 +13,7 @@ class TestSelfDependency(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
         here = os.path.abspath(os.path.dirname(__file__))
-        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", ".."))
+        repo_root = os.path.abspath(os.path.join(here, "..", "..", "..", "..", "..", ".."))
         cls.roles_dir = os.path.join(repo_root, "roles")
 
     def test_no_self_in_run_after(self):
@@ -22,8 +23,7 @@ class TestSelfDependency(unittest.TestCase):
             if not os.path.isdir(role_path) or not os.path.isfile(meta_file):
                 continue
 
-            with open(meta_file, "r") as f:
-                data = yaml.safe_load(f) or {}
+            data = load_yaml_any(meta_file, default_if_missing={}) or {}
 
             run_after = data.get("galaxy_info", {}).get("run_after", [])
             if entry in run_after:

tests/integration/roles/meta/services/run_after/test_services_explicit.py

@@ -30,8 +30,7 @@ from __future__ import annotations
 import unittest
 from pathlib import Path
 
-import yaml
-
+from utils.cache.yaml import load_yaml_any
 from utils.roles.meta_lookup import get_role_run_after
 from utils.service_registry import (
     build_role_to_covered_keys,
@@ -40,7 +39,7 @@ from utils.service_registry import (
 )
 
 
-PROJECT_ROOT = Path(__file__).resolve().parents[5]
+PROJECT_ROOT = Path(__file__).resolve().parents[6]
 ROLES_DIR = PROJECT_ROOT / "roles"
 
 
@@ -48,10 +47,7 @@ def _load_services(role_dir: Path) -> dict:
     services_file = role_dir / "meta" / "services.yml"
     if not services_file.is_file():
         return {}
-    text = services_file.read_text(encoding="utf-8")
-    if not text.strip():
-        return {}
-    data = yaml.safe_load(text)
+    data = load_yaml_any(services_file, default_if_missing={})
     return data if isinstance(data, dict) else {}
 
 

tests/integration/roles/meta/services/test_dynamic_flags.py

@@ -66,10 +66,9 @@ from __future__ import annotations
 import unittest
 from pathlib import Path
 
-import yaml
-
 from utils.annotations.suppress import line_has_rule
 from utils.cache.files import read_text
+from utils.cache.yaml import load_yaml_any
 from utils.entity_name_utils import get_entity_name
 from utils.service_registry import (
     build_covered_key_to_role,
@@ -78,7 +77,7 @@ from utils.service_registry import (
 )
 
 
-PROJECT_ROOT = Path(__file__).resolve().parents[4]
+PROJECT_ROOT = Path(__file__).resolve().parents[5]
 ROLES_DIR = PROJECT_ROOT / "roles"
 
 _RULE = "dynamic-flag"
@@ -165,8 +164,8 @@ class TestServicesDynamicFlags(unittest.TestCase):
             if not services_file.is_file():
                 continue
             try:
-                data = yaml.safe_load(read_text(str(services_file))) or {}
-            except yaml.YAMLError as exc:
+                data = load_yaml_any(services_file, default_if_missing={}) or {}
+            except Exception as exc:
                 offenders.append(f"{role_name}: YAML parse error: {exc}")
                 continue
             if not isinstance(data, dict):

tests/integration/roles/meta/services/test_no_self_reference.py

@@ -30,7 +30,7 @@ from utils.cache.yaml import load_yaml_any
 from utils.entity_name_utils import get_entity_name
 
 
-PROJECT_ROOT = Path(__file__).resolve().parents[4]
+PROJECT_ROOT = Path(__file__).resolve().parents[5]
 ROLES_DIR = PROJECT_ROOT / "roles"
 
 

tests/integration/roles/meta/variants/README.md

@@ -0,0 +1,7 @@
+# Role `meta/variants.yml` Tests 🔀
+
+Integration tests that enforce invariants of each role's `meta/variants.yml`: the matrix-deploy variant list MUST exercise every dynamic flag declared in the role's `meta/services.yml` on both polarities, every service key referenced under `services:` in a variant MUST exist in `services.yml`, and the auth matrix (`oidc` / `oauth2` / `ldap`) MUST cover the LDAP-only branch tracked by [018-playwright-ldap-coverage.md](../../../../../docs/requirements/018-playwright-ldap-coverage.md).
+
+Tests in this directory MUST only cover `meta/variants.yml` shape and coverage. Coverage of `meta/services.yml` itself lives in [`../services/`](../services/); coverage of `meta/main.yml`'s `dependencies:` key lives in [`../dependencies/`](../dependencies/).
+
+For framework, directory layout, and `make test-integration` usage see [integration.md](../../../../../docs/contributing/actions/testing/integration.md).

tests/integration/roles/meta/variants/test_auth_coverage.py

@@ -0,0 +1,188 @@
+"""Integration guard: every role whose `meta/services.yml` declares
+`oidc` / `oauth2` OR `ldap` via the dynamic
+``"{{ '<role>' in group_names }}"`` form MUST exercise the auth
+matrix in `meta/variants.yml`.
+
+Three coverage rules apply per role:
+
+1. If `oidc` or `oauth2` is dynamic, `variants.yml` MUST contain at
+   least one variant pinning that service to ``enabled: true`` AND
+   ``shared: true`` together, and at least one pinning it to
+   ``enabled: false`` AND ``shared: false`` together. Mixed-polarity
+   pinning (e.g. ``enabled: true`` with ``shared: false``) does not
+   count toward either side — the deploy modes the variants reflect
+   are "auth fully present" vs. "auth fully absent".
+2. The same rule applies to `ldap` whenever it is dynamic.
+3. When both an auth service (`oidc` or `oauth2`) AND `ldap` are
+   dynamic in the same role, at least one variant MUST pin the auth
+   service to ``enabled: false`` WHILE pinning ``ldap.enabled: true``.
+   That is the LDAP-only branch tracked by
+   [docs/requirements/018-playwright-ldap-coverage.md](../../../../docs/requirements/018-playwright-ldap-coverage.md);
+   without it, the matrix never exercises the LDAP authentication
+   path in isolation from OIDC and the spec defaults to OIDC-only
+   coverage by accident.
+
+The check sits next to
+[test_variants_coverage.py](./test_variants_coverage.py), which
+enforces the broader "every dynamic flag needs both polarities"
+contract per individual flag. This test layers the auth-specific
+combinatorial requirement on top.
+"""
+
+from __future__ import annotations
+
+import unittest
+from pathlib import Path
+from typing import List, Tuple
+
+from utils.cache.yaml import load_yaml_any
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[5]
+ROLES_DIR = PROJECT_ROOT / "roles"
+
+_AUTH_KEYS: Tuple[str, ...] = ("oidc", "oauth2")
+_LDAP_KEY: str = "ldap"
+
+_MISSING = object()
+
+
+def _is_dynamic_flag(value) -> bool:
+    return isinstance(value, str) and "in group_names" in value
+
+
+def _service_is_dynamic(services: dict, key: str) -> bool:
+    entry = services.get(key)
+    if not isinstance(entry, dict):
+        return False
+    return any(_is_dynamic_flag(entry.get(f)) for f in ("enabled", "shared"))
+
+
+def _load_yaml(path: Path) -> object:
+    if not path.is_file():
+        return None
+    try:
+        return load_yaml_any(str(path), default_if_missing=None)
+    except Exception:
+        return None
+
+
+def _variant_overrides_for(variant: dict, service_key: str, flag: str) -> object:
+    services = variant.get("services") if isinstance(variant, dict) else None
+    if not isinstance(services, dict):
+        return _MISSING
+    entry = services.get(service_key)
+    if not isinstance(entry, dict):
+        return _MISSING
+    if flag not in entry:
+        return _MISSING
+    return entry[flag]
+
+
+def _has_pair_polarity(
+    variants: List[dict], service_key: str, polarity: bool
+) -> bool:
+    """True iff some variant pins ``services.<key>.enabled`` AND
+    ``services.<key>.shared`` to literal *polarity* in the same entry."""
+    for variant in variants:
+        e = _variant_overrides_for(variant, service_key, "enabled")
+        s = _variant_overrides_for(variant, service_key, "shared")
+        if e is polarity and s is polarity:
+            return True
+    return False
+
+
+def _has_ldap_only_variant(variants: List[dict], auth_keys: List[str]) -> bool:
+    """True iff some variant pins ``ldap.enabled: true`` while pinning
+    every dynamic auth service (`oidc` / `oauth2`) to ``enabled:
+    false`` in the SAME entry."""
+    for variant in variants:
+        if _variant_overrides_for(variant, _LDAP_KEY, "enabled") is not True:
+            continue
+        if all(
+            _variant_overrides_for(variant, ak, "enabled") is False
+            for ak in auth_keys
+        ):
+            return True
+    return False
+
+
+class TestAuthVariantsCoverage(unittest.TestCase):
+    def test_oidc_oauth2_ldap_variants_cover_auth_matrix(self):
+        offenders: List[str] = []
+
+        for role_dir in sorted(p for p in ROLES_DIR.iterdir() if p.is_dir()):
+            role_name = role_dir.name
+            services = _load_yaml(role_dir / "meta" / "services.yml")
+            if not isinstance(services, dict):
+                continue
+
+            dynamic_auth = [k for k in _AUTH_KEYS if _service_is_dynamic(services, k)]
+            ldap_dynamic = _service_is_dynamic(services, _LDAP_KEY)
+
+            if not dynamic_auth and not ldap_dynamic:
+                continue
+
+            variants_path = role_dir / "meta" / "variants.yml"
+            variants_raw = _load_yaml(variants_path)
+            if not isinstance(variants_raw, list):
+                offenders.append(
+                    f"{role_name}: dynamic auth/ldap services in services.yml "
+                    f"but {variants_path.relative_to(PROJECT_ROOT)} is missing "
+                    f"or not a YAML list."
+                )
+                continue
+            variants = [v if isinstance(v, dict) else {} for v in variants_raw]
+
+            for ak in dynamic_auth:
+                if not _has_pair_polarity(variants, ak, True):
+                    offenders.append(
+                        f"{role_name}: services.{ak} is dynamic but no variant "
+                        f"pins ``services.{ak}.enabled: true`` AND "
+                        f"``services.{ak}.shared: true`` together. Add a "
+                        f"variant entry with both flags true."
+                    )
+                if not _has_pair_polarity(variants, ak, False):
+                    offenders.append(
+                        f"{role_name}: services.{ak} is dynamic but no variant "
+                        f"pins ``services.{ak}.enabled: false`` AND "
+                        f"``services.{ak}.shared: false`` together. Add a "
+                        f"variant entry with both flags false."
+                    )
+
+            if ldap_dynamic:
+                if not _has_pair_polarity(variants, _LDAP_KEY, True):
+                    offenders.append(
+                        f"{role_name}: services.ldap is dynamic but no variant "
+                        f"pins ``services.ldap.enabled: true`` AND "
+                        f"``services.ldap.shared: true`` together. Add a "
+                        f"variant entry with both flags true."
+                    )
+                if not _has_pair_polarity(variants, _LDAP_KEY, False):
+                    offenders.append(
+                        f"{role_name}: services.ldap is dynamic but no variant "
+                        f"pins ``services.ldap.enabled: false`` AND "
+                        f"``services.ldap.shared: false`` together. Add a "
+                        f"variant entry with both flags false."
+                    )
+
+            if dynamic_auth and ldap_dynamic:
+                if not _has_ldap_only_variant(variants, dynamic_auth):
+                    auth_list = " / ".join(f"services.{k}" for k in dynamic_auth)
+                    offenders.append(
+                        f"{role_name}: both ldap and {auth_list} are dynamic, "
+                        f"but no variant pins ``ldap.enabled: true`` together "
+                        f"with ``enabled: false`` on the auth service(s) — the "
+                        f"LDAP-only branch is never exercised. See "
+                        f"docs/requirements/018-playwright-ldap-coverage.md."
+                    )
+
+        if offenders:
+            self.fail(
+                "Auth variant coverage is incomplete:\n"
+                + "\n".join(f"  - {o}" for o in offenders)
+            )
+
+
+if __name__ == "__main__":  # pragma: no cover
+    unittest.main()

tests/integration/roles/meta/variants/test_coverage.py

@@ -38,7 +38,7 @@ from pathlib import Path
 from utils.cache.yaml import load_yaml_any
 
 
-PROJECT_ROOT = Path(__file__).resolve().parents[4]
+PROJECT_ROOT = Path(__file__).resolve().parents[5]
 ROLES_DIR = PROJECT_ROOT / "roles"
 
 

tests/integration/roles/meta/variants/test_services_match.py

@@ -0,0 +1,90 @@
+"""Integration guard: every service key referenced under ``services:``
+in a role's `meta/variants.yml` MUST exist as a top-level key in the
+same role's `meta/services.yml`.
+
+Why
+---
+
+The matrix-deploy CLI deep-merges each variant onto the role's
+`meta/services.yml` to produce one effective `applications.<role>`
+config per variant entry. Keys that exist in `variants.yml` but NOT
+in `services.yml` survive the merge as dead config — at best a typo
+that silently does nothing, at worst a stale reference to a service
+that was renamed or removed (and now silently fails to flip its
+flags). Either way the variant promises coverage for a service the
+role does not actually declare.
+
+This test catches that drift early. The check is intentionally
+asymmetric: extra keys in `services.yml` (services declared but not
+overridden by any variant) are fine and tracked by
+[test_variants_coverage.py](./test_variants_coverage.py); only the
+opposite direction (variant overrides → services declared) is what
+this guard enforces.
+"""
+
+from __future__ import annotations
+
+import unittest
+from pathlib import Path
+from typing import List
+
+from utils.cache.yaml import load_yaml_any
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[5]
+ROLES_DIR = PROJECT_ROOT / "roles"
+
+
+def _load_yaml(path: Path) -> object:
+    if not path.is_file():
+        return None
+    try:
+        return load_yaml_any(str(path), default_if_missing=None)
+    except Exception:
+        return None
+
+
+class TestVariantsServicesMatch(unittest.TestCase):
+    def test_variants_only_reference_services_declared_in_services_yml(self):
+        offenders: List[str] = []
+
+        for role_dir in sorted(p for p in ROLES_DIR.iterdir() if p.is_dir()):
+            role_name = role_dir.name
+            services = _load_yaml(role_dir / "meta" / "services.yml")
+            if not isinstance(services, dict):
+                continue
+            declared_keys = {k for k in services.keys() if isinstance(k, str)}
+
+            variants_raw = _load_yaml(role_dir / "meta" / "variants.yml")
+            if not isinstance(variants_raw, list):
+                continue
+
+            for index, variant in enumerate(variants_raw):
+                if not isinstance(variant, dict):
+                    continue
+                variant_services = variant.get("services")
+                if not isinstance(variant_services, dict):
+                    continue
+                for key in variant_services.keys():
+                    if not isinstance(key, str):
+                        continue
+                    if key in declared_keys:
+                        continue
+                    offenders.append(
+                        f"{role_name}: variants.yml[{index}].services.{key} "
+                        f"is not declared as a top-level key in "
+                        f"meta/services.yml. Either add ``{key}:`` to "
+                        f"services.yml or drop the override from this "
+                        f"variant entry."
+                    )
+
+        if offenders:
+            self.fail(
+                "variants.yml references services not declared in "
+                "services.yml:\n"
+                + "\n".join(f"  - {o}" for o in offenders)
+            )
+
+
+if __name__ == "__main__":  # pragma: no cover
+    unittest.main()

tests/lint/ansible/variables/README.md

@@ -0,0 +1,12 @@
+# Ansible Variable Lints 🪪
+
+Lint tests that enforce variable-lifecycle invariants across the Ansible tree: every name a role declares MUST be consumed somewhere, and every name a role passes MUST be referenced by the play / template stack that receives it.
+
+Tests in this directory MUST only cover variable definition / consumption checks. Two orthogonal scopes live here:
+
+- `test_role_and_group_vars_used.py` scans top-level keys of `roles/<role>/{vars,defaults}/main.yml` and `group_vars/**/*.yml` against every project `.yml` / `.yaml` / `.j2` file. The suppression rule key is `unused-var` (see [suppression.md](../../../../docs/contributing/actions/testing/suppression.md)).
+- `test_vars_usage_in_yaml.py` scans task-level `vars:` blocks against the same consumer set. The two scanners never overlap because their declaration shapes differ.
+
+Both tests route every read through the project caches (`utils.cache.files`, `utils.cache.yaml`); contributors adding a new variable scanner MUST do the same instead of calling `yaml.safe_load*` or `open()` directly.
+
+For framework, directory layout, and `make test-lint` usage see [lint.md](../../../../docs/contributing/actions/testing/lint.md).

tests/lint/ansible/variables/test_vars_usage_in_yaml.py

@@ -2,9 +2,9 @@ import unittest
 from pathlib import Path
 import re
 from typing import Any, Iterable, Set, List, Dict, Tuple
-import yaml
 
 from utils.cache.files import iter_project_files, read_text
+from utils.cache.yaml import load_yaml_all
 
 
 def repo_root() -> Path:
@@ -55,7 +55,7 @@ class TestVarsPassedAreUsed(unittest.TestCase):
 
     def _load_yaml_documents(self, path: Path) -> List[Any]:
         try:
-            return list(yaml.safe_load_all(read_text(str(path)))) or []
+            return list(load_yaml_all(path, default_if_missing=()))
         except Exception:
             # File may contain heavy templating or anchors; skip structural parse
             return []

utils/cache/yaml.py

@@ -125,6 +125,46 @@ def load_yaml_any(path, *, default_if_missing: Any = _MISSING) -> Any:
     return _load_raw(path, default_if_missing=default_if_missing)
 
 
+_CACHE_ALL: Dict[Tuple[str, int, int], Tuple[Any, ...]] = {}
+
+
+def load_yaml_all(path, *, default_if_missing: Any = _MISSING) -> Tuple[Any, ...]:
+    """Memoised multi-document YAML load.
+
+    Returns a tuple of every document parsed by ``yaml.safe_load_all``
+    in declaration order. Use this for files that legitimately carry
+    several `---`-separated YAML documents.
+
+    Cache semantics mirror :func:`load_yaml_any`:
+    cache key is ``(resolved-path, mtime_ns, size)``; cached value is
+    the same tuple instance for every caller, so callers MUST treat
+    the returned value as read-only.
+
+    Empty / missing files behave like ``load_yaml_any`` — when
+    ``default_if_missing`` is provided it is returned wrapped in a
+    one-tuple iff truthy, otherwise an empty tuple.
+    """
+    p = Path(path)
+    if not p.exists():
+        if default_if_missing is _MISSING:
+            raise FileNotFoundError(p)
+        return (default_if_missing,) if default_if_missing else ()
+
+    sig = _signature(p)
+    if sig in _CACHE_ALL:
+        return _CACHE_ALL[sig]
+
+    path_str = sig[0]
+    for stale_key in [k for k in _CACHE_ALL if k[0] == path_str and k != sig]:
+        _CACHE_ALL.pop(stale_key, None)
+
+    with p.open("r", encoding="utf-8") as f:
+        # This module IS the cache; calling itself would recurse.
+        docs = tuple(yaml.safe_load_all(f))  # noqa: direct-yaml
+    _CACHE_ALL[sig] = docs
+    return docs
+
+
 def dump_yaml(path, data: Mapping[str, Any]) -> None:
     """Write `data` to `path` as YAML and evict the cached entry.
 
@@ -174,10 +214,12 @@ def load_yaml_str(text: str) -> Any:
 
 
 def _drop_path(path) -> None:
-    """Drop every cache entry that matches `path` (any mtime/size)."""
+    """Drop every cache entry (single- and multi-doc) for `path`."""
     path_str = _path_key(path)
     for stale_key in [k for k in _CACHE if k[0] == path_str]:
         _CACHE.pop(stale_key, None)
+    for stale_key in [k for k in _CACHE_ALL if k[0] == path_str]:
+        _CACHE_ALL.pop(stale_key, None)
 
 
 def invalidate(path) -> None:
@@ -201,6 +243,7 @@ def _reset() -> None:
     orchestrates calls to all of them.
     """
     _CACHE.clear()
+    _CACHE_ALL.clear()
 
 
 # Backwards-compatible alias for in-tree callers that may still reference