# Cache Helpers 💾

This package contains the in-process cache layers Infinito.Nexus uses to keep CLI tools, lookup plugins, and Ansible filters fast without re-reading or re-merging the same data multiple times per Python process.

## Modules 🧭

| Module | Caches |
|---|---|
| [yaml.py](yaml.py) | A single YAML file's parsed root, keyed by absolute path. Every other module routes its YAML reads through here so each file is parsed at most once per process. |
| [files.py](files.py) | The project tree's full path list (one walk per process) and per-file UTF-8 contents (`read_text`). Used by lint/integration tests and CLI tools that scan the repo. |
| [base.py](base.py) | Cross-cutting helpers: filesystem constants (`PROJECT_ROOT`, `ROLES_DIR`, `DEFAULT_TOKENS_FILE`), `_deep_merge`, cache-key + content-fingerprint signatures, the cross-domain re-entry guard, and the templar-render machinery. The only ansible-coupled symbol is `_render_with_templar`, which lazy-imports its dependency. |
| [applications.py](applications.py) | Per-app variants + defaults + `get_merged_applications`. **Strictly ansible-free at import time** so the GitHub Actions runner-host CLI path (`cli.administration.deploy.development.init` → `plan_dev_inventory_matrix` → `get_variants`) keeps working without ansible installed. |
| [carrier.py](carrier.py) | The play-scoped carrier of the rendered applications payload: the fact name `_INFINITO_APPLICATIONS_RENDERED`, `merged_applications_cache_key`, and the reader `get_merged_applications` consults before it renders. |
| [users/](users/) | User definitions, token store hydration, alias materialization, `get_user_defaults`, `get_merged_users` (package; `users/placeholders.py` substitutes `DOMAIN_PRIMARY` / `ORGANIZATION` / `SOFTWARE_NAME` ahead of the templar render pass). |
| [domains.py](domains.py) | Canonical-domains map derived from the merged applications view: `get_merged_domains`. A read issued while the applications render is in progress (the re-entry guard is set) sees the unrendered view, so it is cached under its own key and never answers a later read. |
| [`__init__.py`](__init__.py) | Owns the package-level `_reset_cache_for_tests()` orchestrator that clears every cache plus the shared fingerprint memo in one call. |

## When To Use Which 🎯

- Use `utils.cache.yaml.load_yaml(path)` (or `load_yaml_any(path)` when the YAML root is a list or scalar) whenever a CLI tool, filter, or library reads individual YAML files where the same path MAY be touched repeatedly within a single invocation.
- Import the cached getters directly from the domain module that owns them: `utils.cache.applications` for `get_application_defaults` / `get_variants` / `get_merged_applications`, `utils.cache.users` for `get_user_defaults` / `get_merged_users`, and `utils.cache.domains` for `get_merged_domains`. The package's `_reset_cache_for_tests` is exposed at `from utils.cache import _reset_cache_for_tests`.

The layers are orthogonal: `yaml.py` caches "this file's parsed root", the `applications`/`users`/`domains` modules cache "this aggregated, rendered payload". They do not invalidate each other and MAY both be active at the same time.

## Lifetime 🕒

Both layers are process-wide in-process caches. They intentionally do NOT track on-disk file mtimes; CLI tools are short-lived and the assumption "the on-disk file does not change while my process runs" holds. The single exception is `utils.cache.yaml.dump_yaml`, which writes through and evicts the cached entry for the path it just wrote so a tool that edits a file mid-run sees the new content on the next read.

Inside a play the process boundary is the task: Ansible forks a fresh worker per task, so `_MERGED_APPLICATIONS_CACHE` starts empty in every worker. `get_merged_applications` therefore also accepts the play-scoped carrier fact `_INFINITO_APPLICATIONS_RENDERED` (payload plus cache key, parked once by the constructor stage) and serves it when the key matches. See [applications.md](../../docs/contributing/artefact/files/plugins/lookup/applications.md) for the contract.

## Test-Only Helpers 🧪

Each module owns a private `_reset()` that clears its own cache dicts. The package-level `utils.cache._reset_cache_for_tests()` orchestrates all six (`base`, `applications`, `users`, `domains`, `yaml`, `files`) plus the shared fingerprint memo. Unit tests that exercise the cached paths MUST call it in `setUp` to guarantee clean state across test cases.

## Design Guidelines 📐

- You MUST NOT introduce a third parallel cache for the same concern. If you need to cache a derived shape, add a getter to the appropriate domain module (`applications`, `users`, or `domains`) that builds on the existing primitives.
- You SHOULD prefer the explicit cached getter over re-implementing the same memoisation in a calling module.
- You MUST NOT mutate values returned by `utils.cache.yaml.load_yaml`. The same dict instance is returned to every caller; callers that need to mutate MUST `copy.deepcopy()` the result first. Each domain getter documents whether it returns a fresh deep copy or the cached instance; respect that contract.
- You MUST keep dependencies inside this package minimal. Cache primitives MUST remain importable from filter plugins, lookup plugins, and CLI tools without dragging in heavy dependencies. **Specifically: `applications.py` MUST stay ansible-free at import time.** See CI runs 24934007615 / 24935979190 for the regression that motivated the split. Tests in `tests/unit/python/utils/cache/test_data.py::TestImportableWithoutAnsible` pin both the import-time and call-time invariants.

## Example Imports ✍️

```python
from utils.cache.yaml import load_yaml, load_yaml_any

from utils.cache.applications import (
    get_application_defaults,
    get_merged_applications,
    get_variants,
)
from utils.cache.users import get_merged_users, get_user_defaults
from utils.cache.domains import get_merged_domains

# Tests
from utils.cache import _reset_cache_for_tests
```

## Schema 🗺️

```mermaid
flowchart TD
    decl["meta/services.yml<br/>a flag reading group_names<br/>114 role files"]
    ctor["01_constructor.yml:191<br/>applications_carrier"]
    fact["_INFINITO_APPLICATIONS_RENDERED<br/>per-host fact"]
    lookup["lookup('database', ...)<br/>and every applications consumer"]
    key["base.py:103<br/>_stable_variables_signature"]
    proc["_MERGED_APPLICATIONS_CACHE<br/>process-global dict"]
    render["base.py:198<br/>_render_with_templar"]
    consumer["database.py:121<br/>central_enabled = shared"]
    name["postgres_postgres<br/>vs pretix_database"]

    decl -->|"read by"| render
    ctor -->|"parks"| fact
    lookup -->|"computes"| key
    key -->|"1. looked up in"| proc
    proc -->|"2. miss: falls back to"| fact
    fact -->|"3. miss: renders"| render
    render -->|"substitutes group_names,<br/>then stores under key"| proc
    render --> consumer
    consumer -->|"selects"| name

    proc -.->|"TRIP-WIRE: read BEFORE the<br/>host's own fact, so an inherited<br/>entry outranks it"| fact
    render -.->|"TRIP-WIRE: main process renders<br/>task names for one host; forked<br/>workers inherit that entry"| proc
    key -.->|"TRIP-WIRE: carrier.py:68 stringifies<br/>only one side, so a non-scalar key<br/>part silently never matches again"| fact
```

The key covers `group_names`. `domains.py` derives from the applications view and shares that dependency; `users` reads `group_names` nowhere.