# Cache Helpers 💾

This package contains the in-process cache layers Infinito.Nexus uses to keep CLI tools, lookup plugins, and Ansible filters fast without re-reading or re-merging the same data multiple times per Python process.

## Modules 🧭

| Module | Caches |
|---|---|
| [yaml.py](yaml.py) | A single YAML file's parsed root, keyed by absolute path. Every other module routes its YAML reads through here so each file is parsed at most once per process. |
| [files.py](files.py) | The project tree's full path list (one walk per process) and per-file UTF-8 contents (`read_text`). Used by lint/integration tests and CLI tools that scan the repo. |
| [base.py](base.py) | Cross-cutting helpers: filesystem constants (`PROJECT_ROOT`, `ROLES_DIR`, `DEFAULT_TOKENS_FILE`), `_deep_merge`, cache-key + content-fingerprint signatures, the cross-domain re-entry guard, and the templar-render machinery. The only ansible-coupled symbol is `_render_with_templar`, which lazy-imports its dependency. |
| [applications.py](applications.py) | Per-app variants + defaults + `get_merged_applications`. **Strictly ansible-free at import time** so the GitHub Actions runner-host CLI path (`cli.administration.deploy.development.init` → `plan_dev_inventory_matrix` → `get_variants`) keeps working without ansible installed. |
| [carrier.py](carrier.py) | The play-scoped carrier of the rendered applications payload: the fact name `_INFINITO_APPLICATIONS_RENDERED`, `merged_applications_cache_key`, and the reader `get_merged_applications` consults before it renders. |
| [users/](users/) | User definitions, token store hydration, alias materialization, `get_user_defaults`, `get_merged_users` (package; `users/placeholders.py` substitutes `DOMAIN_PRIMARY` / `ORGANIZATION` / `SOFTWARE_NAME` ahead of the templar render pass). |
| [domains.py](domains.py) | Canonical-domains map derived from the merged applications view: `get_merged_domains`. A read issued while the applications render is in progress (the re-entry guard is set) sees the unrendered view, so it is cached under its own key and never answers a later read. |
| [`__init__.py`](__init__.py) | Owns the package-level `_reset_cache_for_tests()` orchestrator that clears every cache plus the shared fingerprint memo in one call. |

## When To Use Which 🎯

- Use `utils.cache.yaml.load_yaml(path)` (or `load_yaml_any(path)` when the YAML root is a list or scalar) whenever a CLI tool, filter, or library reads individual YAML files where the same path MAY be touched repeatedly within a single invocation.
- Import the cached getters directly from the domain module that owns them: `utils.cache.applications` for `get_application_defaults` / `get_variants` / `get_merged_applications`, `utils.cache.users` for `get_user_defaults` / `get_merged_users`, and `utils.cache.domains` for `get_merged_domains`. The package's `_reset_cache_for_tests` is exposed at `from utils.cache import _reset_cache_for_tests`.

The layers are orthogonal: `yaml.py` caches "this file's parsed root", the `applications`/`users`/`domains` modules cache "this aggregated, rendered payload". They do not invalidate each other and MAY both be active at the same time.

## Lifetime 🕒

Both layers are process-wide in-process caches. They intentionally do NOT track on-disk file mtimes; CLI tools are short-lived and the assumption "the on-disk file does not change while my process runs" holds. The single exception is `utils.cache.yaml.dump_yaml`, which writes through and evicts the cached entry for the path it just wrote so a tool that edits a file mid-run sees the new content on the next read.

Inside a play the process boundary is the task: Ansible forks a fresh worker per task, so `_MERGED_APPLICATIONS_CACHE` starts empty in every worker. `get_merged_applications` therefore also accepts the play-scoped carrier fact `_INFINITO_APPLICATIONS_RENDERED` (payload plus cache key, parked by the constructor stage and again after every role, every nested application load and every persisted token) and serves it when the key matches. See [applications.md](../../docs/contributing/artefact/files/plugins/lookup/applications.md) for the contract.

While one render runs, the re-entry guard also holds that render's unrendered view and its cache key. A nested `get_merged_applications` call from a lookup inside a role template (`config`, `tls`, `domain`, ...) with the same key returns that view instead of copying and merging the role defaults again. The view is shared by every nested call of the render, so callers MUST NOT mutate it. The guard drops it when the render ends.

## Test-Only Helpers 🧪

Each module owns a private `_reset()` that clears its own cache dicts. The package-level `utils.cache._reset_cache_for_tests()` orchestrates all six (`base`, `applications`, `users`, `domains`, `yaml`, `files`) plus the shared fingerprint memo. Unit tests that exercise the cached paths MUST call it in `setUp` to guarantee clean state across test cases.

## Design Guidelines 📐

- You MUST NOT introduce a third parallel cache for the same concern. If you need to cache a derived shape, add a getter to the appropriate domain module (`applications`, `users`, or `domains`) that builds on the existing primitives.
- You SHOULD prefer the explicit cached getter over re-implementing the same memoisation in a calling module.
- You MUST NOT mutate values returned by `utils.cache.yaml.load_yaml`. The same dict instance is returned to every caller; callers that need to mutate MUST `copy.deepcopy()` the result first. Each domain getter documents whether it returns a fresh deep copy or the cached instance; respect that contract.
- You MUST keep dependencies inside this package minimal. Cache primitives MUST remain importable from filter plugins, lookup plugins, and CLI tools without dragging in heavy dependencies. **Specifically: `applications.py` MUST stay ansible-free at import time.** See CI runs 24934007615 / 24935979190 for the regression that motivated the split. Tests in `tests/unit/python/utils/cache/test_data.py::TestImportableWithoutAnsible` pin both the import-time and call-time invariants.

## Example Imports ✍️

```python
from utils.cache.yaml import load_yaml, load_yaml_any

from utils.cache.applications import (
    get_application_defaults,
    get_merged_applications,
    get_variants,
)
from utils.cache.users import get_merged_users, get_user_defaults
from utils.cache.domains import get_merged_domains

# Tests
from utils.cache import _reset_cache_for_tests
```

## Schema 🗺️

```mermaid
flowchart TD
    decl["meta/services.yml<br/>a flag reading group_names<br/>114 role files"]
    ctor["01_constructor.yml:191<br/>applications_carrier"]
    fact["_INFINITO_APPLICATIONS_RENDERED<br/>per-host fact"]
    lookup["lookup('database', ...)<br/>and every applications consumer"]
    key["base.py:103<br/>_stable_variables_signature"]
    proc["_MERGED_APPLICATIONS_CACHE<br/>process-global dict"]
    render["base.py:198<br/>_render_with_templar"]
    consumer["database.py:121<br/>central_enabled = shared"]
    name["postgres_postgres<br/>vs pretix_database"]

    decl -->|"read by"| render
    ctor -->|"parks"| fact
    lookup -->|"computes"| key
    key -->|"1. looked up in"| proc
    proc -->|"2. miss: falls back to"| fact
    fact -->|"3. miss: renders"| render
    render -->|"substitutes group_names,<br/>then stores under key"| proc
    render --> consumer
    consumer -->|"selects"| name

    proc -.->|"TRIP-WIRE: read BEFORE the<br/>host's own fact, so an inherited<br/>entry outranks it"| fact
    render -.->|"TRIP-WIRE: main process renders<br/>task names for one host; forked<br/>workers inherit that entry"| proc
    key -.->|"TRIP-WIRE: carrier.py:68 stringifies<br/>only one side, so a non-scalar key<br/>part silently never matches again"| fact
```

The key covers `group_names`. `domains.py` derives from the applications view and shares that dependency; `users` reads `group_names` nowhere.