mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
perf(applications): render the applications SPOT once per play instead of once per task
Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.
The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.
The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.
Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
03e42bae6d
commit
113113e553
11 files changed
+350
-10
No files matched your search
@@ -18,6 +18,7 @@ The `applications` lookup is the ONLY supported runtime entry point for merged a
|
||||
| `lookup('applications')` | Full merged mapping keyed by `application_id`. |
|
||||
| `lookup('applications', '<application_id>')` | Merged entry for that application. Raises when missing. |
|
||||
| `lookup('applications', '<application_id>', default_value)` | Merged entry, or `default_value` when the entry is missing. |
|
||||
| `lookup('applications', carrier=True)` | `{"key": [...], "applications": {...}}`: the full mapping plus the cache key it is valid under. Reserved for the constructor's carrier task (see below). |
|
||||
|
||||
## Data Sources 📚
|
||||
|
||||
@@ -26,7 +27,19 @@ The lookup merges exactly two sources:
|
||||
1. **Defaults** discovered from each role's per-topic meta files: `roles/*/meta/server.yml`, `roles/*/meta/rbac.yml`, `roles/*/meta/services.yml`, `roles/*/meta/volumes.yml`, plus `roles/*/meta/secrets.yml` (post-`apply_schema()`). Variants from `roles/*/meta/variants.yml` deep-merge over the assembled per-role payload. See [layout.md](../../../../design/role/services/layout.md).
|
||||
2. **Overrides** supplied through the normal Ansible variable `applications` in inventory, group vars, host vars, or role vars.
|
||||
|
||||
No intermediate merged `applications` fact exists. No other source is consulted.
|
||||
No other source is consulted.
|
||||
|
||||
## Play-Scoped Carrier 🧊
|
||||
|
||||
Ansible forks a fresh worker process for every task. The in-process cache inside `get_merged_applications` therefore never survives from one task to the next, and every task that touches the lookup would render the full tree again (about 16 to 23 seconds per task on a CI runner, measured on the workspace jobs).
|
||||
|
||||
The constructor stage renders once through the [applications_carrier](../../../../../../plugins/action/applications_carrier.py) action and parks the result together with its cache key in the host fact `_INFINITO_APPLICATIONS_RENDERED` (the action calls `lookup('applications', carrier=True)`). Every later worker inherits that fact at fork time; `get_merged_applications` serves it whenever the key still matches and renders only when it does not (a changed `applications`, `users`, `DOMAIN_PRIMARY` or `SYSTEM_EMAIL_DOMAIN`).
|
||||
|
||||
- The task sits in `tasks/stages/01_constructor.yml` after the `networks`, `design` and `service_provider` merges, after the `add_host` placements and after the token store, so the payload renders with the same `group_names`, `groups` and user records a role would see. Moving it earlier bakes stale values into the play.
|
||||
- The action masks its own result: the fact is the whole rendered tree with every application's credentials, so it never reaches a log regardless of `MASK_CREDENTIALS_IN_LOGS`.
|
||||
- User records inside the payload are the token store's state at render time. Readers of tokens captured later in the play MUST go through `lookup('users', ...)` or `lookup('config', <app>, 'users.<key>...')`, both of which read the users cache fresh.
|
||||
- The carrier is private to the lookup. Consumers MUST NOT read `_INFINITO_APPLICATIONS_RENDERED`; they keep going through `lookup('applications')`, which is what makes the carrier transparent.
|
||||
- The carrier is the only materialised merged view a play MAY hold. The rule against a top-level `applications` fact above is unchanged.
|
||||
|
||||
## Adding an Application Entry ➕
|
||||
|
||||
@@ -69,6 +82,7 @@ applications:
|
||||
|---|---|
|
||||
| [applications.py](../../../../../../plugins/lookup/applications.py) | Runtime entry point for the `applications` lookup. |
|
||||
| [applications.py (cache)](../../../../../../utils/cache/applications.py) | Shared aggregation helper that builds and caches defaults (`get_application_defaults`, `get_variants`, `get_merged_applications`). |
|
||||
| [carrier.py](../../../../../../utils/cache/carrier.py) | The play-scoped carrier: fact name, `merged_applications_cache_key`, and the reader that serves the carried payload. |
|
||||
| [test_applications.py](../../../../../../tests/unit/python/plugins/lookup/test_applications.py) | Unit tests covering the full-dict, single-entry, override, strict missing, and non-strict missing cases. |
|
||||
|
||||
For the related users pattern see [users.md](users.md).
|
||||
|
||||
@@ -47,6 +47,8 @@ Rendering happens at **play runtime**, never during the build pipeline above. Th
|
||||
|
||||
`get_merged_applications` is invoked transitively by `lookup('applications', ...)` and by `lookup('config', application_id, 'services.<x>.<y>')`. Tasks that read service flags through these lookups always see real Python values, never raw Jinja strings.
|
||||
|
||||
The render runs once per play. The constructor stage parks the payload and its cache key in the host fact `_INFINITO_APPLICATIONS_RENDERED`, and every task's worker process (Ansible forks one per task) serves that payload instead of rendering again. See [applications.md](../artefact/files/plugins/lookup/applications.md) for the carrier contract.
|
||||
|
||||
A role's task that reads its own service flag therefore picks up the runtime-resolved value, even when the inventory carried a Jinja string for that flag: the conditional evaluates against the play's actual `group_names`, which reflects which other application groups the host joined.
|
||||
|
||||
## Interaction between the three inclusion mechanisms 🧬
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Render the applications SPOT once and park it as the play-scoped carrier fact.
|
||||
|
||||
Ansible forks a fresh worker per task, so the in-process render cache of
|
||||
``lookup('applications')`` is cold in every task until something renders in
|
||||
a place all later workers inherit. This action renders in the worker of one
|
||||
constructor task and returns the payload as a host fact; every later worker
|
||||
inherits that fact at fork time and ``get_merged_applications`` serves it
|
||||
instead of rendering. Sits after the token store in
|
||||
``tasks/stages/01_constructor.yml`` so the embedded user records already
|
||||
carry the loaded tokens::
|
||||
|
||||
- name: 🧊 Render the applications SPOT once and carry it into every task's worker
|
||||
applications_carrier: {}
|
||||
|
||||
The result is masked by the action itself: the fact is the whole rendered
|
||||
tree, about 0.4 MB with every application's credentials, and printing it at
|
||||
``-v`` is neither credential debugging nor readable.
|
||||
|
||||
Args:
|
||||
none. Any argument is an error.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ansible.errors import AnsibleActionFail
|
||||
from ansible.plugins.action import ActionBase
|
||||
from ansible.plugins.loader import lookup_loader
|
||||
|
||||
from utils.cache.carrier import APPLICATIONS_RENDERED_FACT
|
||||
|
||||
|
||||
class ActionModule(ActionBase):
|
||||
TRANSFERS_FILES = False
|
||||
|
||||
def run(
|
||||
self, tmp: Any = None, task_vars: dict[str, Any] | None = None
|
||||
) -> dict[str, Any]:
|
||||
result = super().run(tmp, task_vars)
|
||||
task_vars = task_vars or {}
|
||||
if self._task.args:
|
||||
raise AnsibleActionFail("applications_carrier takes no arguments")
|
||||
|
||||
lookup = lookup_loader.get(
|
||||
"applications", loader=self._loader, templar=self._templar
|
||||
)
|
||||
carrier = lookup.run([], variables=task_vars, carrier=True)[0]
|
||||
|
||||
result.update(
|
||||
changed=False,
|
||||
ansible_facts={APPLICATIONS_RENDERED_FACT: carrier},
|
||||
_ansible_no_log=True,
|
||||
)
|
||||
return result
|
||||
@@ -9,6 +9,7 @@ from utils.cache import _reset_cache_for_tests as _reset_runtime_lookup_cache
|
||||
|
||||
# nocheck: lookup-cache-import (this lookup IS the applications SPOT provider)
|
||||
from utils.cache.applications import get_merged_applications
|
||||
from utils.cache.carrier import merged_applications_cache_key
|
||||
|
||||
|
||||
def _reset_cache_for_tests() -> None:
|
||||
@@ -22,21 +23,39 @@ class LookupModule(LookupBase):
|
||||
variables: dict[str, Any] | None = None,
|
||||
**kwargs: Any,
|
||||
) -> list[Any]:
|
||||
"""
|
||||
Terms:
|
||||
0) full merged mapping
|
||||
1) application_id
|
||||
2) application_id, default value
|
||||
|
||||
Keyword arguments:
|
||||
roles_dir: roles tree to build from (defaults to the repository's)
|
||||
carrier: with no terms, return ``{"key": [...], "applications": {...}}``
|
||||
for the constructor to park in `_INFINITO_APPLICATIONS_RENDERED`
|
||||
"""
|
||||
terms = terms or []
|
||||
if len(terms) > 2:
|
||||
raise AnsibleError(
|
||||
"applications: expected 0, 1, or 2 terms: "
|
||||
"lookup('applications'[, application_id[, default]])"
|
||||
)
|
||||
carrier = bool(kwargs.get("carrier", False))
|
||||
if carrier and terms:
|
||||
raise AnsibleError("applications: carrier=True takes no terms")
|
||||
|
||||
variables = variables or getattr(self._templar, "available_variables", {}) or {}
|
||||
roles_dir = kwargs.get("roles_dir")
|
||||
applications = get_merged_applications(
|
||||
variables=variables
|
||||
or getattr(self._templar, "available_variables", {})
|
||||
or {},
|
||||
roles_dir=kwargs.get("roles_dir"),
|
||||
variables=variables,
|
||||
roles_dir=roles_dir,
|
||||
templar=getattr(self, "_templar", None),
|
||||
)
|
||||
|
||||
if carrier:
|
||||
key = merged_applications_cache_key(variables, roles_dir=roles_dir)
|
||||
return [{"key": [key[0], list(key[1])], "applications": applications}]
|
||||
|
||||
if len(terms) == 0:
|
||||
return [applications]
|
||||
|
||||
|
||||
@@ -180,6 +180,9 @@
|
||||
name: sys-token-store
|
||||
when: run_once_sys_token_store is not defined
|
||||
|
||||
- name: 🧊 Render the applications SPOT once and carry it into every task's worker
|
||||
applications_carrier: {}
|
||||
|
||||
- name: 🛠️ Preload shared services
|
||||
ansible.builtin.include_role:
|
||||
name: sys-service-loader
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from ansible.errors import AnsibleActionFail
|
||||
|
||||
from plugins.action.applications_carrier import ActionModule
|
||||
from utils.cache.carrier import APPLICATIONS_RENDERED_FACT
|
||||
|
||||
|
||||
class _FakeTask:
|
||||
def __init__(self, args=None):
|
||||
self.args = {} if args is None else dict(args)
|
||||
|
||||
|
||||
class _FakeLookup:
|
||||
def __init__(self, carrier):
|
||||
self.carrier = carrier
|
||||
self.calls = []
|
||||
|
||||
def run(self, terms, variables=None, **kwargs):
|
||||
self.calls.append((terms, variables, kwargs))
|
||||
return [self.carrier]
|
||||
|
||||
|
||||
def _make_action(task: _FakeTask) -> ActionModule:
|
||||
action = object.__new__(ActionModule)
|
||||
action._task = task
|
||||
action._loader = None
|
||||
action._templar = None
|
||||
return action
|
||||
|
||||
|
||||
class TestApplicationsCarrierActionPlugin(unittest.TestCase):
|
||||
def test_parks_the_carrier_as_a_masked_fact(self):
|
||||
carrier = {"key": ["roles", ["a", "b", "c", "d"]], "applications": {"x": {}}}
|
||||
lookup = _FakeLookup(carrier)
|
||||
action = _make_action(_FakeTask())
|
||||
with (
|
||||
patch(
|
||||
"plugins.action.applications_carrier.ActionBase.run",
|
||||
autospec=True,
|
||||
return_value={},
|
||||
),
|
||||
patch(
|
||||
"plugins.action.applications_carrier.lookup_loader.get",
|
||||
return_value=lookup,
|
||||
),
|
||||
):
|
||||
result = action.run(task_vars={"applications": {}})
|
||||
|
||||
self.assertIs(result["ansible_facts"][APPLICATIONS_RENDERED_FACT], carrier)
|
||||
self.assertTrue(result["_ansible_no_log"])
|
||||
self.assertFalse(result["changed"])
|
||||
terms, variables, kwargs = lookup.calls[0]
|
||||
self.assertEqual(terms, [])
|
||||
self.assertEqual(variables, {"applications": {}})
|
||||
self.assertTrue(kwargs["carrier"])
|
||||
|
||||
def test_rejects_arguments(self):
|
||||
action = _make_action(_FakeTask(args={"foo": 1}))
|
||||
with (
|
||||
patch(
|
||||
"plugins.action.applications_carrier.ActionBase.run",
|
||||
autospec=True,
|
||||
return_value={},
|
||||
),
|
||||
self.assertRaises(AnsibleActionFail),
|
||||
):
|
||||
action.run(task_vars={})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -9,6 +9,7 @@ from ansible.errors import AnsibleError
|
||||
|
||||
from plugins.lookup.applications import LookupModule, _reset_cache_for_tests
|
||||
from utils.cache import base as cache_base
|
||||
from utils.cache.carrier import merged_applications_cache_key
|
||||
from utils.cache.yaml import dump_yaml_str
|
||||
from utils.roles.mapping import ROLE_FILE_META_SERVICES
|
||||
|
||||
@@ -103,6 +104,32 @@ class TestApplicationsLookup(unittest.TestCase):
|
||||
)[0]
|
||||
self.assertEqual(result, default_value)
|
||||
|
||||
def test_carrier_form_returns_cache_key_and_full_mapping(self) -> None:
|
||||
variables = {
|
||||
"applications": {
|
||||
"web-app-foo": {"services": {"smtp": {"host": "override.example.org"}}}
|
||||
}
|
||||
}
|
||||
roles_dir = str(self._tmp / "roles")
|
||||
result = self.lookup.run(
|
||||
[], variables=variables, roles_dir=roles_dir, carrier=True
|
||||
)[0]
|
||||
key = merged_applications_cache_key(variables, roles_dir=roles_dir)
|
||||
self.assertEqual(result["key"], [key[0], list(key[1])])
|
||||
self.assertEqual(
|
||||
result["applications"]["web-app-foo"]["services"]["smtp"]["host"],
|
||||
"override.example.org",
|
||||
)
|
||||
|
||||
def test_carrier_form_rejects_terms(self) -> None:
|
||||
with self.assertRaises(AnsibleError):
|
||||
self.lookup.run(
|
||||
["web-app-foo"],
|
||||
variables={},
|
||||
roles_dir=str(self._tmp / "roles"),
|
||||
carrier=True,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -19,6 +19,10 @@ from pathlib import Path
|
||||
|
||||
from utils.cache import _reset_cache_for_tests
|
||||
from utils.cache import applications as cache_apps
|
||||
from utils.cache.carrier import (
|
||||
APPLICATIONS_RENDERED_FACT,
|
||||
merged_applications_cache_key,
|
||||
)
|
||||
from utils.cache.yaml import load_yaml_any
|
||||
from utils.roles.mapping import (
|
||||
ROLE_DIR_META_ADDONS,
|
||||
@@ -335,6 +339,71 @@ class TestGetMergedApplicationsRespectsOverrides(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestGetMergedApplicationsCarrierFact(unittest.TestCase):
|
||||
"""The constructor renders once and parks the payload with its cache key in
|
||||
`_INFINITO_APPLICATIONS_RENDERED`. A worker process whose in-process cache is
|
||||
empty serves that payload when the key matches and renders only when it
|
||||
does not.
|
||||
"""
|
||||
|
||||
def setUp(self) -> None:
|
||||
_reset_cache_for_tests()
|
||||
|
||||
@staticmethod
|
||||
def _variables() -> dict:
|
||||
return {
|
||||
"applications": {
|
||||
"web-app-foo": {"services": {"foo": {"image": "override"}}}
|
||||
}
|
||||
}
|
||||
|
||||
def test_matching_key_serves_the_carried_payload_without_rendering(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
roles = _seed_minimal_roles(Path(tmp))
|
||||
variables = self._variables()
|
||||
key = merged_applications_cache_key(variables, roles_dir=roles)
|
||||
carried = {"web-app-foo": {"carried": True}}
|
||||
variables[APPLICATIONS_RENDERED_FACT] = {
|
||||
"key": [key[0], list(key[1])],
|
||||
"applications": carried,
|
||||
}
|
||||
merged = cache_apps.get_merged_applications(
|
||||
variables=variables, roles_dir=roles, templar=None
|
||||
)
|
||||
self.assertIs(merged, carried)
|
||||
|
||||
def test_stale_key_falls_back_to_a_fresh_render(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
roles = _seed_minimal_roles(Path(tmp))
|
||||
variables = self._variables()
|
||||
key = merged_applications_cache_key(variables, roles_dir=roles)
|
||||
variables[APPLICATIONS_RENDERED_FACT] = {
|
||||
"key": [key[0], ["stale"] * len(key[1])],
|
||||
"applications": {"web-app-foo": {"carried": True}},
|
||||
}
|
||||
merged = cache_apps.get_merged_applications(
|
||||
variables=variables, roles_dir=roles, templar=None
|
||||
)
|
||||
self.assertNotIn("carried", merged["web-app-foo"])
|
||||
self.assertEqual(
|
||||
merged["web-app-foo"]["services"]["foo"]["image"], "override"
|
||||
)
|
||||
|
||||
def test_malformed_carrier_is_ignored(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
roles = _seed_minimal_roles(Path(tmp))
|
||||
for carrier in ("not-a-mapping", {"key": "x", "applications": {}}, {}):
|
||||
_reset_cache_for_tests()
|
||||
variables = self._variables()
|
||||
variables[APPLICATIONS_RENDERED_FACT] = carrier
|
||||
merged = cache_apps.get_merged_applications(
|
||||
variables=variables, roles_dir=roles, templar=None
|
||||
)
|
||||
self.assertEqual(
|
||||
merged["web-app-foo"]["services"]["foo"]["image"], "override"
|
||||
)
|
||||
|
||||
|
||||
class TestRecursiveMetaWalk(unittest.TestCase):
|
||||
"""`_build_role_base_config` walks the whole `meta/` tree: every
|
||||
`meta/**/*.yml` lands at the matching nested path under
|
||||
|
||||
Vendored
+3
@@ -10,6 +10,7 @@ This package contains the in-process cache layers Infinito.Nexus uses to keep CL
|
||||
| [files.py](files.py) | The project tree's full path list (one walk per process) and per-file UTF-8 contents (`read_text`). Used by lint/integration tests and CLI tools that scan the repo. |
|
||||
| [base.py](base.py) | Cross-cutting helpers: filesystem constants (`PROJECT_ROOT`, `ROLES_DIR`, `DEFAULT_TOKENS_FILE`), `_deep_merge`, cache-key + content-fingerprint signatures, the cross-domain re-entry guard, and the templar-render machinery. The only ansible-coupled symbol is `_render_with_templar`, which lazy-imports its dependency. |
|
||||
| [applications.py](applications.py) | Per-app variants + defaults + `get_merged_applications`. **Strictly ansible-free at import time** so the GitHub Actions runner-host CLI path (`cli.administration.deploy.development.init` → `plan_dev_inventory_matrix` → `get_variants`) keeps working without ansible installed. |
|
||||
| [carrier.py](carrier.py) | The play-scoped carrier of the rendered applications payload: the fact name `_INFINITO_APPLICATIONS_RENDERED`, `merged_applications_cache_key`, and the reader `get_merged_applications` consults before it renders. |
|
||||
| [users/](users/) | User definitions, token store hydration, alias materialization, `get_user_defaults`, `get_merged_users` (package; `users/placeholders.py` substitutes `DOMAIN_PRIMARY` / `ORGANIZATION` / `SOFTWARE_NAME` ahead of the templar render pass). |
|
||||
| [domains.py](domains.py) | Canonical-domains map derived from the merged applications view: `get_merged_domains`. |
|
||||
| [`__init__.py`](__init__.py) | Owns the package-level `_reset_cache_for_tests()` orchestrator that clears every cache plus the shared fingerprint memo in one call. |
|
||||
@@ -25,6 +26,8 @@ The layers are orthogonal: `yaml.py` caches "this file's parsed root", the `appl
|
||||
|
||||
Both layers are process-wide in-process caches. They intentionally do NOT track on-disk file mtimes; CLI tools are short-lived and the assumption "the on-disk file does not change while my process runs" holds. The single exception is `utils.cache.yaml.dump_yaml`, which writes through and evicts the cached entry for the path it just wrote so a tool that edits a file mid-run sees the new content on the next read.
|
||||
|
||||
Inside a play the process boundary is the task: Ansible forks a fresh worker per task, so `_MERGED_APPLICATIONS_CACHE` starts empty in every worker. `get_merged_applications` therefore also accepts the play-scoped carrier fact `_INFINITO_APPLICATIONS_RENDERED` (payload plus cache key, parked once by the constructor stage) and serves it when the key matches. See [applications.md](../../docs/contributing/artefact/files/plugins/lookup/applications.md) for the contract.
|
||||
|
||||
## Test-Only Helpers 🧪
|
||||
|
||||
Each module owns a private `_reset()` that clears its own cache dicts. The package-level `utils.cache._reset_cache_for_tests()` orchestrates all six (`base`, `applications`, `users`, `domains`, `yaml`, `files`) plus the shared fingerprint memo. Unit tests that exercise the cached paths MUST call it in `setUp` to guarantee clean state across test cases.
|
||||
|
||||
Vendored
+7
-5
@@ -25,8 +25,8 @@ from .base import (
|
||||
_render_with_templar,
|
||||
_resolve_override_mapping,
|
||||
_resolve_roles_dir,
|
||||
_stable_variables_signature,
|
||||
)
|
||||
from .carrier import _carried_applications, merged_applications_cache_key
|
||||
from .schema_credentials import extract_default_credentials
|
||||
from .yaml import load_yaml as _load_yaml_cached
|
||||
from .yaml import load_yaml_any as _load_yaml_any_cached
|
||||
@@ -396,14 +396,16 @@ def get_merged_applications(
|
||||
|
||||
variables = variables or {}
|
||||
resolved_roles_dir = _resolve_roles_dir(roles_dir=roles_dir)
|
||||
cache_key = (
|
||||
_cache_key(resolved_roles_dir),
|
||||
_stable_variables_signature(variables),
|
||||
)
|
||||
cache_key = merged_applications_cache_key(variables, roles_dir=resolved_roles_dir)
|
||||
cached = _MERGED_APPLICATIONS_CACHE.get(cache_key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
carried = _carried_applications(variables, cache_key)
|
||||
if carried is not None:
|
||||
_MERGED_APPLICATIONS_CACHE[cache_key] = carried
|
||||
return carried
|
||||
|
||||
defaults = get_application_defaults(roles_dir=resolved_roles_dir)
|
||||
|
||||
overrides = _resolve_override_mapping(variables, "applications", templar=templar)
|
||||
|
||||
Vendored
+73
@@ -0,0 +1,73 @@
|
||||
"""Play-scoped carrier of the rendered applications payload.
|
||||
|
||||
Ansible forks a fresh worker process per task, so the in-process
|
||||
`_MERGED_APPLICATIONS_CACHE` starts empty in every worker. The constructor
|
||||
stage renders once and parks the payload plus its cache key in the host fact
|
||||
named by `APPLICATIONS_RENDERED_FACT`; every later worker inherits that fact
|
||||
at fork time and `get_merged_applications` serves it instead of rendering.
|
||||
Ansible-free at import time like the rest of the package.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from .base import _cache_key, _resolve_roles_dir, _stable_variables_signature
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import os
|
||||
|
||||
APPLICATIONS_RENDERED_FACT = "_INFINITO_APPLICATIONS_RENDERED"
|
||||
|
||||
|
||||
def merged_applications_cache_key(
|
||||
variables: Mapping[str, Any] | None,
|
||||
*,
|
||||
roles_dir: str | os.PathLike[str] | None = None,
|
||||
) -> tuple[str, tuple]:
|
||||
"""Key under which one rendered applications payload stays valid.
|
||||
|
||||
Args:
|
||||
variables: The play's variable scope. Only the subset that
|
||||
`_stable_variables_signature` reads influences the key.
|
||||
roles_dir: Roles tree the payload is built from; defaults to the
|
||||
repository's `roles/`.
|
||||
|
||||
Returns:
|
||||
``(resolved roles dir, stable variables signature)``.
|
||||
"""
|
||||
return (
|
||||
_cache_key(_resolve_roles_dir(roles_dir=roles_dir)),
|
||||
_stable_variables_signature(variables),
|
||||
)
|
||||
|
||||
|
||||
def _carried_applications(
|
||||
variables: Mapping[str, Any], cache_key: tuple[str, tuple]
|
||||
) -> dict[str, Any] | None:
|
||||
"""Return the payload carried in `APPLICATIONS_RENDERED_FACT` when its
|
||||
key equals *cache_key*, else ``None``.
|
||||
|
||||
Args:
|
||||
variables: The play's variable scope, read for the carrier fact.
|
||||
cache_key: The key the caller would render under.
|
||||
"""
|
||||
carrier = variables.get(APPLICATIONS_RENDERED_FACT)
|
||||
if not isinstance(carrier, Mapping):
|
||||
return None
|
||||
key = carrier.get("key")
|
||||
payload = carrier.get("applications")
|
||||
if (
|
||||
not isinstance(payload, Mapping)
|
||||
or not isinstance(key, (list, tuple))
|
||||
or len(key) != 2
|
||||
or not isinstance(key[1], (list, tuple))
|
||||
):
|
||||
return None
|
||||
if [str(key[0]), [str(part) for part in key[1]]] != [
|
||||
cache_key[0],
|
||||
list(cache_key[1]),
|
||||
]:
|
||||
return None
|
||||
return payload
|
||||
Reference in new issue
Block a user