mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
kevinveenbirkenbachandClaude Fable 5.1
113113e553
Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.
The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.
The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.
Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
76 lines
2.5 KiB
Python
76 lines
2.5 KiB
Python
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
from ansible.errors import AnsibleError
|
|
from ansible.plugins.lookup import LookupBase
|
|
|
|
from utils.cache import _reset_cache_for_tests as _reset_runtime_lookup_cache
|
|
|
|
# nocheck: lookup-cache-import (this lookup IS the applications SPOT provider)
|
|
from utils.cache.applications import get_merged_applications
|
|
from utils.cache.carrier import merged_applications_cache_key
|
|
|
|
|
|
def _reset_cache_for_tests() -> None:
|
|
_reset_runtime_lookup_cache()
|
|
|
|
|
|
class LookupModule(LookupBase):
|
|
def run(
|
|
self,
|
|
terms: list[Any] | None,
|
|
variables: dict[str, Any] | None = None,
|
|
**kwargs: Any,
|
|
) -> list[Any]:
|
|
"""
|
|
Terms:
|
|
0) full merged mapping
|
|
1) application_id
|
|
2) application_id, default value
|
|
|
|
Keyword arguments:
|
|
roles_dir: roles tree to build from (defaults to the repository's)
|
|
carrier: with no terms, return ``{"key": [...], "applications": {...}}``
|
|
for the constructor to park in `_INFINITO_APPLICATIONS_RENDERED`
|
|
"""
|
|
terms = terms or []
|
|
if len(terms) > 2:
|
|
raise AnsibleError(
|
|
"applications: expected 0, 1, or 2 terms: "
|
|
"lookup('applications'[, application_id[, default]])"
|
|
)
|
|
carrier = bool(kwargs.get("carrier", False))
|
|
if carrier and terms:
|
|
raise AnsibleError("applications: carrier=True takes no terms")
|
|
|
|
variables = variables or getattr(self._templar, "available_variables", {}) or {}
|
|
roles_dir = kwargs.get("roles_dir")
|
|
applications = get_merged_applications(
|
|
variables=variables,
|
|
roles_dir=roles_dir,
|
|
templar=getattr(self, "_templar", None),
|
|
)
|
|
|
|
if carrier:
|
|
key = merged_applications_cache_key(variables, roles_dir=roles_dir)
|
|
return [{"key": [key[0], list(key[1])], "applications": applications}]
|
|
|
|
if len(terms) == 0:
|
|
return [applications]
|
|
|
|
application_id = str(terms[0]).strip()
|
|
default_provided = len(terms) == 2
|
|
default_value = terms[1] if default_provided else None
|
|
|
|
if application_id in applications:
|
|
return [applications[application_id]]
|
|
|
|
if default_provided:
|
|
return [default_value]
|
|
|
|
raise AnsibleError(
|
|
f"applications: application '{application_id}' not found. "
|
|
f"Known application ids: {sorted(applications)}"
|
|
)
|