mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 03:38:23 +00:00
kevinveenbirkenbachandClaude Fable 5.1
113113e553
Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.
The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.
The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.
Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
74 lines
2.3 KiB
Python
74 lines
2.3 KiB
Python
"""Play-scoped carrier of the rendered applications payload.
|
|
|
|
Ansible forks a fresh worker process per task, so the in-process
|
|
`_MERGED_APPLICATIONS_CACHE` starts empty in every worker. The constructor
|
|
stage renders once and parks the payload plus its cache key in the host fact
|
|
named by `APPLICATIONS_RENDERED_FACT`; every later worker inherits that fact
|
|
at fork time and `get_merged_applications` serves it instead of rendering.
|
|
Ansible-free at import time like the rest of the package.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Mapping
|
|
from typing import TYPE_CHECKING, Any
|
|
|
|
from .base import _cache_key, _resolve_roles_dir, _stable_variables_signature
|
|
|
|
if TYPE_CHECKING:
|
|
import os
|
|
|
|
APPLICATIONS_RENDERED_FACT = "_INFINITO_APPLICATIONS_RENDERED"
|
|
|
|
|
|
def merged_applications_cache_key(
|
|
variables: Mapping[str, Any] | None,
|
|
*,
|
|
roles_dir: str | os.PathLike[str] | None = None,
|
|
) -> tuple[str, tuple]:
|
|
"""Key under which one rendered applications payload stays valid.
|
|
|
|
Args:
|
|
variables: The play's variable scope. Only the subset that
|
|
`_stable_variables_signature` reads influences the key.
|
|
roles_dir: Roles tree the payload is built from; defaults to the
|
|
repository's `roles/`.
|
|
|
|
Returns:
|
|
``(resolved roles dir, stable variables signature)``.
|
|
"""
|
|
return (
|
|
_cache_key(_resolve_roles_dir(roles_dir=roles_dir)),
|
|
_stable_variables_signature(variables),
|
|
)
|
|
|
|
|
|
def _carried_applications(
|
|
variables: Mapping[str, Any], cache_key: tuple[str, tuple]
|
|
) -> dict[str, Any] | None:
|
|
"""Return the payload carried in `APPLICATIONS_RENDERED_FACT` when its
|
|
key equals *cache_key*, else ``None``.
|
|
|
|
Args:
|
|
variables: The play's variable scope, read for the carrier fact.
|
|
cache_key: The key the caller would render under.
|
|
"""
|
|
carrier = variables.get(APPLICATIONS_RENDERED_FACT)
|
|
if not isinstance(carrier, Mapping):
|
|
return None
|
|
key = carrier.get("key")
|
|
payload = carrier.get("applications")
|
|
if (
|
|
not isinstance(payload, Mapping)
|
|
or not isinstance(key, (list, tuple))
|
|
or len(key) != 2
|
|
or not isinstance(key[1], (list, tuple))
|
|
):
|
|
return None
|
|
if [str(key[0]), [str(part) for part in key[1]]] != [
|
|
cache_key[0],
|
|
list(cache_key[1]),
|
|
]:
|
|
return None
|
|
return payload
|