Files
computer-playbook/utils/cache/carrier.py
T
kevinveenbirkenbachandClaude Opus 5.5 778c18fb23 perf(constructor): carry the applications SPOT into the bootstrap roles
The constructor parked no applications carrier between the merge block
and the token store, so every lookup in the svc-swarm-node, svc-storage,
svc-registry and svc-virt bootstrap rendered the SPOT cold, 25 to 30 s
each on a loaded host.

A carrier there was unsafe: in swarm, svc-storage-nfs-server mounts the
NFS export over the controller's DIR_VAR_LIB, so the tokens file the
render reads can appear in the middle of that section, and the cache key
did not notice. merged_applications_cache_key now includes the tokens
file's path, mtime and size, as the users cache key already did, so a
carrier rejects itself once the tokens change. A carrier parked after
the add_host tasks then serves the bootstrap roles. It is parked only in
swarm mode: in compose the section runs in about a second, so a carrier
there would add a render without saving one.

In a swarm drill of web-app-prometheus up to the known NFS probe-write
failure, "Resolve the NFS server backend flavor" went from 29.6 s to
0.07 s, "Probe-mount the NFS export" from 26.4 s to 0.4 s, "Mount
cluster-shared infinito state on controller" from 23.7 s to 0.5 s and
"Ensure Docker-In-Docker DNS works" from 30.0 s to 3.6 s; the run to
that point took 768 s instead of 1735 s.

Closes #925

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 03:21:17 +02:00

84 lines
2.6 KiB
Python

"""Play-scoped carrier of the rendered applications payload.
Ansible forks a fresh worker process per task, so the in-process
`_MERGED_APPLICATIONS_CACHE` starts empty in every worker. The constructor
stage renders once and parks the payload plus its cache key in the host fact
named by `APPLICATIONS_RENDERED_FACT`; every later worker inherits that fact
at fork time and `get_merged_applications` serves it instead of rendering.
Ansible-free at import time like the rest of the package.
"""
from __future__ import annotations
from collections.abc import Mapping
from typing import TYPE_CHECKING, Any
from .base import (
_cache_key,
_resolve_roles_dir,
_stable_variables_signature,
_tokens_file_signature,
)
if TYPE_CHECKING:
import os
APPLICATIONS_RENDERED_FACT = "_INFINITO_APPLICATIONS_RENDERED"
def merged_applications_cache_key(
variables: Mapping[str, Any] | None,
*,
roles_dir: str | os.PathLike[str] | None = None,
) -> tuple[str, tuple]:
"""Key under which one rendered applications payload stays valid.
Args:
variables: The play's variable scope. Only the subset that
`_stable_variables_signature` reads, and the tokens file it
resolves to, influence the key.
roles_dir: Roles tree the payload is built from; defaults to the
repository's `roles/`.
Returns:
``(resolved roles dir, stable variables signature extended by the
tokens file's path, mtime and size)``.
"""
from .users import _resolve_tokens_file
tokens = _tokens_file_signature(_resolve_tokens_file(variables))
return (
_cache_key(_resolve_roles_dir(roles_dir=roles_dir)),
(*_stable_variables_signature(variables), *(str(part) for part in tokens)),
)
def _carried_applications(
variables: Mapping[str, Any], cache_key: tuple[str, tuple]
) -> dict[str, Any] | None:
"""Return the payload carried in `APPLICATIONS_RENDERED_FACT` when its
key equals *cache_key*, else ``None``.
Args:
variables: The play's variable scope, read for the carrier fact.
cache_key: The key the caller would render under.
"""
carrier = variables.get(APPLICATIONS_RENDERED_FACT)
if not isinstance(carrier, Mapping):
return None
key = carrier.get("key")
payload = carrier.get("applications")
if (
not isinstance(payload, Mapping)
or not isinstance(key, (list, tuple))
or len(key) != 2
or not isinstance(key[1], (list, tuple))
):
return None
if [str(key[0]), [str(part) for part in key[1]]] != [
cache_key[0],
list(cache_key[1]),
]:
return None
return payload