perf(cache): let nested lookups reuse the unrendered applications view

Every cold render of the applications SPOT walks the templates of all
269 roles, and those templates call lookup('config'), lookup('tls'),
lookup('domain') and others. Each of these re-enters
get_merged_applications while the render guard is set, and each one
rebuilt the unrendered view from scratch: a deepcopy of all role
defaults plus the override merge, about 8 ms, never cached. A local
cold render of the development inventory made 351 such calls, 2.82 s
of a 6.86 s render.

The outer render now keeps a copy of its unrendered view and its cache
key on the render guard, taken before the inventory volumes are
popped, so it is the same view a nested call used to build. A nested
call with the same key returns it; any other key still builds its own.
The guard drops it when the render ends.

Measured on the same inventory: the nested calls took 0.07 s instead
of 2.82 s and the cold render 3.60 s instead of 6.86 s. The rendered
SPOT is unchanged: the old and new dumps differ in 69 user passwords,
which the development inventory regenerates on every read (two runs of
the old code differ in 68), and in one absolute checkout path, the only
other difference.
In CI each cold render took 40 to 77 s and the heaviest swarm cell of
#950 missed its budget by a few minutes (#984).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

tests/unit/python/utils/cache/test_applications.py

@@ -16,6 +16,7 @@ import tempfile
 import textwrap
 import unittest
 from pathlib import Path
+from unittest import mock
 
 from utils.cache import _reset_cache_for_tests
 from utils.cache import applications as cache_apps
@@ -422,6 +423,60 @@ class TestGetMergedApplicationsCarrierFact(unittest.TestCase):
                 )
 
 
+class TestNestedLookupsReuseTheUnrenderedView(unittest.TestCase):
+    def setUp(self) -> None:
+        _reset_cache_for_tests()
+
+    def test_nested_calls_during_one_render_build_the_defaults_once(self):
+        with tempfile.TemporaryDirectory() as tmp:
+            roles = _seed_minimal_roles(Path(tmp))
+            _write(
+                roles / "web-app-foo" / ROLE_FILE_META_SERVICES,
+                """
+                foo:
+                  image: "pre-{{ nested }}"
+                  version: "v-{{ nested }}"
+                """,
+            )
+            variables = {
+                "applications": {
+                    "web-app-foo": {"volumes": {"data": {"type": "volume"}}}
+                }
+            }
+            nested_views = []
+
+            class _NestingTemplar:
+                def __init__(self) -> None:
+                    self.available_variables: dict = {}
+
+                def template(self, value, **_kwargs):
+                    nested_views.append(
+                        cache_apps.get_merged_applications(
+                            variables=self.available_variables,
+                            roles_dir=roles,
+                            templar=self,
+                        )
+                    )
+                    return "rendered"
+
+            with mock.patch.object(
+                cache_apps,
+                "get_application_defaults",
+                wraps=cache_apps.get_application_defaults,
+            ) as defaults:
+                rendered = cache_apps.get_merged_applications(
+                    variables=variables, roles_dir=roles, templar=_NestingTemplar()
+                )
+
+            self.assertEqual(defaults.call_count, 1)
+            self.assertGreaterEqual(len(nested_views), 2)
+            self.assertTrue(all(view is nested_views[0] for view in nested_views))
+            self.assertIn("volumes", nested_views[0]["web-app-foo"])
+            self.assertEqual(
+                rendered["web-app-foo"]["services"]["foo"]["image"], "rendered"
+            )
+
+
 class TestRecursiveMetaWalk(unittest.TestCase):
     """`_build_role_base_config` walks the whole `meta/` tree: every
     `meta/**/*.yml` lands at the matching nested path under

utils/cache/README.md

@@ -28,6 +28,8 @@ Both layers are process-wide in-process caches. They intentionally do NOT track
 
 Inside a play the process boundary is the task: Ansible forks a fresh worker per task, so `_MERGED_APPLICATIONS_CACHE` starts empty in every worker. `get_merged_applications` therefore also accepts the play-scoped carrier fact `_INFINITO_APPLICATIONS_RENDERED` (payload plus cache key, parked by the constructor stage and again after every role, every nested application load and every persisted token) and serves it when the key matches. See [applications.md](../../docs/contributing/artefact/files/plugins/lookup/applications.md) for the contract.
 
+While one render runs, the re-entry guard also holds that render's unrendered view and its cache key. A nested `get_merged_applications` call from a lookup inside a role template (`config`, `tls`, `domain`, ...) with the same key returns that view instead of copying and merging the role defaults again. The view is shared by every nested call of the render, so callers MUST NOT mutate it. The guard drops it when the render ends.
+
 ## Test-Only Helpers 🧪
 
 Each module owns a private `_reset()` that clears its own cache dicts. The package-level `utils.cache._reset_cache_for_tests()` orchestrates all six (`base`, `applications`, `users`, `domains`, `yaml`, `files`) plus the shared fingerprint memo. Unit tests that exercise the cached paths MUST call it in `setUp` to guarantee clean state across test cases.

utils/cache/applications.py

@@ -406,15 +406,23 @@ def get_merged_applications(
         _MERGED_APPLICATIONS_CACHE[cache_key] = carried
         return carried
 
+    guarded = getattr(_RENDER_GUARD, "applications", False)
+    if guarded:
+        unrendered = getattr(_RENDER_GUARD, "unrendered_applications", None)
+        if unrendered is not None and unrendered[0] == cache_key:
+            return unrendered[1]
+
     defaults = get_application_defaults(roles_dir=resolved_roles_dir)
 
     overrides = _resolve_override_mapping(variables, "applications", templar=templar)
 
     merged = merge_with_defaults(defaults, overrides)
 
-    if getattr(_RENDER_GUARD, "applications", False):
+    if guarded:
         return merged
 
+    _RENDER_GUARD.unrendered_applications = (cache_key, copy.deepcopy(merged))
+
     for app_cfg in merged.values():
         if isinstance(app_cfg, dict):
             app_cfg.pop("volumes", None)
@@ -435,6 +443,7 @@ def get_merged_applications(
         )
     finally:
         _RENDER_GUARD.applications = False
+        _RENDER_GUARD.unrendered_applications = None
 
     for app_id in rendered:
         raw_volumes = get_canonical_volumes(app_id)