3 Commits

Include renames

Author	SHA1	Message	Date
kevinveenbirkenbachandClaude Opus 4.7	fe54854fe5	refactor(resolver): variant-aware combined resolver and per-round include ... Single SPOT for matrix-deploy topology ======================================= Before this change two paths derived the deploy plan from different sources and could disagree: * Group inclusion (`--include` / `group_names`): driven by `cli.meta.applications.resolution.combined.CombinedResolver`, which read each role's `meta/services.yml` raw and ignored `meta/variants.yml` entirely. A variant that flipped `services.<X>.enabled` to a literal `true` did NOT pull `<X>`'s provider role into the deploy plan. * Inventory bake (`host_vars/applications.<app>.*`): driven by the variant-aware payload merge in `cli.deploy.development.inventory`, so the role saw the variant-merged services map at deploy time. The split surfaced concretely for `web-app-wordpress` variant 0 (`services.discourse.enabled: true` in `meta/variants.yml`): the inventory baked `discourse.enabled = true`, the wp-discourse plugin task gate fired, and the install loop tried `docker exec discourse` against a container the variant-blind resolver had never pulled. This commit collapses both consumers onto the same per-round, variant-merged services map. The variant-aware resolver (Phase 1+2) is the additive plumbing, but the deploy/init wiring (Phase 3-5) is what actually changes behaviour at runtime. Variant-aware resolver (Phase 1+2) ---------------------------------- `role_introspection.load_shared_service_roles_for_app` accepts a `services_override` kwarg; `CombinedResolver.__init__` accepts a `{role_name: services_dict}` map and routes it through the loader. A `CombinedResolver` instance is round-specific when overrides are supplied — its per-role edge cache is variant-resolved, do NOT reuse across rounds. The combined-resolver CLI gains `--services-overrides <path>`. `cli.deploy.development.deps.resolve_run_after` threads the in-container path of the overrides JSON into the resolver invocation. Per-round include planner (Phase 3) ----------------------------------- `cli.deploy.development.inventory.plan_dev_inventory_matrix` now takes `primary_apps` (just the user-named apps) instead of a pre-resolved `include` list and returns a 4-tuple per round `(round_index, inv_dir, round_variants, include_R)`. Per round it: 1. Computes `round_variants` clamped to each primary's variant count. 2. Builds variant-merged services overrides for every role with a `meta/variants.yml`, picking that role's variant for the round (clamped) — so transitive deps with their own variants follow the round. 3. Runs `CombinedResolver(services_overrides=...)` per primary and unions the result with the primary apps in user order. 4. Extends `round_variants` with the discovered deps' clamped variant indices so the deploy step's per-round purge logic stays accurate. `build_dev_inventory_matrix` keeps its name but signature changes: it now takes `primary_apps` and an optional `include_filter` for SERVICES_DISABLED-style consumer filters. Init / deploy wiring (Phase 4-5) -------------------------------- `init.py` and `deploy.py` consume the per-round `include_R` directly, drop the old `resolve_deploy_ids_for_apps` pre-resolution step, and apply the SERVICES_DISABLED provider filter on the include set so the inventory `--include` flag and the final ansible deploy ids stay in lockstep. Behaviour invariance for non-variant roles ------------------------------------------ Apps with no `meta/variants.yml`, or without a `services.<X>.enabled` override on any variant, behave identically to before: * The override map for those roles is empty — resolver falls through to the disk-only path. * Per-round `include_R` collapses to the same set every round. * Inventory bake outputs the same `applications.<app>.*` shape. The behaviour change is scoped to roles that legitimately use variants to flip topology. In the current repo only `web-app-wordpress` is affected (variant 0 pulls `web-app-discourse` in; variant 1 leaves it out, exercising the multisite-without-discourse path). Test surface ------------ * `tests/unit/cli/deploy/development/test_inventory.py` updated for the new planner signature, mocking `_resolve_round_include` / `_build_services_overrides_for_round` so the planner's tuple shape is asserted independently of the live resolver. * `tests/unit/cli/deploy/development/test_matrix_deploy.py` adds a `_entry()` helper that defaults `include` to the keys of `round_variants`, so existing fixtures stay terse — round 0 of the deploy loop deploys whatever is in `include_R`, and the variant-aware planner naturally emits the same set. * `tests/unit/utils/cache/test_data.py` pins the planner contract (`primary_apps=`, 4-tuple plan entries with `include_R`). * `tests/utils/__init__.py` added so the `tests.lint.filesystem.test_init_files` lint test passes end-to-end (the directory existed in the tree without an __init__.py and tripped the lint). Verification ------------ * `make test` green (162 tests, 1 skipped) on the merged tree. * `make deploy-fresh-kept-apps APPS="web-app-wordpress" FULL_CYCLE=true SERVICES_DISABLED="matomo,email"` end-to-end: round 0 (variant 0, single-site + Discourse integration) and round 1 (variant 1, multisite) each ran PASS 1 (sync) and PASS 2 (async), all four passes with `failed=0 rescued=0`. Round 0's per-round include contained `web-app-discourse`; round 1's did not — exactly the variant-driven topology the resolver is supposed to produce. The unrelated Joomla CLI install race (CI run 25079786137 `Deploy web-app-joomla`) is parked separately as infinito-nexus/core#209; its candidate fix is in that issue body, not in this commit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-29 16:28:12 +02:00
kevinveenbirkenbachandClaude Opus 4.7	b3de2a4d7c	refactor(meta): land req-011 meta/info.yml + utils/cache cleanup ... req-011 — descriptive role-level metadata * Move logo / homepage / video / display out of meta/main.yml.galaxy_info into a new project-owned roles/<role>/meta/info.yml (file-root convention, no `info:` wrapper). * 78 role meta/main.yml files migrated via ruamel round-trip (key-order / quoting / inline-comments preserved). Roles with none of the four fields are left untouched (no empty info.yml created). * web-app-dashboard/lookup_plugins/docker_cards.py reads logo + display from meta/info.yml; description / galaxy_tags continue to come from galaxy_info (Galaxy-spec fields). * docker_cards.py now routes ALL its file reads through the process-wide caches: load_yaml_any for vars/main.yml, meta/main.yml, meta/info.yml; read_text from utils.cache.files for README.md. The orphan `import yaml` is gone. New utils/cache/files (renamed from tests/utils/fs) * The process-wide filesystem cache (read_text, iter_project_files, iter_project_files_with_content, _all_project_files) is not test-specific; it now lives under utils/cache/files.py next to the other process-wide caches. 34 consumer imports rewritten across tests/ and docs/. The empty tests/utils/ package is removed. * New unit suite tests/unit/utils/cache/test_files.py — 12 cases covering read_text, iter_project_files (skip-dirs, extensions, exclude_tests, exclude_dirs, walk-cache invariants), the combined iterator, non-UTF-8 silent skip, and _reset. utils/cache redundancy cleanup * applications.py + users.py + cli/create/inventory/services_disabler.py drop their direct yaml.safe_load calls and their private _load_yaml_mapping/_load_yaml_variant_list copies; everything routes through utils.cache.yaml.load_yaml / load_yaml_any. The dead utils.cache.base._load_yaml_mapping is removed. * users.py:_load_store_users (tokens.yml read) likewise migrated. * _load_yaml_variant_list moves out of base.py to applications.py (its only consumer) and is renamed _load_variants_overrides — also routes through the cached loader now. * utils.cache.yaml._reset_cache_for_tests is renamed _reset (kept as alias) for parity with the per-domain reset helpers. * utils.cache.files grows a _reset() that clears its two LRU caches. * utils.cache.__init__._reset_cache_for_tests now orchestrates ALL six caches (applications, users, domains, base, yaml, files) so tests get clean state across cache layers — needed because applications/users now share the yaml cache and would otherwise serve stale parses across reset boundaries. * utils.cache.files imports PROJECT_ROOT from base instead of redefining it. * tests/unit/utils/cache/test_data.py + test_base.py: obsolete TestLoadYamlMapping / TestLoadYamlVariantList classes deleted (covered by test_yaml.py + test_variants.py). * README.md updated for the new module list (files, yaml) and the six-module orchestrator. Galaxy schema lint * tests/lint/repository/test_role_meta_main_galaxy_schema.py: split galaxy_info field requirements into hard-required {author, description, license} and soft-recommended {min_ansible_version, platforms}. Soft-recommended -> emit a GitHub Actions ::warning:: annotation via utils.annotations.message.warning (titled "meta/main.yml galaxy_info" with file= pointing at the offender), no test failure. Rationale: min_ansible_version + platforms are only enforced by Galaxy when *publishing*, which Infinito.Nexus does not do. * Galaxy lint goes green (was 226 offenders before req-011 + license_url/repository/documentation strip; was 62 before this relaxation; now 0 hard offenders, 62 soft warnings). * roles/svc-opt-keyboard-color/meta/main.yml: backfill `license` (only role missing the hard-required field). New lint * tests/lint/repository/test_role_meta_info.py — enforces meta/info.yml file-root convention, the four-field allowlist (logo, homepage, video, display), and shape (logo.class string, homepage/video non-empty strings, display bool). Also fails if any meta/main.yml.galaxy_info still carries one of the four migrated fields. Docs * docs/contributing/design/services/layout.md: new section "Descriptive Role-Level Metadata — meta/info.yml" with allowed- fields table, optionality rule, materialised path, and consumer note. File-Layout table grows the meta/info.yml row; meta/main.yml row gets the explicit "no logo/homepage/video/display" note. req-011 added to header and Related Pages. * utils/cache/README.md: files.py + yaml.py rows added; six-module reset orchestrator documented. Validation * Unit: 1573 passed. * Lint: 55 passed. * Integration: 161 passed (1 skipped) via pre-commit hook. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-27 12:27:17 +02:00
kevinveenbirkenbachandClaude Opus 4.7	9c8cc1ddcb	test(utils/fs): cache project file walks + contents across test run ... Adds tests/utils/fs.py with process-level @lru_cache for both the project-wide os.walk and per-file read_text(), then migrates 26 tests (24 modified + 2 new) that previously re-walked/re-read the tree independently to share one cached pass. Headline win: tests/integration/jinja/test_variable_usage.py drops from ~26s to ~1.3s (≈19x), jinja cluster as a whole ~14x. Other migrated tests benefit from the shared content cache when the same file is read by multiple scans in one pytest run. Non-caching logic changes (kept intentionally): - test_variable_usage.py: added `if varname not in content: continue` fast-path before the line loop (pure short-circuit, identical result). - test_test_file_naming.py: exempt tests/utils/ from the test_*.py prefix rule so the new shared helpers don't trip the lint. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-23 20:29:54 +02:00