mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-10 04:36:40 +00:00
0310629287b4d64d3be050f00a29d03a7790f00a
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  kevinveenbirkenbachandClaude Opus 4.7
|
8e293bd921 |
fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".
Fix:
- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
full_cycle, variant) from `passthrough.STATIC_KEYS` and from
`default.env`. They are per-invocation; .env is the persistent SPOT
and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
`os.environ.get("KEY", "")` so the missing-key path stops at "no
filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
(lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
name in every doc; routing-knob env vars now use the same name so
the cmdline form, the recipe env-prefix, and the consumer reads
all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
knobs whose alias differs from the env name (`type` →
INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
auto-exports cmdline variables into the recipe's shell env, so the
identical-name `$(if $(apps),apps="$(apps)")` lines were pure
noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
`extra_env` per `docker exec` call and forwards `disable` via
`-e`, so the container env never needs the value frozen at
compose-up time.
- Update the 3 actual leak call sites
(07/08/09_deploy_*.sh) to the make-alias form
(`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
the deliberate make/env parity, not a typo.
Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.
Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
10cb02461e |
fix(inventory): bake only variants.yml overrides into matrix-deploy --vars
Since
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
a10555cd08 |
feat(inventory): variant-aware credentials generation
InventoryManager.resolve_schema_includes_recursive previously read the
root role's base meta/services.yml when discovering shared providers,
regardless of which matrix variant the planner had pinned for the
round. For a role whose variants.yml enables additional services per
variant (e.g. svc-bkp-container-2-local variant 2 enabling
services.ldap), the base config had no `enabled+shared` flag for those
services, so the recursive resolver never visited the variant's
shared providers — their credentials were never generated, and the
deploy crashed in the provider's env.j2 with "credentials is defined
in schema at roles/<provider>/meta/schema.yml but not set in
application '<provider>'".
Wires the active variant through the existing inventory pipeline:
- utils/manager/inventory.py: InventoryManager(variant=None|int). When
set, load_role_config_by_path overlays variants.yml[N] onto the root
role's meta config (only for the root — transitive providers keep
their own base lookup). variant=None preserves the pre-refactor base
behavior for every standalone caller.
- cli/administration/inventory/credentials: new --variant N flag,
forwarded to InventoryManager.
- cli/administration/inventory/provision: new --app-variants <JSON>
flag ({app_id: variant_index}), forwarded per-app to the credentials
subprocess.
- cli/administration/deploy/development/inventory/builder.py: emits
--app-variants from spec.variant_selectors() when the round pins any
variant. Apps without variants.yml collapse to a single empty
overlay in get_variants so this is a no-op for them.
10 unit tests cover: variant overlay for root role, variant=None
fallback to base, out-of-range fallback, no overlay on non-root
paths, --variant CLI forwarding (set + default), --app-variants
generator forwarding (set + omit), builder --app-variants emission
(set + omit).
Caveat: bkp-c2l CI variant 1 still loads svc-db-openldap transitively
at runtime (outside the planner's variant closure) — a separate
sys-service-loader / role-meta bug, not addressed here.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
9a578952c3 |
feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
bd1a3a69ad |
fix(deploy/development/inventory/planner): split per-round include from union purge_set
`3c1d5d821` solved the cross-round backbone-abriss problem by making `include` equal the union of every variant's closure for every round. Side-effect: every matrix round then ALSO deployed the full union, so `08_deploy_performance.sh` saw `dashboard.infinito.example` still answering HTTP 200 after a matomo-only `deploy-fresh-purged-apps` — the deploy step was no longer matomo-only at all. Split the two concerns instead: - `include` (PlanEntry[3]) goes back to the per-round variant-closure. Both `init`/`builder` (`--include` for the inventory bake) and `deploy` (`--id` for ansible) consume it. Each round therefore deploys ONLY its own variant-closure again — the pre-`3c1d5d821` semantics. - `purge_set` (PlanEntry[4], new) carries the union across all rounds. `deploy.py` uses it for the inter-round wipe instead of the previous round's include. Result: between rounds the WHOLE matrix footprint is purged, not just the previous round's variant slice — a variant pinning `services.<X>.shared: false` cannot leave its bundled provider on the host for a sibling round that wires the same service shared. The backbone-abriss `3c1d5d821` was hunting stays fixed: whichever round needs the backbone re-deploys it immediately after the wipe (purge runs at the start of each round, not the end). `previous_round_include` tracking is gone. The first iteration skips the purge (nothing to wipe yet), which also covers `total_rounds == 1` and `--variant N` pinned single-folder modes cleanly. Unrelated, but staged together because it is part of the same investigation: - `roles/dev-nix/templates/nix.conf.j2`: enable `keep-derivations` / `keep-outputs`, so `nix-collect-garbage -d` (run by the aggressive prune step) does not drop the build closure of pkgmgr's flake; this was making every deploy re-fetch the build inputs from cache.nixos.org and overshoot pkgmgr's update timeout. - `roles/pkgmgr/tasks/01_core.yml`: raise the pkgmgr update timeout from 600s to 1800s and lower the retry count from 6 to 3, so a slow fetch on a single try no longer pushes total wallclock past three hours. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
a2b5269244 |
refactor(inventory): drop unused variant-only matrix mode
Commit 16f323319 added a strict variant-only include rule + conflict detector in `inventory/variants.py`, gated behind the presence of any primary's `meta/variants.yml`. In practice the rule never paid off: * The CI failure that motivated it (matomo's matrix transition leaving a stale dashboard nginx vhost -> 502) is already addressed by the nginx-purge primitive shipped in commit bfd04e4a1. * The existing `CombinedResolver` path already implements the "any role says true or `in group_names` -> pull" semantics via `utils.roles.applications.services.registry.is_explicit_truth`, so every needed service still gets pulled and services that all roles mark False stay out of the include. * The strict literal-equality conflict detector was a false-positive factory: 79/80 variant-equipped primaries failed it against the current repo state because roles legitimately disagree on `enabled` flags for shared services. This commit removes the entire variant-only branch and restores the single-path planner. Round-index clamping for non-primary deps is left exactly as before 16f323319 -- only the additions are reverted. The file-split refactor from 16f323319 is intentionally kept; only the `variants.py` submodule, the corresponding `test_variants.py`, and the variant-only gate test class in `test_planner.py` are removed. The `__init__.py` re-exports + `__all__` are pruned to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
f102c229f0 |
refactor(inventory): split monolith into SRP submodules + variant-only mode
Two changes ship together because the variant-only matrix-deploy logic
was written into the legacy 697-line inventory.py and then re-split:
* Source package (max 240 lines per submodule, well under the 300 budget):
- spec.py DevInventorySpec + PlanEntry data types
- payload.py _resolve_variant_payloads + _bake_overrides
- legacy_resolver.py CombinedResolver fallback (default-mode path)
- variants.py strict variant-only include resolution
- planner.py plan_dev_inventory_matrix + filter_plan_to_variant
- builder.py build_dev_inventory + build_dev_inventory_matrix
- __init__.py public-API re-exports + back-compat shims
* Test package mirrors the source 1:1 with a shared _fixtures.make_spec
to avoid duplicating the DevInventorySpec defaults across files.
* New variant-only matrix mode (gated on any primary shipping
meta/variants.yml): only services whose merged variant payload pins
both enabled and shared to literal True become inventory groups;
Jinja flags never qualify (group_names does not exist yet at build
time); run_after only orders the include, never expands it;
cross-primary disagreement on a (service-key, round) raises ValueError.
Default mode (no variants.yml on any primary) keeps the legacy
CombinedResolver path untouched.
* Docs in design/{inventory,variants}.md updated to point at the new
per-submodule paths; test_nginx_vhosts.py's Test classes now also
inherit directly from unittest.TestCase so the AST-level lint in
test_all_test_files_have_tests.py picks them up (transitive base via
NginxVhostsTestBase alone is not enough).
Coverage: 458 cli unit tests, 349 utils unit tests, 99 dev-deploy tests,
all green; ruff check clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|