New ``cli.administration.deploy.swarm`` package wraps the strict
``cli.administration.deploy.dedicated`` runner with a layered closure
so swarm deploys never require operators (or test scripts) to
enumerate every infra role:
effective --id = seed (operator --id, else inventory_role_groups)
∪ dep-walk(seed via applications_if_group_and_all_deps)
∪ default-placement safety net
The dep-walker is the same SPOT
(``utils.roles.applications.in_group_deps.applications_if_group_and_all_deps``)
the runtime constructor consumes via ``lookup('applications_current_play')``,
so the CLI-side closure and the in-play role set stay in sync.
The dedicated CLI keeps strict explicit-only semantics: ``main`` was
split into ``parse_args`` + ``run_from_args`` so the swarm wrapper
can share parser, mode wiring, validator and runner verbatim and
only mutate ``args.id``.
Meta-side: ``svc-db-postgres`` gains ``default_placement: manager``
(semantic symmetry with mariadb). ``default_placement`` is now a
PLACEMENT hint only ("if deployed, run on the manager"); deploy
decisions come from the dep-walk. Apps that declare ``mariadb:``
pull svc-db-mariadb; apps that declare ``postgres:`` pull
svc-db-postgres; apps that declare neither get no central DB.
Shared utils extracted so the four consumers stay aligned:
* ``utils.inventory.groups.inventory_has_group``: YAML + INI static
inventory group probe (extracted from ValidDeployId).
* ``utils.roles.meta_lookup.{get_role_default_placement,
iter_roles_with_default_placement}``: single source of truth for
``default_placement`` discovery (lookup plugin is now a thin wrapper).
Test pipeline (``scripts/tests/deploy/swarm/*``):
* Heredoc-Python shells converted to proper modules under
``utils.tests.swarm.*`` and invoked via ``python -m``:
- extend_inventory.py (was 08_extend_inventory.sh)
- write_extras.py (was 09_write_extras.sh dynamic half)
- derive_includes.py (replaces 06's hardcoded includes array)
- write_hosts_entries.py (replaces 02's hardcoded /etc/hosts heredoc;
derives every canonical + www-redirect domain from APP_ID via
canonical_domains_map + generate_all_domains, so the test
works for any app in the matrix without per-app maintenance)
* 01_create_network.sh deleted (single docker command inlined in workflow).
* 11_deploy.sh deleted (swarm CLI now called directly in workflow).
* 09_write_extras.sh: static half moved to inventories/development/swarm.yml.
* Remaining shell scripts renumbered 01-13 without gaps.
* 08_wait_converge.sh: case block on DB_DEP replaced with single
conditional (container name == DB_DEP).
* 06_provision_inventory.sh: hardcoded includes array replaced with
`mapfile -t includes < <(python -m utils.tests.swarm.derive_includes)`.
* Workflow YAML step references updated to new script names + module
invocations.
New tests (39 covering the closure surface):
* tests/unit/cli/administration/deploy/swarm/test_closure.py (14)
* tests/unit/utils/inventory/test_groups.py (11)
* tests/unit/utils/roles/test_meta_lookup.py (+3 for default_placement)
* tests/unit/utils/roles/validation/test_deploy_id.py (unchanged
8 tests still pass after refactor onto the shared util).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".
Fix:
- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
full_cycle, variant) from `passthrough.STATIC_KEYS` and from
`default.env`. They are per-invocation; .env is the persistent SPOT
and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
`os.environ.get("KEY", "")` so the missing-key path stops at "no
filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
(lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
name in every doc; routing-knob env vars now use the same name so
the cmdline form, the recipe env-prefix, and the consumer reads
all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
knobs whose alias differs from the env name (`type` →
INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
auto-exports cmdline variables into the recipe's shell env, so the
identical-name `$(if $(apps),apps="$(apps)")` lines were pure
noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
`extra_env` per `docker exec` call and forwards `disable` via
`-e`, so the container env never needs the value frozen at
compose-up time.
- Update the 3 actual leak call sites
(07/08/09_deploy_*.sh) to the make-alias form
(`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
the deliberate make/env parity, not a typo.
Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.
Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Split the local deploy scripts into a verb-based hierarchy under
scripts/tests/deploy/local/deploy/apps/{initialize,reinstall,update}/
plus scripts/tests/deploy/local/deploy/bundles/, with each verb folder
exposing all.sh (full discovery) and selection.sh (one or more apps via
INFINITO_APPS). The Makefile targets retain their existing names and now
dispatch to the new paths. The reinstall and bundles flows take over the
down + up cycle that used to be a Make prerequisite, and bundles/fresh.sh
now correctly delegates to apps/reinstall/selection.sh (the previous
delegate path was broken after the move). Each new directory carries a
README.md documenting scope, entry points, and when to use which verb.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- utils/inventory/bundle_apps.py: replace the local REPO_ROOT
`Path(__file__).resolve().parents[2]` with the canonical
`from utils import PROJECT_ROOT`. Move `Iterable`/`Path` imports under
`TYPE_CHECKING` since they are now only referenced in type hints.
- Makefile: strip trailing newline from the redeploy-bundles target file.
- new utils.inventory.bundle_apps resolver aggregates and dedupes role
groups from inventories/bundles/{servers,workstations}/<name>/inventory.yml
- new make targets deploy-bundles (fresh-purged) and redeploy-bundles
(reuse-kept), driven by BUNDLES=... env, wrapped by
scripts/tests/deploy/local/deploy/{bundles,redeploy-bundles}.sh
- unit tests for the resolver under tests/unit/utils/inventory/
- document the new targets in docs/contributing/tools/make.md,
docs/contributing/actions/testing.md, inventories/bundles/README.md
- collapse multi-line Makefile comments to one-liners for consistency
- disable nextcloud bbb plugin auto-enable (commented out)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 4.7