Files
computer-playbook/cli/deploy/development/init.py
T
kevinveenbirkenbachandClaude Opus 4.7 d97ed7a9e8 feat(deploy): per-role matrix-deploy variants with folder-per-round model
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.

Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
  variant twice and ran the async pass against a host left in a
  different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
  as a runtime extra-var and required cross-stage coordination.

How:
- `roles/<role>/meta/variants.yml` (file format docs at
  docs/contributing/artefact/files/role/variants.md): YAML list of
  deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
  variant 0 (= legacy payload). 6 roles get their dev overrides moved
  here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
  drops the retired `_active_variants` runtime selector. Loader always
  returns variant 0 as the default; non-zero variants reach
  `get_merged_applications` as inventory-level `applications.<app>`
  overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
  `plan_dev_inventory_matrix` (pure planner shared between init and
  deploy), `build_dev_inventory_matrix` (materialises one folder per
  round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
  paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
  cleanup (only apps whose variant changed AND that are about to
  re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
  round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
  sync + async per round so the async re-deploy targets exactly the
  host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
  bash callers can inject per-run context into in-container helpers
  without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
  (Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
  test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
  switch to a single init + single deploy `--full-cycle`. The
  `make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
  `INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
  their in-container heredocs to dedicated helpers in
  `scripts/tests/deploy/local/utils/`, removing the bash-in-bash
  quoting hell.

Tests:
- `tests/unit/utils/test_variants.py` (renamed from
  `test_application_variants.py`): loader edge cases, variant-zero
  default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
  variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
  per-round deploy filter, cleanup decisions, `--variant` pinning,
  `--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
  (new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.

Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
  (folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
  flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
  admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
  recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
  declaration" rule; `applications.md` renamed its SoT header for
  consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:47:48 +02:00

169 lines
5.4 KiB
Python

# cli/deploy/development/init.py
from __future__ import annotations
import argparse
import json
import os
from typing import Any, Dict
from .common import make_compose, resolve_deploy_ids_for_apps
from .inventory import (
DevInventorySpec,
build_dev_inventory,
filter_plan_to_variant,
plan_dev_inventory_matrix,
)
from .storage import detect_storage_constrained
from ...meta.runtime import detect_runtime
def add_parser(sub: argparse._SubParsersAction) -> None:
p = sub.add_parser(
"init",
help="Create development inventory inside the infinito container.",
)
p.add_argument(
"--distro",
default=os.environ.get("INFINITO_DISTRO", "arch"),
choices=["arch", "debian", "ubuntu", "fedora", "centos"],
help="Target distro (compose env INFINITO_DISTRO).",
)
p.add_argument(
"--inventory-dir",
default=os.environ.get("INVENTORY_DIR"),
required=os.environ.get("INVENTORY_DIR") is None,
help=(
"Inventory directory base (default: $INVENTORY_DIR). "
"When the included apps declare more than one matrix-deploy "
"variant, sibling folders `<dir>-0`, `<dir>-1`, ... are "
"created; otherwise the directory is used as-is."
),
)
g = p.add_mutually_exclusive_group(required=True)
g.add_argument(
"--apps",
help="One or more application ids (will include run_after deps automatically).",
)
g.add_argument(
"--include",
help="Comma-separated list of application ids to include (no deps resolution).",
)
p.add_argument(
"--threshold-gib",
type=int,
default=100,
help="Free-space threshold (GiB) below which STORAGE_CONSTRAINED is enabled (default: 100).",
)
p.add_argument(
"--force-storage-constrained",
choices=["true", "false"],
default=None,
help="Override storage detection explicitly.",
)
p.add_argument(
"--vars",
default=None,
help="JSON object merged into inventory variables (overrides win).",
)
p.add_argument(
"--variant",
type=int,
default=_env_variant(),
help=(
"Pin the matrix init to a single round (zero-based index). "
"Useful when only one variant of a multi-variant app needs to "
"be (re-)materialised. Defaults to the VARIANT environment "
"variable when set, otherwise full-matrix mode."
),
)
p.set_defaults(_handler=handler)
def _env_variant() -> int | None:
raw = os.environ.get("VARIANT", "").strip()
if not raw:
return None
try:
return int(raw)
except ValueError:
raise SystemExit(
f"VARIANT environment variable must be an integer, got {raw!r}"
)
def handler(args: argparse.Namespace) -> int:
compose = make_compose(distro=args.distro)
if args.apps:
include = resolve_deploy_ids_for_apps(compose, args.apps)
else:
include = [x.strip() for x in (args.include or "").split(",") if x.strip()]
if not include:
raise SystemExit("Resolved include list is empty")
extra_vars: Dict[str, Any] | None = None
if args.vars is not None:
try:
parsed = json.loads(args.vars)
except Exception as exc:
raise SystemExit(f"--vars must be valid JSON: {exc}")
if not isinstance(parsed, dict):
raise SystemExit("--vars must be a JSON object")
extra_vars = parsed
if args.force_storage_constrained is not None:
storage_constrained = args.force_storage_constrained == "true"
else:
storage_constrained = detect_storage_constrained(
compose, threshold_gib=int(args.threshold_gib)
)
plan = plan_dev_inventory_matrix(
roles_dir=str(compose.repo_root / "roles"),
include=include,
base_inventory_dir=str(args.inventory_dir),
)
try:
plan = filter_plan_to_variant(plan, args.variant)
except ValueError as exc:
raise SystemExit(f"--variant: {exc}")
runtime = os.environ.get("RUNTIME") or detect_runtime()
services_disabled = os.environ.get("SERVICES_DISABLED", "")
for _round_index, inv_dir, round_variants in plan:
spec = DevInventorySpec(
inventory_dir=inv_dir,
include=tuple(include),
storage_constrained=storage_constrained,
runtime=runtime,
extra_vars=extra_vars,
services_disabled=services_disabled,
active_variants=round_variants,
)
build_dev_inventory(compose, spec)
if len(plan) == 1:
_, inv_dir, round_variants = plan[0]
non_zero = {a: i for a, i in round_variants.items() if i}
suffix = f" variants={non_zero}" if non_zero else ""
print(
f">>> Inventory initialized at {inv_dir} "
f"(include={','.join(include)} storage_constrained={storage_constrained}){suffix}"
)
else:
print(
f">>> Matrix inventory initialized in {len(plan)} folders "
f"(include={','.join(include)} storage_constrained={storage_constrained}):"
)
for round_index, inv_dir, round_variants in plan:
non_zero = {a: i for a, i in round_variants.items() if i}
print(
f" [round {round_index}] {inv_dir}"
+ (f" variants={non_zero}" if non_zero else "")
)
return 0