Replace the no-op `docker restart` recovery with `POST
/service/rest/v1/repositories/{name}/invalidate-cache` against each
apt-format proxy repo (Debian, Ubuntu, …) discovered dynamically
from Nexus' REST API. Container restart never re-validated cached
content; the new path makes Nexus actually re-fetch from upstream
on the next request and keeps repo definitions, blobstore, and
admin credentials intact.
Co-located changes:
- fresh-purged-app.sh: source scripts/meta/env/all.sh so the cache
override env (incl. INFINITO_PACKAGE_CACHE_FRONTEND_IP) is set
for the deploy CLI — without this, apt traffic bypasses Nexus.
- utils/nexus/list_apt_proxy_repos.py: stdin JSON → stdout name list,
exit 1 on malformed input / no matches; used by the bash helper
for discovery (parser lives outside the heredoc for testability).
- tests/unit/utils/nexus/test_list_apt_proxy_repos.py: 10 cases
covering filter rules, empty/invalid input, non-dict entries,
missing/empty names, and a realistic live payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the registry-cache or package-cache (Nexus / nginx frontend)
serves apt repository metadata whose `Valid-Until` window has elapsed,
Docker builds inside an Ansible-driven deploy abort with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
which surfaces as a generic non-zero exit from the wrapped deploy
command and forces the operator to manually wipe the caches before the
next attempt.
Add `scripts/tests/deploy/local/utils/cache-retry.sh` exposing
`deploy_with_cache_retry <label> -- <cmd> [args...]`. The helper runs
the wrapped command with combined stdout+stderr teed to a temp log,
captures the wrapped command's real exit code via PIPESTATUS (with a
disciplined `set +e` / restore-previous-state dance so the caller's
`set -e` does not abort the script before the helper can react), and
on non-zero exit grep's the captured log for the stale-cache
signature `Release file ... is expired` / `Valid-Until ... expired`.
If matched, the helper wipes both cache stacks via `make cache-clean`
(registry-cache + package-cache through
scripts/system/cache/clean.sh), runs `docker builder prune -af` and
`docker image prune -af` to drop cached build layers and unused base
images, and re-runs the wrapped command exactly once. Non-stale-cache
failures bubble up unchanged so the caller's existing error handling
is preserved. Live in the deploy/update script layer rather than the
Ansible handler because only the host-side scripts can reach the
registry-cache + package-cache containers and the host Docker engine.
Wire the helper into all five local deploy entrypoints:
- fresh-purged-app.sh (both `--full-cycle` and the sync-only branch)
- fresh-kept-app.sh
- fresh-kept-all.sh
- reuse-kept-app.sh (added SCRIPT_DIR/REPO_ROOT locally because the
script did not yet have them)
- reuse-kept-all.sh
CI deploy paths (scripts/tests/deploy/ci/*) are intentionally left
alone — CI runs do not share the developer's local cache stack and
they handle stale metadata differently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
09_deploy_reuse.sh ran `make deploy-reuse-kept-apps APPS=web-app-matomo`
without VARIANT, which makes reuse-kept-app.sh fall back to the
unsuffixed `${INVENTORY_DIR}` path. The matrix-init in step 08 only
writes `${INVENTORY_DIR}-0` / `-1` because matomo declares 2 variants
(see roles/web-app-matomo/meta/variants.yml; planner contract:
cli/administration/deploy/development/inventory/planner.py:107), so
the reuse pass always missed the inventory and aborted with exit 2.
Also surface sibling inventory dirs as suggestions when the lookup
misses, so the next caller hitting this gets a hint instead of a bare
ENOENT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.
Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
variant twice and ran the async pass against a host left in a
different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
as a runtime extra-var and required cross-stage coordination.
How:
- `roles/<role>/meta/variants.yml` (file format docs at
docs/contributing/artefact/files/role/variants.md): YAML list of
deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
variant 0 (= legacy payload). 6 roles get their dev overrides moved
here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
drops the retired `_active_variants` runtime selector. Loader always
returns variant 0 as the default; non-zero variants reach
`get_merged_applications` as inventory-level `applications.<app>`
overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
`plan_dev_inventory_matrix` (pure planner shared between init and
deploy), `build_dev_inventory_matrix` (materialises one folder per
round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
cleanup (only apps whose variant changed AND that are about to
re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
sync + async per round so the async re-deploy targets exactly the
host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
bash callers can inject per-run context into in-container helpers
without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
(Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
switch to a single init + single deploy `--full-cycle`. The
`make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
`INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
their in-container heredocs to dedicated helpers in
`scripts/tests/deploy/local/utils/`, removing the bash-in-bash
quoting hell.
Tests:
- `tests/unit/utils/test_variants.py` (renamed from
`test_application_variants.py`): loader edge cases, variant-zero
default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
per-round deploy filter, cleanup decisions, `--variant` pinning,
`--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
(new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.
Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
(folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
declaration" rule; `applications.md` renamed its SoT header for
consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Replace ONLY_APP with space-separated WHITELIST across CI and local/act deploy tests
* Rename build-test-matrix.sh to scripts/meta/resolve/apps.sh and apply optional whitelist filter
* Add manual CI entry workflow to run orchestrator with distros/whitelist inputs
https://chatgpt.com/share/698c406c-0968-800f-ae84-1bcca023363c
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.
https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
- Add local dedicated_distro.sh implementing the same two-pass deploy logic
as CI (ASYNC_ENABLED=false/true) without destructive cleanup
- Switch test-local-dedicated to pure ENV-based interface (TEST_DEPLOY_TYPE, APP)
- Refactor local deploy helpers into utils/ structure:
- move reset.sh to utils/reset.sh
- split cleanup into targeted purge helpers (entity, web, lib)
- Update Makefile targets to reflect new structure and workflows
https://chatgpt.com/share/6982804e-1f60-800f-809e-629d0fe62de3
