07_compose_minimal.sh claimed to disable 'all variable (group-conditional)
services' and then named seven by hand. tor was not among them, so a
deploy meant to be minimal pulled svc-net-tor, which hard-depends on
web-svc-mirror, whose group-conditional prometheus flag the closure
resolver treats as true — dragging the whole web stack in. Steps 12 and
13 passed no disable at all, which is why a mariadb swarm test deployed
keycloak, mailu, matomo, prometheus and dashboard.
The set is now derived from the contract the repo already enforces
(tests/integration/roles/meta/services/test_dynamic_flags.py): a key is
group-conditional when enabled OR shared is a Jinja expression. Checking
only enabled would miss mariadb, postgres, redis, memcached, typesense.
Exclusions are load-bearing, not cosmetic: dashboard because disabling
the primary app aborts init; javascript because it aliases cdn, which
web-app-dashboard declares a hard dependency with no runtime fallback;
mariadb and postgres because steps 12 and 13 test them.
variable_services assigns to a local first — disable="$(cmd)" does not
set the exit status of a simple command, so a failing CLI would have
yielded an empty disable set and steps 12/13 would have run the full
deploy and reported green.
roundtrip could not take disable at all; it now reaches both the compose
and the swarm leg.
Not verified by a deploy: 33 of the keys are new to step 07, and the
variant axis of svc-db-mariadb/svc-db-postgres collapses because tor is
its only dimension.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The swarm gate asked whether the drill ran by grepping the job log for
"Matrix-deploy <app>: provision/deploy/e2e/verify per round" — a copy of a
GitHub Actions step display name. d9a4498b3 renamed that step to
"🚀 Drill <apps> per distro: <distros>" and did not touch roundtrip.sh, so at
19d56f07e the literal existed in exactly one place in the tree: the grep itself.
It could no longer match anything, and four of the five workspace legs of run
30586213932 false-failed on deploys that had passed — svc-db-postgres converged,
answered on 5432, completed its DR drill and reported "Job succeeded" three lines
above the failure.
This is the second time the same coupling drifted. 0ca2ed56c is titled "align
roundtrip swarm gate to the live workflow step name" and records that the
previous stale marker made every swarm leg false-fail on a passing deploy.
Nothing cross-checked the literal, so it rots silently after each rename.
The marker now comes from the drill itself, printed after 07_assert_state.sh.
Every step before it runs under set -euo pipefail and the EXIT trap tears down
without printing, so the line is unreachable unless the drill completed for that
app. APP_ID and INFINITO_DISTRO are :?-guarded at the top of the script, so the
marker cannot degrade into an empty string that matches everything. The gate
matches it with grep -qF and names what it checked instead of asserting "empty
matrix" — a diagnosis that was never true and sent this investigation into the
wrong layer.
The gate itself stays. 214672a94 added it because act exits 0 on a zero-leg
matrix, which silently false-passed every local swarm roundtrip.
tests/unit/scripts/test_roundtrip_marker.py pins emitter and gate to the same
literal, pins the marker behind the state assertion, and fails if the old step
name reappears — the cross-check whose absence allowed both drifts.
Verified against the failing run's log: 0 occurrences of the old literal, while
DR drill PASSED, ran=1 skipped=0 and "Job succeeded" prove the drill reached the
point where the new marker is emitted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The discovery SPOT (cli.meta.ci.query) now runs one query shape per mode:
compose and host stay whole-role with variants packed into bundles, swarm
ranks and budget-cuts on per-variant rows and emits role#variant tokens
through apps.sh into variant_bundles, so the cut can pick a subset of a
role's variants and the --matrix view is the selection priority.
Replace the --unique flag with visible dna/clone columns (heaviest role
per dna cluster stays ahead of the cut), add the integrated column and
sort, count host bundles like compose, and give the deploy plan an id
counter, per-variant weights and display-width-aware alignment matching
the matrix renderer. Add i8ma*/i8pa* inspection aliases per mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A default like ${BACKUP_KEY_PATH:-/tmp/swarm-nfs-backup.key} duplicates
a path contract that belongs in a SPOT (default.env or a role vars
file) and drifts silently. The guard covers every ALL-CAPS variable
whose inline default starts with a slash, allows per-line nocheck
suppression, and its failure message forbids dodging the syntax via
conditional assignments. Legacy offenders carry explicit nocheck
reasons (act tooling runs before default.env exists, POSIX TMPDIR
convention).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the 'meta roles applications complexity' report:
- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.
Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the scripts/tests/environment deploy_* steps (07/08/09) to compose_*; add 12_swarm.sh (svc-db-mariadb via the swarm-* targets) and 13_roundtrip.sh (svc-db-postgres via make roundtrip), with MARIADB_APP/POSTGRES_APP in common.sh. Rename every make act-swarm-* target to make swarm-* (drop the act- prefix) across the Makefile (moved to the alphabetical slot), the deploy/act, deploy/swarm and e2e helper scripts, and the agent iteration docs. Strip mid-code narration comments from the touched scripts per the comment policy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gate grepped a stale step-name marker ("across variant rounds") that never appears
in the act log, so every swarm leg false-failed even on a passing deploy. Match the
actual workflow step name instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After the main merge the discover job emits objects ([{apps,variant,variant_slug}]) for the variant matrix, but the swarm workflow still consumed them via the string-form 'matrix: apps:', so matrix.apps became an object, act produced no legs, the deploy job never ran, and act exited 0: every local swarm roundtrip false-passed.
Switch the swarm matrix to 'include:' (matching the compose workflow) so matrix.apps is the app id again, and guard roundtrip.sh to require the 'Matrix-deploy <app> across variant rounds' marker in the log or exit 1, so a skipped deploy can never pass as green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the `2>/dev/null || true` that swallowed the roundtrip skip-check (a failed
get_role_skip silently became "no skip"), and replace `${PYTHON:-python3}` with
strict `${PYTHON}` in roundtrip.sh and run.sh. Both source
scripts/meta/env/load.sh (which sources python.sh, always setting PYTHON or
failing) before the usage, so the fallback was dead code that only masked a
PYTHON-unset setup error as a confusing system-python failure. Now such a
failure aborts loudly under set -euo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a `skip: [compose, swarm]` field on a role's meta/services.yml primary entity
so an app can opt out of the test-deploy matrix per deploy mode, replacing the
hardcoded exclude regex in apps.sh (the "TODO: Remove exclude condition"). The
filter threads through both discovery paths: the CI workflows pass
INFINITO_DEPLOY_MODE -> apps.sh -> `cli meta roles applications type
--skip-mode`, and roundtrip.sh honours the same get_role_skip SPOT per app/mode.
Apply skip: [compose, swarm] to svc-storage-nfs-server and svc-storage-nfs-client
(NFS storage is swarm-only and set up via the cluster inventory, not the matrix)
and web-opt-rdr-www (the previously regex-excluded app).
Covered by get_role_skip and skip_mode-filter unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Split the single complexity __main__.py into graph (discovery + BFS),
model (ComplexityRow + scoring), render (table/json/string) and cli
(argparse + main); __main__ stays a thin entry point.
Add a 'base' column (sha1 of the role's name unioned with its embedded
services, sorted) so roles in the same mutual-embedding cluster share a
key, and a 'siblings' column listing the rest of that cluster. --unique
keeps one role per base. roundtrip's default app list now passes
--unique to skip redundant same-cluster deploys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
make roundtrip with no apps= now iterates every application, most-complex first (complexity --sort total --order desc --format string). Also marks roundtrip.sh executable and moves the roundtrip target into alphabetical order.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
make roundtrip apps="<app> ..." iterates each role through every deploy mode in order (compose, then swarm), stopping at the first failure. Per-step logs under TMPDIR; each validated swarm cluster is released unless keep=true. The mode sequence is overridable (modes="compose swarm") so kubernetes can be appended later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5