Commit Graph
20 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 cb1ff29951 fix(ci): decide discovery's row selection on the set that actually deploys
clone and covered_by both name a stand-in row: another row carries this one's coverage, so the sweep may drop it. Both were decided over every row and then applied to a filtered set, so a run that selected a subset deferred to a stand-in the filter had already removed - the row was cut in favour of a deploy that never happens. --whitelist svc-db-mariadb yielded no row at all, and the swarm-zombie job passed green while drilling nothing.

The sort's last-resort tie-breaker was drawn per process, so discovery and the plan renderer could name different assignments for the same run. INFINITO_DISCOVERY_SEED derives it from the row's own identity instead; the workflow feeds github.run_id, so the rotation still happens, once per run rather than once per process.

Measured against the previous order: on the mode-only filter both cut sets are identical (swarm 220 rows, 44 cut; compose 243 rows, 54 cut), so the full sweep is unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 dc6fbe24d9 feat(ci): give every variant its own row with rotating deploy axes
Variant bundling packed several variants onto one runner and made the
job count depend on storage sums. Every variant now gets its own matrix
row instead, and the two things that used to be fixed per workflow --
which deploy mode a row runs in, and whether it sits behind the node
onion -- become axes assigned per row.

Both axes are a deterministic rotation over the row's position and the
sweep number, never random, so a red job reproduces by re-running the
same sweep. Tor turns on sweep//2 so it does not flip in lockstep with
the mode: a row walks all four mode/tor combinations over four sweeps
instead of only two.

A role offers at most two modes in practice, since swarm requires its
own stack and host requires the absence of one.

Drops with the bundler: limits.py, the variant_bundle env handlers,
INFINITO_VARIANT_BUNDLE_SIZE/_MAX_STORAGE, the per-role
meta/tests.yml variant_bundle_size key and its only user, and the
ComplexityRow bundles/jobs fields with the --deploy-mode and --max-jobs
flags that consumed them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:04 +02:00
kevinveenbirkenbachandClaude Opus 5 0bed81c048 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Twelve conflicts, resolved by intent rather than by side.

Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.

web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').

handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.

svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.

Six follow-on failures the merge produced, fixed at the root:

- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
  design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
  the prefix lint forbids; the pair now lives in utils/github/variant/ as
  bundles.py and bundle_limits.py, with the -m entry point, the ruff
  per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
  three bare timeout literals; both route through gotoOnion and resolveTimeout
  now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
  distros that ship docker without it, the loop device is already detached when
  the probe failed before attaching, and modprobe zfs may fail because the
  /dev/zfs check decides.

make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 03:00:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 373795d84e style: apply autoformat wrapping to two long literals
Formatter output from make quality: the git ls-tree argument list in
complexity.model and the xwiki job-name dict in the report_failures test
wrapped across lines. No semantic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6651450894 feat(complexity): mark whether each role exists on origin/main
Add an in_main column (🌳) to the role complexity matrix: it resolves
the role directories present under roles/ at origin/main once and flags
every row true/false, so a role that is new on the current branch stands
out. When the ref cannot be resolved (a non-git checkout or an unfetched
main) the marker falls back to present for every role rather than
falsely reporting them new.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a1f5494f2c refactor(lifecycle): single-source the CI-tested lifecycle envelope
Read INFINITO_LIFECYCLES from default.env via a new tested_lifecycles() helper; derive TESTED_LIFECYCLES, the README overview envelope, the guide-select role filter, and the invokable-lifecycle lint from it. Add "maintained" to the envelope and drop the redundant hardcoded "alpha beta rc stable" strings from workflows, CLI help, and docs.

Guide selection now skips pre-alpha/eol roles (153 -> 117 testable).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:08:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 087fb0942c fix(ci): access discovery env keys strictly and justify the tie-breaker noqa
The discovery query read every INFINITO_* knob with a silent .get default, which the env lint forbids: defaults live in default.env, and an unset key must fail loudly instead of masking a broken environment. All five accesses go through os.environ[...] now; the documented empty-value branches (sort spec falling back to default.env, auto job slots) stay explicit logic. The complexity model's random tie-breaker suppression carries its required justification.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 17:11:04 +02:00
kevinveenbirkenbach 4ac72f49bc Autoformat 2026-07-18 16:30:52 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cb67adfffa feat(ci): select swarm deploys per variant and split discovery into three mode queries
The discovery SPOT (cli.meta.ci.query) now runs one query shape per mode:
compose and host stay whole-role with variants packed into bundles, swarm
ranks and budget-cuts on per-variant rows and emits role#variant tokens
through apps.sh into variant_bundles, so the cut can pick a subset of a
role's variants and the --matrix view is the selection priority.

Replace the --unique flag with visible dna/clone columns (heaviest role
per dna cluster stays ahead of the cut), add the integrated column and
sort, count host bundles like compose, and give the deploy plan an id
counter, per-variant weights and display-width-aware alignment matching
the matrix renderer. Add i8ma*/i8pa* inspection aliases per mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:26:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8e685e751f feat(complexity): test_* columns driven by meta/tests.yml skip, wide-safe symbols
meta/services.yml modes states where a role RUNS; a new meta/tests.yml
skip list deactivates TESTING a mode without touching that capability.
The table gains test_compose/test_swarm/test_host columns (base column
minus skip), --filter/--sort and the JSON/YAML payloads know the new
fields, and CI discovery (apps.sh) now filters on test_<mode> == true.
svc-storage-nfs-client is the first consumer and skips all three modes.

Also replaces the nine VS16-based header/lifecycle emojis (like the
old host/jobs/weight symbols) with East-Asian-Wide ones: terminals
render VS16 sequences 2 cells wide while the width model said 1,
shifting every header right of them off its column. A guard test pins
all table symbols to wide, VS16-free codepoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b5e5554883 fix(complexity): gate the host column on the lifecycle envelope
compose and swarm columns go through _tested_apps (invokable +
lifecycle envelope + per-mode skip) but host only checked non-stack +
modes.host.enabled, so host CI discovery reacted to every lifecycle:
61 roles instead of the 24 inside the alpha/beta/rc/stable envelope.
Route host through _tested_apps("host") in both the role-level and
per-variant row builders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 672e93246b feat(deploy): add host deploy mode + matrix symbol view
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').

Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7a6268fa96 refactor(deploy): replace skip with services modes
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Fable 5 1e3e3802a1 feat(meta/complexity): stack column and swarm-implies-stack
Add a `stack` column to the complexity matrix: true when a role renders
its own container stack (ships templates/*compose*.yml.j2), false for
host-only roles (backup crons, wireguard, swapfile), the swarm
infrastructure roles themselves, and central-webserver content
providers. Exposed as a sort key and filter field.

Make `swarm` imply `stack`: a role with no compose template of its own
is never a swarm stack-deploy target, so swarm is forced false when
stack is false. Because scripts/meta/resolve/apps.sh selects the swarm
deploy jobs via `--filter "swarm == true"`, this drops 17 stackless
roles (106 -> 89) from the swarm job list; all keep compose testing and
their swarm overlay coverage still runs through the webserver/proxy
role, so no real deploy coverage is lost.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:37:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e0559a10d8 feat(ci): configurable lifecycle envelope + mode selection
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.

Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:11:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e704e9a74d chore: autoformat + lint fixes; drop Invariant comment marker
autoformat reformats (settings sort, complexity/model.py, test_database_query.py, keycloak vars wrap); shorten akaunting swarm task name to <=80 chars; delete narration comments from keycloak vars; remove 'invariant' from the comment linter's valid marker set. make autoformat + make test verified green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:54:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 81a0c022fc feat(ci): filter-driven app discovery with a job budget
Drive the test-deploy matrix discovery through the complexity report instead of the type CLI: apps.sh selects the mode's tested roles via the 'compose'/'swarm' column (which already bakes in invokable + tested-lifecycle + per-mode skip), orders them coverage-first, and hard-caps the cumulative per-mode runner budget.

- complexity gains --deploy-mode {compose,swarm} (makes 'bundles'/'jobs' the runner count for that mode: compose packs variants into size/storage bundles, swarm runs one per deployable variant) and --max-jobs N (keep only the leading rows whose cumulative jobs stay below N).
- apps.sh emits whole role names; variant_bundles/output_apps.sh expand them into the matrix unchanged. Manual INFINITO_WHITELIST becomes a 'name %% {…}' filter clause applied before the cut, so a manual run only counts the apps it actually invokes.
- INFINITO_DEPLOY_MODE is now required; INFINITO_MAX_JOBS, INFINITO_DISCOVERY_SORT and INFINITO_REQUIRED_STORAGE move to default.env (the SPOT) + the passthrough handler, so no shell-side defaults remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:14:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 87c78275b8 feat(meta/complexity): multi-key sort, filter DSL, CI columns
Extend the 'meta roles applications complexity' report:

- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.

Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:33:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 72aa792db3 style(cli-meta): satisfy ruff in complexity model/render
Two pre-existing ruff findings in the complexity reporter (from an earlier
refactor) blocked `make autoformat` / `make test`: model.py used a bare
hashlib.sha1 for a non-security cache key (S324) and render.py built a list with
a for-append loop (PERF401). Pass usedforsecurity=False and switch to
list.extend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 18:31:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bc1184dd10 refactor(complexity-cli): split modules, add base/siblings + --unique
Split the single complexity __main__.py into graph (discovery + BFS),
model (ComplexityRow + scoring), render (table/json/string) and cli
(argparse + main); __main__ stays a thin entry point.

Add a 'base' column (sha1 of the role's name unioned with its embedded
services, sorted) so roles in the same mutual-embedding cluster share a
key, and a 'siblings' column listing the rest of that cluster. --unique
keeps one role per base. roundtrip's default app list now passes
--unique to skip redundant same-cluster deploys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00