Commit Graph
758 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 bc047de312 feat(git): gate merges on signed incoming commits
A merge commit bypassed every signature check: test-signed only inspects
HEAD and the pre-commit hook does not run for merge commits. Add
scripts/git/assert_merge_signed.sh, which fails when any commit the merge
brings in (HEAD..MERGE_HEAD) reports %G? status N, wire it up as
make test-merge-signed, and register it as a pre-merge-commit hook so
pre-commit install now also installs that hook type.

Fast-forward merges create no commit and skip the hook (use --no-ff to
gate them); conflicted merges fall back to the regular pre-commit and
pre-push gates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:26:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3dba48cac9 chore(backup): drop a redundant stdout default and apply shfmt
The pre-deploy snapshot task only evaluates changed_when for executed loop
items, where the script module always sets stdout, so the default('') filter
hid nothing. pre_deploy_snapshot.sh is reindented by shfmt with no logic
change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 094c02a69d feat(diagnostics): single SPOT rescue script with CI-uploaded snapshots
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.

The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.

Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1dc58df31e fix(backup): unify the pre-deploy snapshot into one gated loop
The container and NFS pre-deploy snapshots ran two near-identical role
scripts and two constructor tasks. Merge them into one shared script under
scripts/system/backup/ whose content check handles a file (databases.csv)
or a directory (export base), and drive both from a single looped
constructor task keyed by role, unit, and source. Each iteration keeps its
own group_names gate, its own timeout, and reports changed only when it did
not skip.

The shared script also reaches the fresh-host skip loudly: systemctl
list-unit-files exits non-zero when the version glob matches nothing, so the
listing now sits in an if instead of a bare command substitution that
aborted the snapshot under set -euo pipefail (the compose test-development
failure). The volume e2e test starts the unit through an if and dumps the
unit journal on failure so the real start error surfaces.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1a908a6e5b perf(tests): parallelize suites, hoist fixtures, gate ansible-lint
Run the python suites through pytest-xdist when pytest is importable
(id-level collection parity with unittest discover verified per suite;
serial unittest stays the fallback), with -rP so warnings printed by
passing tests, like the update annotations, still reach the log. run.sh
self-provisions the dev extras through scripts/install/dev-extras.sh,
which stamps inside the venv and skips read-only venvs; make onboard
pre-warms the same path in the running container.

Hoist per-test cache resets and roles scans to setUpClass in the lookup
and resolvable suites (a per-test fingerprint-memo reset closes an
id()-reuse cache-poisoning flake the hoist exposed), inject generation
names into the nfs backup script test instead of sleeping, and narrow
ansible-lint scope to real ansible content. ansible-lint additionally
sits behind a content fingerprint gate that skips the run when nothing
it walks has changed; without a usable git (container bind mounts) the
gate disables itself and the full run remains.

Drop the import-linter layer and ignore for the deleted deps module;
the contract now locks compose -> common -> env. Measured on the host:
unit 101s to 20s, integration 90s to 24s, lint tests 43s to 19s,
repeated ansible-lint 53s to 0.4s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 21:20:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 939a386247 fix(package-cache): persist rotated Nexus admin password
Store the rotated admin password inside the container (umask 077) during
bootstrap and read it back on later runs, instead of assuming the
configured INFINITO_CACHE_PACKAGE_ADMIN_PASSWORD still matches; fall back
to the configured value when the stored file is missing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f08d71534a feat(tooling): operator alias, shortcut, skill, cheat and onboard system
Add the i8* terminal aliases (make install-alias) and ai8* agent
conversation shortcuts (docs/.../agents/alias.md), move skill
distribution to the external INFINITO_SKILLS_REPOSITORY (make
install-skills), and add make cheat, make alias and make onboard. Introduce
custom.env operator overrides (win over defaults and handlers) and pin
INFINITO_ALIAS_MD / INFINITO_ALIAS_REPOSITORY / INFINITO_SKILLS_REPOSITORY
in default.env. Drop the in-repo skills-lock.json and its update-skills CI
job. AGENTS.md instructs agents to suggest the matching cli/prompt alias
when an operator writes something an alias covers. Lints enforce alias
sorting, i8/ai8 prefixes, no consonant clusters, full make-target
coverage, and terminal/agent shortcut non-overlap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bd37f1855b fix(ci): resolve package install script from default.env in env bootstrap
01_install.sh runs before python3 exists, so load.sh is skipped and
INFINITO_PACKAGE_INSTALL_SCRIPT was unset; grep it from default.env like
scripts/install/package.sh already does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 00:39:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8519642373 refactor(storage): SPOT the nfs export base and client version in their roles
storage.nfs shrinks to the deployment-specific server address: export_base moves into svc-storage-nfs-server's services.yml (the server dictates where exports root) and the mount protocol version into svc-storage-nfs-client's services.yml (the client owns how it mounts). Both are read through utils.storage.nfs loaders by the nfs lookups, the env handler and 00_topology, eliminating the unrendered-variables class for these values; the group_vars assert shrinks accordingly and the swarm test extras stop restating role defaults.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:54:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9f8aa6e9b6 refactor(env): SPOT the install scripts and derive the nfs export base from group_vars
INFINITO_PYTHON_INSTALL_SCRIPT and INFINITO_DOCKER_CLI_INSTALL_SCRIPT join the default.env SPOT: the Dockerfile consumes them as required args (:? in the RUN as last defense), all three image builders (compose.yml, build.sh, push.sh) pass them hard-required, the shell consumers read the SPOT via default.env grep and the Makefile via BASH_ENV. INFINITO_SWARM_NFS_EXPORT_BASE is no longer a static default: a new env handler derives it from the deploy SPOT group_vars/all/15_storage.yml, 00_topology greps that file (and the STATE_SUBDIR constant) directly, and the swarm test extras stop restating the group_vars value. Stale hadolint directives are restored and recognised; leftover narration comments in the touched files are dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9f53210fc6 fix(swarm-test): source the env SPOT in the node bootstrap script
04_bootstrap_nodes.sh consumed INFINITO_PACKAGE_INSTALL_SCRIPT and INFINITO_NODE_SRC_DIR from the ambient environment; local make targets provide them via BASH_ENV, but plain workflow steps do not, so every swarm CI job died at the required-var guard. Source load.sh explicitly like copy_worktree_to_node.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bcd6e82f14 refactor(install): SPOT the package install script path
INFINITO_PACKAGE_INSTALL_SCRIPT in default.env replaces the scripts/install/package.sh literal in the Dockerfile (new required build arg), the environment install step, and both swarm bootstrap call sites. Shell consumers hard-require the variable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 17:46:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a7fbb94972 fix(swarm-test): bound diagnostics against wedged dockerd and SPOT the nfs state path
A wedged DinD dockerd hung a single diagnostics docker exec until the 360-minute job timeout cancelled the job together with its artifact uploads (job 86013153099 sat 5h in service logs). Route every diagnostics exec through a timeout wrapper, tail-limit the log dumps, bound showmount, cap the collect step at 30 minutes, and dump the pending services' log tails once on the final converge failure. The NFS state path moves into a single source of truth: STATE_SUBDIR in utils/storage/nfs.py feeds the nfs_state_subdir lookup (deploy side) and the derived INFINITO_SWARM_NFS_STATE_PATH env var (harness side); the diagnostics script requires both path vars instead of defaulting, and the swarm inventory drops its redundant export_base restatement.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 17:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2b44d9f7f9 fix(test-e2e-cli): stage CLI test assets on the node and SPOT the node src dir
The per-role existence asserts and the test script path were controller paths evaluated on the remote swarm node (docker connection, node checkout under /opt/infinito-nexus), so every svc-runner swarm job failed on a stat that could never succeed. Assert both required assets in one localhost-delegated loop, stage roles/<app>/files/test onto the node, run the staged script, and delegate the optional defaults/vars probes to the controller (they silently skipped before). svc-runner's test reaches its sources via RUNNER_SRC_DIR (compose: INFINITO_SRC_DIR dev-container mount; swarm: node checkout). The node checkout path itself moves into a single source of truth: INFINITO_NODE_SRC_DIR in default.env, consumed by the bootstrap/copy scripts, the swarm-playwright target, and the swarm inventory extras.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 17:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 09a131d007 refactor(placement): rename meta key default_placement to placement
Rename the meta/services.yml key across all roles, the roles_with_default_placement lookup (now roles_with_placement), the get_role_default_placement / iter_roles_with_default_placement helpers, and every consumer (swarm closure, compose_volumes filter, nfs_prep, container deploy template, constructor stage, extend_inventory) plus their tests and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 02:19:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fbe50d10ed fix(svc-runner): make the runner role swarm-compatible
Workers failed 'docker build: no Dockerfile': the build context, compose file, .env and runner binary were staged on the stack host only while every node deploys its own node-local runner stack. Build and push the image once on the manager (RUNNER_IMAGE_REF via the swarm registry, 04_image.yml) and let workers pull it; render compose.yml (template with lstrip_blocks, matching the replaced stack_host_template action), .env and the extracted run.sh on every node. Add a node_local kwarg to the compose_networks/container_networks/compose_restart/compose_only lookups so this node-local compose stack renders a bridge default network (workers cannot create overlays), keeps restart: unless-stopped (ephemeral runners died after one job) and keeps container_name under swarm; default is unchanged for every other role. Treat build/install.stamp as stale when the venv is missing and exclude ./build from the nested-src copy, closing the nested-e2e ruamel failure class. Regression tests for the node_local render paths; the network include tests now accept lookup kwargs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a6dae38590 fix(ci): auto-build act runner image and probe toolless containers
act/workflow.sh: build local/act-runner-fixed:latest on demand when that tag is selected but the image is absent, avoiding an image-not-found failure. swarm/_context.sh: probe_app_reachable falls back to a bash /dev/tcp connect and finally the container's own Docker health status, so toolless images (e.g. fider's debian-slim without curl/wget/nc, which exit 127 on every in-container probe) are no longer reported unreachable. Both additions are additive: the curl/wget/nc path and the non-matching act tag are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 00:55:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 558c24c85d style(install-act): apply autoformat indentation to act.sh
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:08:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 69dcf690ee fix(ci): provision act via make install-act for act-based targets
12_swarm ran 'make swarm-zombie' which invoked act, but act was never installed in the runtime container, giving 'act: command not found' (Error 127). Add an on-demand installer (scripts/install/act.sh) fronted by 'make install-act' (plus 'make install-act-update' to force the latest), and make the act-based targets (act-workflow, swarm-zombie) depend on it. The static release binary runs on all five default distros; verified on debian, ubuntu, fedora, centos, manjaro.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6c33e66d71 fix(swarm): snapshot PIPESTATUS before indexing in the report collector
The previous commit read 'producer_rc=${PIPESTATUS[0]}' then
'consumer_rc=${PIPESTATUS[1]}', but the first scalar assignment resets
PIPESTATUS to a one-element array, so the second read hit an unbound
index and 'set -u' aborted the collector with exit 1 - failing every
swarm job whose deploy and tests actually passed (jenkins, pixelfed).
Snapshot the whole array in one statement, then index it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 23:22:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 72bb1f7fb0 test(swarm): fail the report collector on a 15-minute docker exec timeout
The collector piped 'docker exec tar | tar' with '|| true', so a hung
'docker exec' blocked the run indefinitely and failures were swallowed.
Bound it with 'timeout 900'; a producer exit of 124 (manager hung) now
fails the collector via PIPESTATUS, while other collection errors stay
best-effort so partial reports do not fail the job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:41:43 +02:00
kevinveenbirkenbachandClaude Fable 5 c6ed8234b4 fix(swarm-tests): flavor-agnostic NFS assert mount and busybox-safe probe
The post-reschedule assert mounted the state export with vers=3, which
ganesha does not serve ('Protocol not supported'), so the marker check
false-negatived and killed the gitea roundtrip; reproduced and the
v4 mount shown to read the marker on the live cluster. Replace the
single mount with a cascade covering every production client_src form:
nfs4 vers=4 against the full path (ganesha), against :/ (kernel fsid=0
root), then the previous vers=3 full-path arm (kernel v3). The block
also returned the umount status instead of the grep result, letting a
missing marker pass silently; propagate the grep rc explicitly.

The in-container reachability probe's last arm used bash /dev/tcp, but
e.g. the ejabberd image ships neither bash nor curl; use busybox
nc -z instead and move the probe chain into probe_app_reachable() in
_context.sh alongside a NFS_CHECK_MOUNTPOINT SPOT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:34:29 +02:00
kevinveenbirkenbachandClaude Fable 5 0e644f8599 fix(joomla): stream plugin dir via exec stdin; lint against container cp
container cp needs its source on the node running the docker CLI, so
role_path sources break on swarm workers. The joomla plugin dir now
ships as a controller-side tarball through container exec -i. A new
repository lint forbids docker/container cp in tracked .yml and .sh
files; the six legitimate same-node uses carry justified nochecks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2a76b19a02 feat(env): pass the root-CA host path to the trust scripts via INFINITO_CA_CERT_HOST
New env handler exports the ca_trust_paths SPOT value (SOFTWARE_NAME read from group_vars) into .env; trust/linux.sh and trust/wsl2.sh consume the variable instead of carrying the path literal and fail loud when it is unset. Stale .env files need a regeneration (make dotenv) before the trust scripts run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9697c9e2f7 fix(ci-swarm): collect playwright reports via docker exec not docker cp
/var/lib/infinito on the swarm manager is an NFS mount inside the container's own mount namespace. docker cp reads the daemon-side rootfs and sees the empty mountpoint, so it copied zero files and the uploaded Playwright artifact was always empty. Stream the reports out through docker exec (in-namespace), the same access pattern the swarm diagnostics already use.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d84f30ed5 fix(web-app-erpnext): satisfy make quality after the swarm fix
Follow-up to the swarm CID/redis fix that make quality flagged:
- Extract the site-bootstrap sequence into 02b_create_site.yml (renamed from
  02b_restart_frappe.yml; now new-site + restart + re-resolve + wait) and include
  it with when: rc \!= 0, so the oversized when-block disappears and the sub-file
  is not parsed at all when the site already exists.
- The backend entrypoint now execs the image's real /usr/local/bin/entrypoint.sh
  after the config-ready gate, so the sites/assets symlink that entrypoint creates
  is preserved instead of relying on a sibling replica to recreate it.
- Drop the now-dead ERPNEXT_BACKEND_CONTAINER_ADDRESS var, shorten two over-long
  OIDC restart task names, and remove stale narration comments.

Also suppress the INFINITO_MAIN_REMOTE/BRANCH env lints in the git pre-push helper:
they are maintainer tooling knobs, not deployment keys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 02:13:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e070bb2d8d ci: fail when upstream main is not merged into the branch
Adds scripts/git/assert_main_merged.sh, which fetches the upstream main
tip and fails unless it is fully merged into HEAD. Wired in two places:

- pre-push: a new make-test-main-merged pre-commit hook (pre-push stage)
  runs 'make test-main-merged' before every push.
- pull request: a new assert-main-merged job checks out the real PR head
  (not the merge ref) and asserts the base branch is merged into it, so a
  PR whose branch lags main goes red.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:47:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e0559a10d8 feat(ci): configurable lifecycle envelope + mode selection
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.

Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:11:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ec8f41772 chore(make): add quality + autoformat-restage targets
quality runs autoformat then the full test suite in one shot as a
pre-commit gate.

autoformat-restage runs autoformat and re-stages files it rewrote that
were already staged, but only when the working tree held no unstaged
tracked changes beforehand -- so a partially-staged hunk is never swept
into the index. Logic lives in scripts/git/autoformat_restage.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:11:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bd65cb3823 fix(web-app-moodle): deliver mail through Mailu on every swarm host
moodle mail failed two ways in a swarm cluster. (1) moodle's build_context
pulled the FULL sys-svc-mail-msmtp role early, which rendered /root/.msmtprc on
the manager before Mailu had created the no-reply token and tripped the role's
run-once flag, so the manager kept an empty-password msmtprc and the health
check failed 'authentication method PLAIN needs a password'. Pull only
tasks_from: ca_bundle so the manager re-renders on the normal post-token pass.
(2) msmtprc forced host=127.0.0.1 for any in-container host, but the NFS server
sits outside the swarm mesh and has no local Mailu, so it hit 'connect to
127.0.0.1:465 refused'. Narrow the 127.0.0.1 branch to compose or a real
svc-swarm-node AND to when Mailu is actually deployed (email.external); else use
email.host, and map the NFS node's test DNS to the manager IP. Also import the
shared self-signed Root CA for the relay's TLS trust.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 339af1ce20 fix(swarm-test): do not fail pre-clean on a raced act container removal
down_act_outer.sh is a best-effort pre-clean (set +e), but its last statement (docker rm -f) can race act's own --rm finalize and return non-zero, so the script exited non-zero and aborted swarm-zombie before the deploy. Force exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2df41fe33d fix(ci/resolve): sort deploy matrix apps ascending after complexity filter
The complexity report orders roles coverage-first and caps them with --max-jobs; that order must drive selection. Once selected, re-sort the surviving roles alphabetically so the swarm and compose deploy matrices list jobs in a stable ascending order. The storage filter and variant_bundles both preserve input order, so the final matrix stays sorted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 eda8a8bc94 fix(autoformat): accept ansible-lint RC 0/2/8, drop narration comments
ansible-lint --fix exits 8 (RC.FIXED_VIOLATIONS) when it writes an auto-fix with zero failures remaining -- a normal outcome, same class as 0 (clean) and 2 (violations remain that --fix could not auto-resolve, same as the markdownlint/eslint rc<=1 pattern in this file). Previously only 0/2 were accepted, so a successful auto-fix could FAIL the whole autoformat run.

Also delete narration comments across the file that only restated the adjacent code; keep the two comments that flag a real trip-wire (status_dir crash-avoidance, write_status param doc).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 66a2db59c3 fix(test/swarm): dump full container logs on converge failure
The converge-failure diagnostics truncated service logs to --tail 50 (and head -100), but a crashing service prints its exception at the TOP of the trace, so the root cause was cut off and non-converging services could not be debugged from CI. Remove the caps in the converge-failure rescue (wait_converged.yml) and the diagnostics collector (12_collect_diagnostics.sh) so the full log, including the exception, is captured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d1528b6aab fix(test/swarm): reap NFS-wedged DinD nodes without sudo in swarm-clean
docker rm -f on a node stuck on a dead NFS mount races (did not receive
an exit event) and leaves the container stopped but not removed. Add a
SIGKILL + plain docker rm retry loop before the host-docker-restart
fallback, so make swarm-clean self-recovers without sudo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 001d880a72 fix(test/swarm): pin roundtrip swarm to one variant per app
A multi-variant app expanded into N swarm matrix entries, and
swarm-zombie ran them all on one host under identical SWARM_NAME
cluster names, so every deploy after the first aborted with
'network ... already exists'. Pin the act matrix to variant 0
(default, overridable via variant=) so a multi-variant app validates
a single cluster, in parity with the compose step's variant=0.
workflow.sh now splits ACT_MATRIX on ';' into multiple --matrix
filters so apps and variant can be pinned together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 81a0c022fc feat(ci): filter-driven app discovery with a job budget
Drive the test-deploy matrix discovery through the complexity report instead of the type CLI: apps.sh selects the mode's tested roles via the 'compose'/'swarm' column (which already bakes in invokable + tested-lifecycle + per-mode skip), orders them coverage-first, and hard-caps the cumulative per-mode runner budget.

- complexity gains --deploy-mode {compose,swarm} (makes 'bundles'/'jobs' the runner count for that mode: compose packs variants into size/storage bundles, swarm runs one per deployable variant) and --max-jobs N (keep only the leading rows whose cumulative jobs stay below N).
- apps.sh emits whole role names; variant_bundles/output_apps.sh expand them into the matrix unchanged. Manual INFINITO_WHITELIST becomes a 'name %% {…}' filter clause applied before the cut, so a manual run only counts the apps it actually invokes.
- INFINITO_DEPLOY_MODE is now required; INFINITO_MAX_JOBS, INFINITO_DISCOVERY_SORT and INFINITO_REQUIRED_STORAGE move to default.env (the SPOT) + the passthrough handler, so no shell-side defaults remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:14:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e5fd4b11d0 fix(test/swarm): NFS state-path, image pull retry, multi-service converge gate
Swarm deploy-test harness fixes (run 28439744622); swarm-test-harness only,
compose path untouched:

- NFS_STATE_PATH (<export>/infinito-state/<vol>): per-volume dirs are nested
  under the fsid=0 state subdir (STORAGE_NFS_STATE_SUBDIR), but the seed,
  assert and diagnostics used the bare export base -> "Directory nonexistent".
  00_topology derives it, the workflow exports it, 09/11/12 consume it.
- pull-retry: pre-pull jrei/systemd-ubuntu with bounded retries before
  docker run, so a Docker Hub registry blip no longer fails the job (exit 125).
- jitsi: derive the converge/drain SERVICE_NAME from proxy.service_key so
  multi-service stacks (jitsi_jitsi-web/-jvb/...) resolve; single-service apps
  fall back to the entity name (unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:51:50 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f02da33128 fix(ci): clear compose deploy-gate deprecation warnings + tighten comment markers
The test-deploy-compose "Fail on deprecation warnings" gate failed ~10 apps
while the deploys themselves succeeded. Sources cleared:

- svc-db-postgres: community.postgresql `db` -> `login_db` (the `db` param
  alias is deprecated, removed in collection 5.0.0)
- assert_no_deprecation_warnings.sh: ignore ansible-core's legacy filter/test
  PluginLoader-aliases deprecation (needs a collection migration with
  meta/runtime.yml, not fixable in role YAML)
- joomla/wordpress: drop the `{{ }}` delimiters from task conditionals
  (when/until/changed_when), per ansible-core 2.23

Also drop `upstream`/`compat`/`security`/`bug` from the comment-policy markers
and delete those comments from the joomla install tasks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:48:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 87c78275b8 feat(meta/complexity): multi-key sort, filter DSL, CI columns
Extend the 'meta roles applications complexity' report:

- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.

Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:33:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1fb96584b refactor(tests): rename swarm make targets to swarm-* and reorganize env steps
Rename the scripts/tests/environment deploy_* steps (07/08/09) to compose_*; add 12_swarm.sh (svc-db-mariadb via the swarm-* targets) and 13_roundtrip.sh (svc-db-postgres via make roundtrip), with MARIADB_APP/POSTGRES_APP in common.sh. Rename every make act-swarm-* target to make swarm-* (drop the act- prefix) across the Makefile (moved to the alphabetical slot), the deploy/act, deploy/swarm and e2e helper scripts, and the agent iteration docs. Strip mid-code narration comments from the touched scripts per the comment policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 80b8d5e9ef fix(ci/compose): copy generated inventory to host before teardown
The 'Zip generated inventory' step reads $INFINITO_INVENTORY_DIR on the
runner host, but the compose deploy writes inventory inside the DiD
container and the cleanup trap removed the container before the always()
zip step ran, so it failed with 'No inventory found under
/home/runner/inventories/github*'. Copy the inventory dir from the
container to the host parent in cleanup() before 'down'/rm; container and
host paths match, so the zip step then finds it. Non-fatal (|| true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d31adc3ab fix(swarm-test): purge inter-round NFS volumes by type instead of a missing nfs key
_context.sh derived NFS_VOLUMES by filtering meta/volumes.yml entries on a per-volume
nfs: key that no role declares, so the inter-round purge cleared no volume data and
round-0 state leaked into round-1. Filter on type==volume instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ca2ed56c3 fix(swarm-test): align roundtrip swarm gate to the live workflow step name
The gate grepped a stale step-name marker ("across variant rounds") that never appears
in the act log, so every swarm leg false-failed even on a passing deploy. Match the
actual workflow step name instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
Alejandro Roman 8f64fbb69e Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-30 10:20:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 80e03b4c20 fix(ci): resolve subset roles with venv python
The 🧩 Subset path in pr_affected_roles.sh exec'd a bare `python`, the setup-python interpreter without PyYAML, so subset_roles.py failed with `ModuleNotFoundError: No module named 'yaml'`. Source scripts/meta/env/load.sh so ${PYTHON} resolves to the venv (where PyYAML is installed), matching the diff resolver path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 01:42:47 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6bbe3f61e3 fix(ci): collect inventory from host inventory dir, not container/tmp
The compose and swarm inventory collectors hard-failed at the 'Zip generated
inventory' step because they looked in the wrong place. Inventory is written on
the runner host at $INFINITO_INVENTORY_DIR (resolve.sh yields
$HOME/inventories/github in CI), but:

- zip_compose_for_upload.sh ran 'docker cp' from infinito_nexus_* containers,
  which dedicated.sh's cleanup trap removes before the always() step runs, so it
  never found a container.
- zip_swarm_for_upload.sh globbed /tmp/inv*, which the matrix orchestrator never
  populates; it defaults its base to $INFINITO_INVENTORY_DIR.

Both now source scripts/meta/env/load.sh to resolve INFINITO_INVENTORY_DIR and
zip that dir plus its per-round '-<n>' variants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b6e79878fd fix(ci): make 🧩 Subset resolver comply with repo lint rules
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.

- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:49 +02:00
kevinveenbirkenbach 19b344d1dc Merge branch 'main' into feature/swarm-nfs 2026-06-30 00:59:49 +02:00