Commit Graph
737 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Fable 5 c6ed8234b4 fix(swarm-tests): flavor-agnostic NFS assert mount and busybox-safe probe
The post-reschedule assert mounted the state export with vers=3, which
ganesha does not serve ('Protocol not supported'), so the marker check
false-negatived and killed the gitea roundtrip; reproduced and the
v4 mount shown to read the marker on the live cluster. Replace the
single mount with a cascade covering every production client_src form:
nfs4 vers=4 against the full path (ganesha), against :/ (kernel fsid=0
root), then the previous vers=3 full-path arm (kernel v3). The block
also returned the umount status instead of the grep result, letting a
missing marker pass silently; propagate the grep rc explicitly.

The in-container reachability probe's last arm used bash /dev/tcp, but
e.g. the ejabberd image ships neither bash nor curl; use busybox
nc -z instead and move the probe chain into probe_app_reachable() in
_context.sh alongside a NFS_CHECK_MOUNTPOINT SPOT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:34:29 +02:00
kevinveenbirkenbachandClaude Fable 5 0e644f8599 fix(joomla): stream plugin dir via exec stdin; lint against container cp
container cp needs its source on the node running the docker CLI, so
role_path sources break on swarm workers. The joomla plugin dir now
ships as a controller-side tarball through container exec -i. A new
repository lint forbids docker/container cp in tracked .yml and .sh
files; the six legitimate same-node uses carry justified nochecks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2a76b19a02 feat(env): pass the root-CA host path to the trust scripts via INFINITO_CA_CERT_HOST
New env handler exports the ca_trust_paths SPOT value (SOFTWARE_NAME read from group_vars) into .env; trust/linux.sh and trust/wsl2.sh consume the variable instead of carrying the path literal and fail loud when it is unset. Stale .env files need a regeneration (make dotenv) before the trust scripts run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9697c9e2f7 fix(ci-swarm): collect playwright reports via docker exec not docker cp
/var/lib/infinito on the swarm manager is an NFS mount inside the container's own mount namespace. docker cp reads the daemon-side rootfs and sees the empty mountpoint, so it copied zero files and the uploaded Playwright artifact was always empty. Stream the reports out through docker exec (in-namespace), the same access pattern the swarm diagnostics already use.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d84f30ed5 fix(web-app-erpnext): satisfy make quality after the swarm fix
Follow-up to the swarm CID/redis fix that make quality flagged:
- Extract the site-bootstrap sequence into 02b_create_site.yml (renamed from
  02b_restart_frappe.yml; now new-site + restart + re-resolve + wait) and include
  it with when: rc \!= 0, so the oversized when-block disappears and the sub-file
  is not parsed at all when the site already exists.
- The backend entrypoint now execs the image's real /usr/local/bin/entrypoint.sh
  after the config-ready gate, so the sites/assets symlink that entrypoint creates
  is preserved instead of relying on a sibling replica to recreate it.
- Drop the now-dead ERPNEXT_BACKEND_CONTAINER_ADDRESS var, shorten two over-long
  OIDC restart task names, and remove stale narration comments.

Also suppress the INFINITO_MAIN_REMOTE/BRANCH env lints in the git pre-push helper:
they are maintainer tooling knobs, not deployment keys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 02:13:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e070bb2d8d ci: fail when upstream main is not merged into the branch
Adds scripts/git/assert_main_merged.sh, which fetches the upstream main
tip and fails unless it is fully merged into HEAD. Wired in two places:

- pre-push: a new make-test-main-merged pre-commit hook (pre-push stage)
  runs 'make test-main-merged' before every push.
- pull request: a new assert-main-merged job checks out the real PR head
  (not the merge ref) and asserts the base branch is merged into it, so a
  PR whose branch lags main goes red.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:47:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e0559a10d8 feat(ci): configurable lifecycle envelope + mode selection
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.

Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:11:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ec8f41772 chore(make): add quality + autoformat-restage targets
quality runs autoformat then the full test suite in one shot as a
pre-commit gate.

autoformat-restage runs autoformat and re-stages files it rewrote that
were already staged, but only when the working tree held no unstaged
tracked changes beforehand -- so a partially-staged hunk is never swept
into the index. Logic lives in scripts/git/autoformat_restage.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 01:11:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bd65cb3823 fix(web-app-moodle): deliver mail through Mailu on every swarm host
moodle mail failed two ways in a swarm cluster. (1) moodle's build_context
pulled the FULL sys-svc-mail-msmtp role early, which rendered /root/.msmtprc on
the manager before Mailu had created the no-reply token and tripped the role's
run-once flag, so the manager kept an empty-password msmtprc and the health
check failed 'authentication method PLAIN needs a password'. Pull only
tasks_from: ca_bundle so the manager re-renders on the normal post-token pass.
(2) msmtprc forced host=127.0.0.1 for any in-container host, but the NFS server
sits outside the swarm mesh and has no local Mailu, so it hit 'connect to
127.0.0.1:465 refused'. Narrow the 127.0.0.1 branch to compose or a real
svc-swarm-node AND to when Mailu is actually deployed (email.external); else use
email.host, and map the NFS node's test DNS to the manager IP. Also import the
shared self-signed Root CA for the relay's TLS trust.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 339af1ce20 fix(swarm-test): do not fail pre-clean on a raced act container removal
down_act_outer.sh is a best-effort pre-clean (set +e), but its last statement (docker rm -f) can race act's own --rm finalize and return non-zero, so the script exited non-zero and aborted swarm-zombie before the deploy. Force exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2df41fe33d fix(ci/resolve): sort deploy matrix apps ascending after complexity filter
The complexity report orders roles coverage-first and caps them with --max-jobs; that order must drive selection. Once selected, re-sort the surviving roles alphabetically so the swarm and compose deploy matrices list jobs in a stable ascending order. The storage filter and variant_bundles both preserve input order, so the final matrix stays sorted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 eda8a8bc94 fix(autoformat): accept ansible-lint RC 0/2/8, drop narration comments
ansible-lint --fix exits 8 (RC.FIXED_VIOLATIONS) when it writes an auto-fix with zero failures remaining -- a normal outcome, same class as 0 (clean) and 2 (violations remain that --fix could not auto-resolve, same as the markdownlint/eslint rc<=1 pattern in this file). Previously only 0/2 were accepted, so a successful auto-fix could FAIL the whole autoformat run.

Also delete narration comments across the file that only restated the adjacent code; keep the two comments that flag a real trip-wire (status_dir crash-avoidance, write_status param doc).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 66a2db59c3 fix(test/swarm): dump full container logs on converge failure
The converge-failure diagnostics truncated service logs to --tail 50 (and head -100), but a crashing service prints its exception at the TOP of the trace, so the root cause was cut off and non-converging services could not be debugged from CI. Remove the caps in the converge-failure rescue (wait_converged.yml) and the diagnostics collector (12_collect_diagnostics.sh) so the full log, including the exception, is captured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d1528b6aab fix(test/swarm): reap NFS-wedged DinD nodes without sudo in swarm-clean
docker rm -f on a node stuck on a dead NFS mount races (did not receive
an exit event) and leaves the container stopped but not removed. Add a
SIGKILL + plain docker rm retry loop before the host-docker-restart
fallback, so make swarm-clean self-recovers without sudo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 001d880a72 fix(test/swarm): pin roundtrip swarm to one variant per app
A multi-variant app expanded into N swarm matrix entries, and
swarm-zombie ran them all on one host under identical SWARM_NAME
cluster names, so every deploy after the first aborted with
'network ... already exists'. Pin the act matrix to variant 0
(default, overridable via variant=) so a multi-variant app validates
a single cluster, in parity with the compose step's variant=0.
workflow.sh now splits ACT_MATRIX on ';' into multiple --matrix
filters so apps and variant can be pinned together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 81a0c022fc feat(ci): filter-driven app discovery with a job budget
Drive the test-deploy matrix discovery through the complexity report instead of the type CLI: apps.sh selects the mode's tested roles via the 'compose'/'swarm' column (which already bakes in invokable + tested-lifecycle + per-mode skip), orders them coverage-first, and hard-caps the cumulative per-mode runner budget.

- complexity gains --deploy-mode {compose,swarm} (makes 'bundles'/'jobs' the runner count for that mode: compose packs variants into size/storage bundles, swarm runs one per deployable variant) and --max-jobs N (keep only the leading rows whose cumulative jobs stay below N).
- apps.sh emits whole role names; variant_bundles/output_apps.sh expand them into the matrix unchanged. Manual INFINITO_WHITELIST becomes a 'name %% {…}' filter clause applied before the cut, so a manual run only counts the apps it actually invokes.
- INFINITO_DEPLOY_MODE is now required; INFINITO_MAX_JOBS, INFINITO_DISCOVERY_SORT and INFINITO_REQUIRED_STORAGE move to default.env (the SPOT) + the passthrough handler, so no shell-side defaults remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:14:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e5fd4b11d0 fix(test/swarm): NFS state-path, image pull retry, multi-service converge gate
Swarm deploy-test harness fixes (run 28439744622); swarm-test-harness only,
compose path untouched:

- NFS_STATE_PATH (<export>/infinito-state/<vol>): per-volume dirs are nested
  under the fsid=0 state subdir (STORAGE_NFS_STATE_SUBDIR), but the seed,
  assert and diagnostics used the bare export base -> "Directory nonexistent".
  00_topology derives it, the workflow exports it, 09/11/12 consume it.
- pull-retry: pre-pull jrei/systemd-ubuntu with bounded retries before
  docker run, so a Docker Hub registry blip no longer fails the job (exit 125).
- jitsi: derive the converge/drain SERVICE_NAME from proxy.service_key so
  multi-service stacks (jitsi_jitsi-web/-jvb/...) resolve; single-service apps
  fall back to the entity name (unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:51:50 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f02da33128 fix(ci): clear compose deploy-gate deprecation warnings + tighten comment markers
The test-deploy-compose "Fail on deprecation warnings" gate failed ~10 apps
while the deploys themselves succeeded. Sources cleared:

- svc-db-postgres: community.postgresql `db` -> `login_db` (the `db` param
  alias is deprecated, removed in collection 5.0.0)
- assert_no_deprecation_warnings.sh: ignore ansible-core's legacy filter/test
  PluginLoader-aliases deprecation (needs a collection migration with
  meta/runtime.yml, not fixable in role YAML)
- joomla/wordpress: drop the `{{ }}` delimiters from task conditionals
  (when/until/changed_when), per ansible-core 2.23

Also drop `upstream`/`compat`/`security`/`bug` from the comment-policy markers
and delete those comments from the joomla install tasks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:48:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 87c78275b8 feat(meta/complexity): multi-key sort, filter DSL, CI columns
Extend the 'meta roles applications complexity' report:

- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.

Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:33:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1fb96584b refactor(tests): rename swarm make targets to swarm-* and reorganize env steps
Rename the scripts/tests/environment deploy_* steps (07/08/09) to compose_*; add 12_swarm.sh (svc-db-mariadb via the swarm-* targets) and 13_roundtrip.sh (svc-db-postgres via make roundtrip), with MARIADB_APP/POSTGRES_APP in common.sh. Rename every make act-swarm-* target to make swarm-* (drop the act- prefix) across the Makefile (moved to the alphabetical slot), the deploy/act, deploy/swarm and e2e helper scripts, and the agent iteration docs. Strip mid-code narration comments from the touched scripts per the comment policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 80b8d5e9ef fix(ci/compose): copy generated inventory to host before teardown
The 'Zip generated inventory' step reads $INFINITO_INVENTORY_DIR on the
runner host, but the compose deploy writes inventory inside the DiD
container and the cleanup trap removed the container before the always()
zip step ran, so it failed with 'No inventory found under
/home/runner/inventories/github*'. Copy the inventory dir from the
container to the host parent in cleanup() before 'down'/rm; container and
host paths match, so the zip step then finds it. Non-fatal (|| true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d31adc3ab fix(swarm-test): purge inter-round NFS volumes by type instead of a missing nfs key
_context.sh derived NFS_VOLUMES by filtering meta/volumes.yml entries on a per-volume
nfs: key that no role declares, so the inter-round purge cleared no volume data and
round-0 state leaked into round-1. Filter on type==volume instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ca2ed56c3 fix(swarm-test): align roundtrip swarm gate to the live workflow step name
The gate grepped a stale step-name marker ("across variant rounds") that never appears
in the act log, so every swarm leg false-failed even on a passing deploy. Match the
actual workflow step name instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
Alejandro Roman 8f64fbb69e Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-30 10:20:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 80e03b4c20 fix(ci): resolve subset roles with venv python
The 🧩 Subset path in pr_affected_roles.sh exec'd a bare `python`, the setup-python interpreter without PyYAML, so subset_roles.py failed with `ModuleNotFoundError: No module named 'yaml'`. Source scripts/meta/env/load.sh so ${PYTHON} resolves to the venv (where PyYAML is installed), matching the diff resolver path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 01:42:47 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6bbe3f61e3 fix(ci): collect inventory from host inventory dir, not container/tmp
The compose and swarm inventory collectors hard-failed at the 'Zip generated
inventory' step because they looked in the wrong place. Inventory is written on
the runner host at $INFINITO_INVENTORY_DIR (resolve.sh yields
$HOME/inventories/github in CI), but:

- zip_compose_for_upload.sh ran 'docker cp' from infinito_nexus_* containers,
  which dedicated.sh's cleanup trap removes before the always() step runs, so it
  never found a container.
- zip_swarm_for_upload.sh globbed /tmp/inv*, which the matrix orchestrator never
  populates; it defaults its base to $INFINITO_INVENTORY_DIR.

Both now source scripts/meta/env/load.sh to resolve INFINITO_INVENTORY_DIR and
zip that dir plus its per-round '-<n>' variants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b6e79878fd fix(ci): make 🧩 Subset resolver comply with repo lint rules
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.

- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:49 +02:00
kevinveenbirkenbach 19b344d1dc Merge branch 'main' into feature/swarm-nfs 2026-06-30 00:59:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d1e718abf feat(ci): add 🧩 Subset label for explicit role selection on PRs
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).

- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:48 +02:00
Alejandro Roman d832e88cec Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-30 00:14:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0bcd63b658 fix(ci): make 🧩 Subset resolver comply with repo lint rules
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.

- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:08:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ad17c07de feat(ci): add 🧩 Subset label for explicit role selection on PRs
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).

- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:10:50 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e11bf6f31b refactor(lint): drop caveat/workaround comment markers
Remove caveat and workaround from the comment-lint marker vocabulary and delete
the comments that used them across the tree. Move test_comments_valid's own
git-safe.directory note from a marked comment into the function docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:54:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ed7ccbaaf2 fix(ci): collect swarm inventories + hard-fail when none to copy
zip_swarm_for_upload.sh only checked the base /tmp/inv, but the matrix
orchestrator writes per-round dirs /tmp/inv-<n> (and runs as root), so the
artifact was always empty. Collect every /tmp/inv* dir with sudo and de-root
the staged copy before zipping.

Both the swarm and compose collectors now hard-fail (exit 1) when they find no
inventory to copy, instead of echoing and exiting 0, so a silently-missing
artifact surfaces a real provisioning/path bug.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:06:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d95f8d8d22 fix(ansible): clear deploy deprecation warnings + gate against regressions
Remove the [DEPRECATION WARNING] entries the compose/swarm deploys emitted:

- apt_repository (removed in ansible-core 2.25): write docker.list via copy +
  an explicit apt update. deb822_repository is unusable here because
  apt_repo.sh's sanitize_sources deletes .sources every run.
- INJECT_FACTS_AS_VARS (default deprecated in 2.24): use ansible_facts['<name>']
  instead of the auto-injected top-level ansible_os_family / virtualization_* /
  default_ipv4 vars.
- community.mysql.* (removed in community.mysql 6.0.0): switch to ansible.mysql
  and pin the collection in requirements.galaxy.yml.

Add a compose deploy gate (assert_no_deprecation_warnings.sh + workflow step +
test) that fails on [DEPRECATION WARNING] in the deploy log, and a lint test
(test_no_deprecated_ansible_patterns) that forbids the three patterns above so
they cannot return.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 56f0364414 fix(swarm-test): purge named volumes between matrix rounds
The per-round swarm purge ran `docker volume prune -f`, which on Docker 23+
removes only anonymous volumes. A named DB volume such as mariadb_data survives
`docker stack rm`, so the next round booted the prior round's database, whose
root password no longer matched the freshly regenerated credentials, and mariadb
failed with "Access denied for root". Use `docker volume prune -a -f` so named
unused volumes are wiped too and each round re-initialises from clean state.

Swarm-test-harness only; never runs in compose.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:37:56 +02:00
Alejandro Roman 1c80c2413d Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-29 11:36:40 +02:00
kevinveenbirkenbach 3a7f7e6f00 Merge branch 'main' into feature/swarm-nfs
# Conflicts:
#	.github/workflows/images-build-ci.yml
#	.github/workflows/release-version.yml
#	roles/web-app-matomo/meta/services.yml
#	roles/web-app-pihole/meta/server.yml
2026-06-28 22:53:15 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e55001e849 feat(update): one PR per role and compute diffs from latest main
The roles updaters (Docker image versions, repository refs) opened a
single combined PR for all role changes, and the jobs checked out the
trigger commit (actions/checkout default), so updates were computed
against a stale tree and could re-propose already-merged changes.

- Check out 'ref: main' in all update jobs so the diff is always
  computed against the latest main tip at run time.
- Add open_role_prs.sh: split the working-tree changes per role and
  open one isolated branch/PR per changed role, reusing open_pr.sh for
  the dedup/push/PR logic (new UPDATE_DEDUPE_PREFIX hook scopes the
  duplicate check per role; default preserves prior single-PR behavior
  for the skills job).
- Wire the Docker and repository-refs jobs to open_role_prs.sh; skills
  stays a single PR.

Branch: update/<type>-<role>-<date>-<fp>; commit/title gain a (<role>)
suffix and a Scope line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:58:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0df75d83a6 fix(ci): retry buildx setup against transient Docker Hub timeouts
The 'Set up Docker Buildx' step boots BuildKit, which pulls
moby/buildkit from registry-1.docker.io. A transient Docker Hub
timeout there had no retry (only the build step did), so the whole
build job failed at setup.

Pre-pull the BuildKit image with a 7x/20s retry before the buildx
setup, and pin the action to that exact tag via driver-opts. Once the
image is local, buildx falls back to it if the bootstrap pull fails
('pulling failed, using local image'), so a flaky Docker Hub no longer
breaks setup. Applied to both images-build-ci.yml and
release-version.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:58:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 539a078475 fix(svc-storage-nfs-server): select ganesha in DinD CI, kernel default
The github swarm legs picked the kernel NFS flavor inside the privileged DinD nfs-server, where kernel nfsd is not network-namespaced and wedges exportfs. Move flavor selection out of the role meta default and into the dev inventory: meta/services.yml now defaults to kernel (production); the development inventory overrides it to ganesha for the containerized runtimes (dev/act/github) and keeps kernel for real hosts (RUNTIME=host).

Add meta/variants.yml (kernel + ganesha) and a TODO to test both flavors via files/test.sh, plus server-side kernel-nfsd diagnostics in 12_collect_diagnostics.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 06:22:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 82625c36dd fix(swarm-test): wipe the NFS export as root inside the privileged nfs-server
The host-side rm in 01_start_nfs.sh ran in the (often non-root) act runner and could not delete the prior run's root-owned NFS writes, so stale state survived every redeploy. A stale root-owned matomo config.ini.php that the uid-33 web user can no longer rewrite kept matomo unhealthy, so the role's post-health chown never ran and the swarm service never converged. Clear the export contents from inside the privileged container, the only context that can delete root-owned files, so every cluster start begins from an empty export.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 06:22:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 293f70d785 feat(scripts): make GHCR CI image cleanup report its actions
The cleanup step ran silently, so a no-op run was indistinguishable
from a broken one. Add a config banner, per-deletion lines, and a
per-package plus grand-total summary with a kept-reason breakdown, and
an explicit message when nothing matches. Add DRY_RUN support.

Deletion rule: delete a version only when it carries a ci-* tag, has no
semver (v*) or latest tag, and is older than MAX_AGE_DAYS. Released
images and untagged multi-arch child layers are never touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 02:47:32 +02:00
Alejandro Roman 17d6ac904f Merge remote-tracking branch 'upstream/main' into feature/self-host-runners
# Conflicts:
#	.github/workflows/test-deploy-universal.yml
#	.github/workflows/test-deploy-workstation.yml
2026-06-27 23:45:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5b1992005c fix(svc-storage-nfs-server): give the kernel NFS exports distinct fsids
Real GitHub Actions and production use the kernel NFS flavor (RUNTIME not in dev/act); only local act uses ganesha. The kernel exports.j2 set no fsid, so the export base and its nested state subdir (one device, a bind-mount under test) get colliding auto-derived NFSv4 fsids; the controller's mount of /srv/nfs/infinito-state then fails with the server returning 'No such file or directory'. Add fsid=1 (base) and fsid=2 (state subdir), mirroring the ganesha flavor's Export_Id 1/2.

Extend the swarm test 12_collect_diagnostics.sh to dump the nfs-server /etc/exports, exportfs -v, ganesha.conf, export tree, nfsd status, and the controller's NFS reachability, so an export failure is diagnosable from the job log instead of only the client-side ENOENT.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 23:11:39 +02:00
kevinveenbirkenbach 0798a21b2a Merge branch 'main' into feature/swarm-nfs 2026-06-27 21:52:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 214672a946 fix(test-deploy-swarm): actually run the swarm deploy job under act
After the main merge the discover job emits objects ([{apps,variant,variant_slug}]) for the variant matrix, but the swarm workflow still consumed them via the string-form 'matrix: apps:', so matrix.apps became an object, act produced no legs, the deploy job never ran, and act exited 0: every local swarm roundtrip false-passed.

Switch the swarm matrix to 'include:' (matching the compose workflow) so matrix.apps is the app id again, and guard roundtrip.sh to require the 'Matrix-deploy <app> across variant rounds' marker in the log or exit 1, so a skipped deploy can never pass as green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 21:41:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8a586ad373 feat(lint): validate generated distro changelogs + harden the mirror generator
Adds a lint-packages target/workflow that parses the generated packaging metadata with each ecosystem's native parser (dpkg-parsechangelog, rpmspec, bash -n for the PKGBUILD), failing on any parser stderr so warnings cannot slip through. Wired into the make lint fan-out and ci-orchestrator's lint gate; an install group provisions the parsers (best-effort, cross-distro).

Hardens package_mirror so the generated changelogs parse without warnings: every RPM body line gets a '- ' continuation prefix (a column-0 '*' was read as a new %changelog entry -> 'bad date'), and the archived-releases notice is folded into the oldest kept entry instead of trailing after the final entry (free text there made dpkg-parsechangelog warn). Regenerates packaging/debian/changelog and the Fedora spec accordingly.

Adds a test-lint check (test_package_changelog_completeness) asserting both package changelogs list every CHANGELOG.md + docs/changelog release and that their inline window matches CHANGELOG.md, plus unit regressions for the RPM continuation prefix and the Debian footer fold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 21:36:30 +02:00
kevinveenbirkenbach f1c3beee68 ci: add repository cleanup workflow 2026-06-27 21:36:30 +02:00