Commit Graph
840 Commits
Author SHA1 Message Date
kevinveenbirkenbach f2e9ce04d4 fix(git-hooks): run the merge-signed gate on prepare-commit-msg
git writes MERGE_HEAD only after the pre-merge-commit hook returns, so the
gate never found it and aborted every non-fast-forward merge with "No merge
in progress"; git then wrote the merge state via abort_commit(), leaving a
half-finished merge that git status reported as "still merging". The gate
had therefore never verified a single signature.

git also skips pre-merge-commit entirely once a merge conflicts, so the
conflict-resolution commit was unguarded as well. prepare-commit-msg fires
with MERGE_HEAD present on both paths and no-ops on ordinary commits.

Verified in a scratch repo with a real pre-commit install: ordinary commit
passes silently, clean automerge and conflict-resolution commit both block
an unsigned incoming commit, and the extra stash/restore cycle round-trips
unstaged changes byte-identically.

pre_commit install never removes hook types dropped from the config, so the
install script now also uninstalls the stale pre-merge-commit hook.
2026-07-28 17:01:24 +02:00
kevinveenbirkenbachandClaude Opus 5 8aba198aec feat(worktree): isolated per-branch checkouts sharing one cache stack
Parallel checkouts collided on the compose subnet, host ports and container
names, so a quality run in a worktree died at compose-up. Slot each checkout
via custom.env and layer two overrides on top: the shared cache network in
place of a second cache stack, and the shared .git directory without which
git aborts inside the container.

Both overlays gate on the resource they need, not on the slot number, so a
stray INFINITO_INSTANCE cannot make compose demand a value nobody set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:15:59 +02:00
kevinveenbirkenbachandClaude Opus 5 ebd7d2e72b refactor(meta): centralize category/distro SPOTs
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.

meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.

Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.

tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:15:56 +02:00
kevinveenbirkenbachandClaude Opus 5 2d283ed39a feat(ci): resolve image refs from services.yml SPOT
moby/buildkit was hardcoded in four places across two workflows and catthehacker/ubuntu in two act scripts, none of them reachable by the image mirror: utils/docker/image/discovery.py only scans roles/**/meta/services.yml. So the one image the CI build cannot proceed without was pulled straight from Docker Hub, unmirrored, which is what broke the fedora build in run 30160077828.

Declare both as mirror-only services on svc-runner, following the test-e2e-playwright precedent, so the existing mirror pipeline picks them up unchanged. New scripts/meta/resolve/image/service_ref.sh reads image+version back out of the declaration and is the single source for consumers outside Ansible; its mode argument is mandatory (upstream never touches the network, mirror prefers the GHCR tag and falls back to upstream). The fallback is why the mirror job can keep running in parallel with the build instead of gating it.

ACT_PLATFORM_IMAGE moves to an env handler so act consumers read the declared value via BASH_ENV, and workflow.sh now demands it instead of defaulting to a literal. Operator overrides still win through eb.setdefault.

Verified: service_ref.sh in all four paths (upstream, mirror hit, missing mode, invalid mode); the mirror path matches GHCRProvider.image_base() character for character and resolves against the live registry anonymously; ACT_PLATFORM_IMAGE resolves through the real BASH_ENV mechanism; make quality green on all four targets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:53:33 +02:00
kevinveenbirkenbachandClaude Opus 5 21dd54fd4b fix(lint): refresh apt lists before puppeteer provisions chrome
The container image ships with /var/lib/apt/lists emptied, so puppeteer's
--install-deps resolved no package name and chrome-headless-shell then died on
a missing libnspr4. All 262 markdown files with a mermaid block failed to
render, and the wrapper discarded the reason into /dev/null.

Refresh the lists when apt-get is present, and surface the provisioning output
instead of swallowing it, so a failure on a distribution puppeteer cannot serve
names its own cause.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:25:49 +02:00
kevinveenbirkenbachandClaude Opus 5 0bde90e47d fix(swarm-test): derive the DR drill backup providers from the closure
The drill configured svc-bkp-remote-2-local with a fixed manager plus NFS provider list for every app, while the manager only runs a backup repository when the app's include closure pulls svc-bkp-volume-2-local or svc-bkp-secrets-2-local in. For an app whose container_backup consumer is statically off, such as svc-dns-unbound, user-backup never ran on the manager, so the pull was rejected with 'Permission denied (publickey,password)' and the fail-if-any wrapper took the whole unit down at drill step 4/9. Four roles reach that step today: svc-dns-unbound, svc-db-redis, svc-registry-cache and svc-registry-docker.

backup_repos.py expresses the placement once and matrix derives the provider list from it per round. derive_includes accepts the round's variant map because INFINITO_APP_VARIANTS only reaches the subprocess callers. The extras file keeps the device paths the drill reads back and loses the provider list nothing read from it, and the drill banners stop claiming backup legs that were never installed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:29:47 +02:00
kevinveenbirkenbachandClaude Fable 5 caae36a716 fix(setup): prune stale generated group files
tasks/groups/*-roles.yml are gitignored, so branch switches leave
orphans behind; a stale svc-virt-roles.yml injected a phantom group
into the stage order and referenced a role absent on this branch.
Remove any group file not in the current invokable set before
regenerating, and drop narration comments flagged by the comment lint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:46:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4aede4deb9 ci(build): pre-pull syntax frontend with retry
Transient Docker Hub timeouts resolving docker/dockerfile:1 aborted two deploy jobs at '#2 resolve image config'. Pre-pull the frontend from the Dockerfile's syntax line with 3 bounded retries; fail soft so the build still resolves live if Docker Hub stays down. Real build errors remain unretried.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 12:44:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51dbc46876 ci(runner): relocate Docker data-root onto /mnt when roomier
Swarm's nested DinD-node daemons plus the backup DR drill (LUKS loop image,
per-node registry pulls) fill the ~50 GB root fs and trip the <6 GB abort
guard in utils/tests/swarm/matrix.py, while /mnt is a separate ~74 GB
partition sitting idle behind only a swapfile.

Add a runner step that moves the host Docker data-root to /mnt/docker when
/mnt is a separate, roomier partition (detection mirrors enlarge_swap.sh),
before any Docker usage. Best-effort: any failure rolls Docker back onto /
and the step still exits 0, so it can never turn a green deploy red. Wired
into the compose, swarm, and host deploy workflows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 20:05:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e9b0d5f76e fix(swarm-drill): give the reschedule convergence poll 240s for slow apps
The failover-drill assert polled service convergence for 60 iterations (120s).
OpenProject web boots in ~140s, so after 06_drain_worker rescheduled it the
replicas turned healthy ~20s after the poll gave up, and 07_assert_state.sh
reported 'Replicas not fully converged: 1/3' and exited 1 even though the stack
did converge moments later. The loop breaks early once converged, so raising the
ceiling to 120 iterations (240s) costs fast apps nothing and only widens the
window for heavy apps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 04:06:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4181d2194d fix(env): reload .env when it changed since the last source
load.sh short-circuits on INFINITO_ENV_LOADED, so a shell that already
sourced the env kept a stale copy after 'make dotenv' regenerated .env
(make runs in a subshell and cannot reset the parent's guard), and a
freshly added key was missing until a new shell. Gate the short-circuit
on the .env mtime: re-source when it changed, still a fast no-op when it
did not. A stat failure falls back to the previous idempotent behaviour.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6b71d3e8bb feat(ci): deploy branch-new roles first on the push priority line
A push resolves the application roles that exist under roles/ locally but
not at origin/main and passes them as the orchestrator priority line, so
brand-new roles deploy and gate ahead of the regular matrix. The resolver
is stdlib-only (git ls-tree plus an application_id line scan) so it runs
on a bare runner without installing the resolver stack, and yields an
empty set when the baseline ref cannot be resolved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6b1dd1c1d3 refactor(guide): reuse the rescue-diagnostics collector and extract the compose replay
host_deploy.sh drops its bespoke dump: the EXIT trap now runs
utils/diagnostics/container.py on failure before removing the boot
container, because the workflow's failure() collect step only fires
after this script has already torn the container down. That yields the
full snapshot (logs, inspect, in-container systemctl and journal, DiD
recursion) instead of three ad-hoc files, and successful runs skip the
capture entirely.

The compose-mode README replay moves out of test-guide.yml into
scripts/github/guide/compose_deploy.sh unchanged; the workflow step
becomes a one-liner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d71c041e7 fix(guide): capture host-mode boot diagnostics before teardown
The host-deploy EXIT trap now dumps failed units, the tail of the
journal, and the container log from the pkgmgr boot container into the
rescue-diagnostics base before removing the container, so a failed guide
run leaves a readable artifact instead of tearing down silently. The
base directory is hard-required from the workflow environment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ceb6e65e49 refactor(ci): single-source the rescue-diagnostics base dir from default.env
INFINITO_RESCUE_DIAGNOSTICS_BASE joins default.env as the only literal
of the runner-side collection root and is wired through the passthrough
handler into the generated .env. The four deploy workflows load it with
one grep bootstrap step after checkout instead of hardcoding the path in
their env blocks; upload paths and per-app DIR derivations reference the
variable. dedicated.sh sources it from .env alongside the existing
diagnostics keys and hard-requires it, no script-level default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07d31d9001 fix(ci): retry-harden every egress path (curl filter, shell downloads, package managers)
Transient CDN/mirror TLS resets are a recurring hard-fail class (docker.com, PyPI). Harden all three egress paths at their SPOT: the curl filter now emits --retry-all-errors (retries=0 opts a call out), so every ansible curl retries; a new lint requires --retry-all-errors on file-downloading curl and --tries on wget in verbatim files/*.sh (which cannot use the filter), fixing the moodle/alias downloads it flagged; and the update role configures native retries for every package manager - apt Acquire::Retries, dnf retries=, pacman XferCommand over retrying curl - covering all five CI distros (ubuntu shares apt, centos shares dnf).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 22fde6d285 fix(swarm-test): pin cluster node IPs and hostnames deterministically
Every DinD node is attached to a second network by the bootstrap cache connect, so the embedded DNS returns two A records per node name in undefined order; workers that resolved the manager onto the cache network could not reach the registry bound to the lab IP and rejected locally built images with 'No such image', pushing all replicas onto the manager. Pin a fixed lab ipv4_address per node and every node name via extra_hosts (files-first resolution wins over DNS); the subnet and IPs live in default.env as the INFINITO_SWARM_* SPOT consumed by compose interpolation and the topology exports. Proven by A/B: the identical friendica variant-2 swarm job went from worker rejects to fully green including the drain test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:50:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fb78b17901 fix(guide): resolve *.localhost inside the host-mode boot container
The hlth-webserver health service HEAD-requests the deploy's *.infinito.localhost vhosts, but the guide boot container has no resolver mapping for them (no CoreDNS sidecar, no dev-flow dns setup), so the service failed every start and burned 25x60s retries. Install libnss-myhostname on the apt branch and ensure the myhostname NSS module is listed in nsswitch.conf on every distro; it resolves any *.localhost name to loopback where the openresty vhosts answer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 23:22:48 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ff463b7b6a fix(ci): pass GHA repo owner into the DNS DinD env generation
Removing the baked .env forces cli.meta.env to regenerate inside the
container; with GITHUB_ACTIONS=true its on-GHA guard aborts (exit 2) unless
GITHUB_REPOSITORY_OWNER / GITHUB_REPOSITORY is set, so .env is never written
and compose-up dies with KeyError: INFINITO_CONTAINER. Forward the owner and
repository (already resolved on the runner) into each per-distro container.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 12:05:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 43d23ef696 refactor(agents): reorganize operator workflows into i8- skills
Move the operator cheatsheet's copy-paste prompts into per-workflow i8- skills
that each route to their authoritative procedure doc, and reduce the cheatsheet
to a table index plus a usage paragraph, enforced by a new lint. Add skills for
develop, test, quality, refactor, the debug and requirement flows, and
commit / pull-request / push-trigger-pull; rename the iteration skills to
i8-iterate-* and rename the index to i8-help. Wire the active-listening,
dialectic, and robot skills into the flows for clarification, uncertain root
causes, and autonomous execution. Drop the Infinito-specific agent aliases
(alias.md, INFINITO_ALIAS_MD, the make alias agent section, and the two alias
tests); terminal aliases stay.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 04f424bcb3 fix(ci): regenerate .env per distro inside the DNS DinD
The prebuilt CI image ships a .env generated at build time with the debian
default, and load.sh only regenerates when it is missing, so every distro's
DinD ran with debian's INFINITO_CONTAINER and config regardless of the image.
Remove the baked .env before make compose-up so it is regenerated from the
container's INFINITO_DISTRO, restoring the per-distro behaviour the old
matrix flow had on a fresh checkout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 34669658ef fix(guide): install systemd in the initless pkgmgr image before booting it
The host-mode guide booted the pkgmgr distro image with --entrypoint
/sbin/init, but that image ships no init at all (it is a package-manager CLI
tool), so runc rejected the container with exit 127. Install systemd in a
throwaway container, commit an image that carries /sbin/init, and boot that
instead. Also read the post-loop readiness state into a variable so a
degraded systemd is not masked by pipefail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36e4d67df2 fix(dns-test): accept a degraded systemd instead of failing under pipefail
The readiness check piped systemctl is-system-running into grep. Because
is-system-running exits non-zero on a degraded system (only running returns
0) and the script runs under set -o pipefail, the pipeline propagated that
non-zero even when grep matched, so the loop never broke and centos, which
settles at degraded with one failed unit, always exited 1 regardless of the
timeout. Read the state into a variable and match it, so a degraded system
is accepted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 138d40ac92 fix(ci): use fuse-overlayfs for the DNS DinD inner daemon
The per-distro DNS Docker-in-Docker container runs on an overlay rootfs, so
the inner daemon's default overlayfs snapshotter fails to mount overlay on
overlay ("invalid argument") and docker compose up never brings coredns and
infinito up. Install fuse-overlayfs and write /etc/docker/daemon.json with
storage-driver fuse-overlayfs plus containerd-snapshotter disabled before
starting docker, mirroring the swarm node daemon.json that already runs this
way in CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:59:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6768591afc refactor(dns-test): drop the redundant INFINITO_PULL_POLICY shell default
parallel.sh already passes INFINITO_PULL_POLICY=always into the DinD
container, so the shell-side "${INFINITO_PULL_POLICY:-always}" default is
redundant and a second source of truth that contradicts default.env's
INFINITO_PULL_POLICY=never. Read the value from the environment; default.env
stays the single default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc1b35e9a7 fix(guide): unmask systemd-logind before enabling root linger
The host-deploy base image ships systemd-logind masked, so loginctl
enable-linger cannot activate org.freedesktop.login1 over D-Bus and the
guide fails after the image pull with "Could not activate remote peer".
Unmask and start logind before enabling linger; it is a no-op where logind
is already active.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fb731041f4 refactor(ci): extract fork-prereq gate to a script; drop gate jobs
Move the inline fork-prerequisites result assertion out of the
fork-prereqs-ready job into scripts/github/gate/fork_prereqs.sh (result
passed via the RESULT env var; the job now checks out the repo).

The staged diff also drops the code-quality-gate and test-install-gate
aggregator jobs and wires their members (lint, test, security-codeql,
test-install-make, test-install-pkgmgr) directly into the done job's needs,
which still hard-fails on any upstream failure or cancellation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1bbe25f0d0 feat(ci): run DNS tests one job with per-distro Docker-in-Docker
Collapse the per-distro DNS test matrix into a single job that runs each
triggered distro in its own systemd Docker-in-Docker container, built from
that distro's prebuilt CI image, all in parallel. Each container installs
and starts a local docker daemon, brings the dev stack up and runs the
existing dns.sh suite unchanged. The local daemon makes the dev compose
bind-mounts resolve against the baked repo, and the per-container network
namespace keeps the static CoreDNS IP and subnet from colliding across
distros. GITHUB_ACTIONS=true and INFINITO_BUILD=0 pin the deploy to the
pull-and-run path (no cache stack, no local image build); dig is installed
unconditionally so arch keeps the host-query assertions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9deac861e6 refactor(ci): run install tests in one job per mode, distros from SPOT
Collapse the 5-distro matrix in the make and pkgmgr install-test workflows
into a single job that runs every distro in parallel inside extracted
scripts, cutting 5 runners down to 1 per workflow. Each distro installs
against its own copy of the checkout so the concurrent in-container builds
never race on the shared tree. The distro list is read from the
INFINITO_DISTROS SPOT in default.env instead of being hardcoded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 04414a0ec5 revert(swarm-nfs): drop backup-drill rework, openproject bypass, xwiki replicas:1
Back to the pre-swarm-nfs-fix baseline:
- 5c2c8f531 + 985ec7eab: the swarm writer-quiesce DR drill. Deep-dive
  attribution of CI run 29715516546 found it the dominant failure source
  (17 of 23 jobs): [9b/9] handed-off generation mismatch, [6/9] quiesce
  stack-rm network race, [4/9] remote-2-local pull stall, compose discourse
  restore duplicate key.
- d6f980574: openproject swarm ingress to web:8080. Carried a leaking
  proxy_internal_port lookup (no per-app default) that broke the seaweedfs
  dependency front-proxy ('Key web not found'); the bypass-the-Apache-proxy
  approach was never verified against downloads/X-Sendfile.
- 1dd153bad: xwiki replicas:1 + mem 3g. Did not resolve the xwiki superadmin
  failure (still red in run 29715516546), so it is unverified.

Re-approach each from a clean baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 12:11:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5c2c8f531f fix(backup): pass quality-high for the swarm writer-quiesce drill
Follow-up corrections to b449f8157 (swarm DR drill):
- export test.env in 06/07 via set -a so the child db restore/probe
  scripts inherit BKP_TEST_DATABASES_CSV and BKP_TEST_RESTORE_BIN
  instead of aborting at their :? guards before any restore runs
- move db_probe.sh/db_restore.sh into a db/ subfolder (probe.sh,
  restore.sh) and update all callers, so they stop tripping the
  prefix-grouped-files structure lint
- define TEST_E2E_SWARM_DRILL in group_vars/all and drop the now
  redundant default() from test.env.j2 (satisfies the undefined-var
  and spot-default lints together)
- fix a mermaid parse error in the drill README: a ; in a
  sequenceDiagram note was read as a statement separator

make quality-high green (test + lint phases, 15 targets, 0 failures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbach 985ec7eab9 fix(backup): quiesce swarm writers for restore
Defer Swarm SQL replay to the disposable DR drill so shared writers such as Discourse are stopped before the destructive restore. Verify the selected generation with before/after database probes while preserving the Compose restore path.
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 972596a758 fix(guide): document the administrator SSH key step and seed it in CI via --vars
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9563948782 feat(make): add swarm-app-exec, group act exec scripts under exec/
swarm-exec reaches a DinD node; reaching a swarm service container (moodle,
openldap) needed a nested docker exec. Add swarm-app-exec, which resolves one
running replica on the node and runs a base64-encoded command inside it, so
live in-container inspection stays on a sanctioned make helper. Adding a
second exec_* script tripped the prefix-group lint, so exec_node.sh moves to
exec/node.sh and the new one lands at exec/service.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 327fb7a2d2 fix(ci): stop materialising INFINITO_TEST_TYPE into the shared .env
INFINITO_TEST_TYPE is a per-invocation label the Makefile sets differently
per parallel test target (external/integration/lint/unit), not a persistent
default. Registering it as a static passthrough key wrote a per-target-varying
value into the shared .env that make test regenerates concurrently, which
tears the file for the env-file schema lint. Drop it from default.env and the
passthrough handler and suppress the run.sh reference with the
makefile-supplied nocheck that lines 15 and 27 already use, hoisting the junit
path into a variable so the continuation line can carry it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 03:49:24 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ee8d9cde9 fix(ci): extract the guide host-deploy step into a script
The inline run: block for the host-mode deploy exceeded the 11-line
inline-script-block cap. Move the body to scripts/github/guide/host_deploy.sh
and reduce the workflow step to a single bash invocation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 02:39:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 71481983fd fix(swarm): resolve PRIMARY_NFS_VOLUME without a broken-pipe pipe
07_assert_state.sh sources _context.sh under set -o pipefail, and the printf | head-n1 that picked the first NFS volume raised a broken-pipe write error the moment head closed the pipe after line one. Any role with two or more NFS volumes (erpnext ships erpnext_sites and erpnext_logs) crashed the reschedule assertion before it could check the surviving marker. Plain parameter expansion returns the same first line with no pipe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 02:39:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4a398653aa feat(ci): per-target and per-test summaries for the lint and test jobs
parallel.sh mirrors its per-target wall-clock table into
GITHUB_STEP_SUMMARY, so make lint and make test render a target/status/
duration overview on the run page. The pytest runner writes junit XML
per suite into build/test-reports and test.yml renders them via
utils/github/pytest_summary.py: headline counts with total time,
failures opened with durations, every test collapsed slowest-first.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 02:39:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a479760bf4 feat(ci): swarm run summaries with per-host role runtimes
The swarm deploy job now renders the same summary page as compose:
role runtimes, a swarm structure section (topology roles plus live node
list and service placement from the manager) and the Playwright table.
The matrix deploy tees its ansible output and prints the compose
matrix-deploy round/PASS markers, so the runtime parser segments swarm
runs per variant round too.

The runtime table unifies tasks and roles with a Type column (tasks
first), appends the recap total row and flags a mismatch between the
parsed sum and the recap total; rows are read from ROLES RECAP sections
only, which removes the double-counted TASKS RECAP task rows. Each role
row carries one column per host: executed, skipped or failed, with
ignored failures downgraded. The Playwright variant column drops the
redundant variant- prefix. Verified against the logs of runs
29656268771 (swarm) and 29620158441 (compose): every segment's parsed
sum matches the recap total within 0.05s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 02:39:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3ca9f2a4b9 chore(lint): add the lint-dockerfile alias and the script execute bit
The alias lint requires an i8 shorthand for every make target and the filesystem lint requires the execute bit on every script under scripts/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:36:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6538905bd7 refactor(lint): fold the Dockerfile hadolint check into make lint
The separate CI job is gone; hadolint now runs as lint-dockerfile inside the parallel make lint fan-out through the standard wrapper, with the binary installed by the lint toolchain installer like every other tool. The SARIF gate script had no consumer left and is removed. Scope stays the root Dockerfile at the warning threshold, matching the old job.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 20:36:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e7e32d14d4 fix(lint): silence SC2317 on the xargs-invoked mermaid render helper
The container shellcheck flags the render_one body as unreachable
because the function is only invoked through xargs + bash -c; extend
the existing SC2329 directive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:53:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 43918e209c perf(lint): render mermaid diagrams in parallel
The mermaid lint rendered one file at a time, booting a fresh
chrome-headless-shell per diagram. Fan the renders out with xargs -P
capped by INFINITO_WORKER_CPU (the shared parallel-make constant) and
print collected failures serially afterwards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:53:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 47628870bb fix(swarm-test): exclude nfs-false volumes from the primary NFS pick
The context derived PRIMARY_NFS_VOLUME from the first declared volume, but gitlab's first volume is repositories with nfs: false (gitaly cannot run on NFS), so the seed step wrote its pre-drain marker into a state directory that never exists on the export and killed a job whose deploy had already converged. Volumes declaring nfs: false are now skipped, making the drain check target a volume that actually lives on the export (gitlab_shared).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:26:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cb67adfffa feat(ci): select swarm deploys per variant and split discovery into three mode queries
The discovery SPOT (cli.meta.ci.query) now runs one query shape per mode:
compose and host stay whole-role with variants packed into bundles, swarm
ranks and budget-cuts on per-variant rows and emits role#variant tokens
through apps.sh into variant_bundles, so the cut can pick a subset of a
role's variants and the --matrix view is the selection priority.

Replace the --unique flag with visible dna/clone columns (heaviest role
per dna cluster stays ahead of the cut), add the integrated column and
sort, count host bundles like compose, and give the deploy plan an id
counter, per-variant weights and display-width-aware alignment matching
the matrix renderer. Add i8ma*/i8pa* inspection aliases per mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 16:26:30 +02:00
kevinveenbirkenbachandClaude Fable 5 393e0a84df feat(ci): share the discovery query and render a per-mode deploy plan
cli.meta.ci.query is the single source of truth for the app-discovery
query (filter, coverage-first sort, lifecycle envelope, compose
--unique, INFINITO_MAX_JOBS cap with 'auto'); apps.sh shrinks to one
call through it. cli.meta.ci.plan renders one section per deploy mode
in the query's own order (priority line starred, capped regular line,
then the candidates behind the cut), one row per variant, glossary
symbols in the header, with --cli for fixed-width terminal tables.
New glossary symbols: skip, priority, distros, unavailable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:26:30 +02:00
kevinveenbirkenbachandClaude Fable 5 9bce39009c feat(ci): derive deploy-matrix slots from the workflow chain
cli.meta.ci.slots statically and conservatively counts every job the
CI run spawns outside the deploy matrices (reusable workflows opened
recursively, static matrices multiplied out, dynamic ones estimated,
worst entry-*.yml overhead included) and splits the remaining
256-job run budget between compose/swarm/host by the _SHARES weights,
with a per-mode floor and _SLOT_OVERRIDES pins. INFINITO_MAX_JOBS
defaults to 'auto', resolved per mode in apps.sh; the host workflow
drops its hardcoded cap. --matrix renders the budget as a table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Fable 5 e32df7f24e feat(ci): single-node SPOT + priority deploy line
test-deploy-single-node.yml bundles the compose and host deploy tests
behind one reusable workflow; the orchestrator gains a priority input
that deploys focus roles first per mode and runs the remaining roles
only after every priority job succeeded, excluded via the new
INFINITO_BLACKLIST discovery filter in apps.sh. entry-manual exposes
the priority field and treats priority-only dispatches as roles-only.
Workflow dependency diagram added under .github/workflows/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 28ad9d8f5f fix(ci): real rescue artifacts, recover drill on non-stack jobs, stack-rm race tolerance
dedicated.sh streams rescue diagnostics via docker exec tar since docker cp cannot see the systemd container's mount namespace and silently extracted nothing. recover_drill.sh guards on docker availability: non-stack role deploys now drill the nfs and secrets legs instead of hard-failing on the impossible volume leg; stack jobs still drill all three. 04_stack_rm_wait tolerates an already-removed stack and re-issues the rm once services are gone.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 90b3845998 feat(readme): role README generator with Cosmos diagrams, Quick Setup, and lint
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).

The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.

test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.

Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.

Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:33 +02:00