Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 b5e5554883 fix(complexity): gate the host column on the lifecycle envelope
compose and swarm columns go through _tested_apps (invokable +
lifecycle envelope + per-mode skip) but host only checked non-stack +
modes.host.enabled, so host CI discovery reacted to every lifecycle:
61 roles instead of the 24 inside the alpha/beta/rc/stable envelope.
Route host through _tested_apps("host") in both the role-level and
per-variant row builders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5a85fda3b6 chore(ci): rename deploy discover jobs to Host/Compose/Swarm
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ae038c3fa7 fix(prometheus): trim prometheus mem_limit to 1536m
The cadvisor raise pushed nextcloud variant 5 past the 64 GB
per-variant mem_limit budget (64.16 GB). Prometheus was observed at
167 MiB under full scrape load on a 60-container cluster, so 1536m
keeps ~9x headroom while freeing twice the overshoot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ae12567caf feat(diagnostics): widen host journal and dmesg capture windows
journalctl -n 1000 covered ~2 minutes on a busy node, so the kernel
and unit events of a failure 30+ minutes earlier were already gone
from the rescue artifact (run 29448092737 nc0: the 22:27 kill window
was unrecoverable). Capture a 10000-line tail plus a 6h warning-level
journal, the full dmesg ring, and drop the 80-line cap on the OOM
filter.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 de752dbca0 fix(prometheus): raise cadvisor mem_limit to 512m
At 256m cadvisor OOM-loops every 7-20 minutes on nodes running 60+
containers (run 29448092737 nc0: five kills at exactly the 262144kB
cap), burning 26% CPU on restart churn that feeds the memory pressure
it is supposed to observe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5a52ab2024 fix(ci): cap workflow timeouts at 345m and lint the headroom
GitHub hard-kills hosted-runner jobs at 360 minutes, so a 360m budget
leaves the post-failure diagnostics and artifact-upload steps zero
runtime and the failure logs are lost. Cap every deploy workflow at
345m and add a lint that fails any .github/workflows/*.yml
timeout-minutes above 345.

nextcloud swarm shards 2 and 5 in run 29448092737 died at the previous
240m step budget with zero task failures, 10-15 minutes short of a
green update pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 84fc309699 fix(lint): resolve make quality lint-test failures
Surfaced by make quality: alias comments on i8matiso/i8mativaso/i8s; self-path nocheck + drop two narration comments in load.sh (venv note kept in the header); shorten the host deploy job name to <=80 chars; move the legend rationale from an inline comment into the docstring. Also move utils/env/handlers/infinito/fork_{account,repository_url}.py into a fork/ subfolder (shared-prefix rule) and update the handler imports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5532e92eb3 feat(ci): show selected distro in deploy discover names
The compose/swarm/host deploy workflows' discover jobs now display the run's distro list (Navigate/Lure/Enumerate + "(distros)"). test-deploy-swarm gains a distros input (display-only; swarm still deploys the default distro) plumbed from ci-orchestrator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c2e06b9056 fix(ci): capture rescue diagnostics inside the DiD container before teardown
In compose/host mode the whole nested app stack lives inside INFINITO_CONTAINER, which dedicated.sh's EXIT-trap cleanup removes. The separate workflow-level rescue step ran container.py on the runner AFTER that teardown, so it saw an emptied Docker and captured only 12 host files (unlike swarm, whose DiD nodes outlive the deploy step and get recursed into).

Run the recursive container.py capture inside INFINITO_CONTAINER during cleanup, before the existing copy-out and the teardown, so it descends into the app containers. Drop the now-redundant, post-teardown 'Collect rescue diagnostics' step from the compose and host workflows (runner state stays covered by dump_state.sh; the upload step keeps uploading what dedicated.sh writes). container.py is now called once per mode: inside dedicated.sh for compose/host, at runner level for swarm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 672e93246b feat(deploy): add host deploy mode + matrix symbol view
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').

Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 be78aff9f9 fix(mattermost): run swarm setup/kcadm exec on the task's own node
The mmctl readiness/setup execs and the kcadm OIDC-config execs ran container exec against a manager-resolved container id. When Swarm scheduled mattermost (or keycloak) on a worker, that id does not exist on the manager, so the exec failed with 'No such container' and the deploy aborted (nextcloud-3 swarm: mattermost readiness wait exhausted its retries).

Adopt the established resolve_host_cid + delegate_to pattern (matomo, keycloak, opentalk): resolve the hosting node and the node-local container id, then delegate every container exec to that node. Applied to the mattermost mmctl execs, the openldap ldapsearch, and the 10 keycloak kcadm execs; the keycloak chdir is now compose-only. Node-agnostic, no manager pin, compose behaviour unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7a6268fa96 refactor(deploy): replace skip with services modes
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d0ae004eb0 feat(env): fork-identity vars + CI-redeploy aliases
INFINITO_FORK_REPOSITORY_NAME (default.env, static) plus two handlers: INFINITO_FORK_ACCOUNT defaults to the local OS username (getpass.getuser()), and INFINITO_FORK_REPOSITORY_URL composes https://github.com/<account>/<name>. Registered in ORDERED_HANDLERS with the account handler before the URL handler; documented in docs/contributing/environment/variables.md.

Two operator aliases build on it: i8cire (make quality + git-sign-push + retrigger the failed swarm CI jobs of a run) and i8puci (the same without the quality gate). Both take a run id or full URL: a bare id is expanded against INFINITO_FORK_REPOSITORY_URL, a URL is passed through, and a missing INFINITO_FORK_REPOSITORY_URL fails fast.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6fd8a499f3 fix(nextcloud): addon spec locators match the custom_apps webroot
drawio, fileslibreofficeedit and metadata are appstore/upstream apps, so Nextcloud serves their frontend bundles from /custom_apps/<app>/, not the read-only shipped /apps/<app>/. The specs asserted the bundle with a leading-slash locator script[src*="/apps/<app>/"], which cannot substring-match /custom_apps/<app>/ (the char before 'apps' is '_'), so the enabled-app bundle check deterministically returned count 0 across every variant.

Drop the leading slash (script[src*="apps/<app>/"]): it matches both /apps/ and /custom_apps/ webroots, so it is correct for shipped and appstore apps alike. These are the only 3 addon specs in the suite that assert a bundle asset; the served URLs (/custom_apps/drawio/js/main.js etc.) are captured in the CI trace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 31c00eb35a docs(variants): honest all-off swarm-drop rationale
The docstrings claimed an all-off variant 'adds no coverage'. It does:
it is the role's standalone path (every optional service off). Reword to
the real reason swarm skips it — its overlay/placement/VIP risk lives in
the all-enabled variant and compose still runs the standalone path, so the
extra swarm runner is cost without new coverage. No behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ab1c169a6 feat(diagnostics): generalize rescue capture + route nfs-ganesha to journald
container.py now captures, per container, systemctl status and journalctl, plus host-level systemctl list-units. The SPOT rescue diagnostic reaches the top-level systemd nfs-server container, so a nfs-ganesha startup failure is saved without a role-local dump.

nfs-ganesha now logs to syslog via an OPTIONS override (-L SYSLOG) so late runtime failures reach journald and are captured by the generalized diagnostic. LOGFILE= was a no-op because the packaged unit expands ${OPTIONS}, never $LOGFILE. The bespoke ganesha probe/dump block is removed, superseded by the generalized capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f78ec52385 fix(nextcloud): stop the config copy-up race and the root-owned config bind
The custom Dockerfile ran mkdir -p /var/www/html/config, which broke the
stock image's empty-/var/www/html invariant: Docker then had image content
at the shared nextcloud_data mount point and did a non-idempotent copy-up
os.Mkdir(config) per container. compose up --force-recreate starts
nextcloud and cron concurrently, so both raced that mkdir and the loser
crashed with 'failed to mkdir .../_data/config: file exists'. Reproduced
char-for-char locally (31/60 with the infinito bind, 7/80 without, so the
bind is only an aggravator); removing the image content gives 0/80.

Drop the Dockerfile mkdir so /var/www/html stays empty and the stock
entrypoint (reached via nextcloud-init-gate.sh -> /entrypoint.sh) creates
config/ as www-data with no copy-up. The mkdir only existed to pre-seed
config as www-data because binding infinito at /var/www/html/config/infinito
forced Docker to create config/ root:root; move that bind (and its
NEXTCLOUD_DOCKER_CONF_ADD_PATH loader path) out to /var/www/infinito so
config/ is owned by www-data and config.php stays writable. Keeps type:
bind, so the swarm config-object fix from 699008a16 is not reverted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 18:28:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1ca329d4a6 fix(nextcloud): resolve the php-fpm upstream at request time, not config parse
The internal proxy declared a static upstream to nextcloud:9000, which
nginx resolves at config-parse time and treats as fatal (exit 1) on
NXDOMAIN. In swarm the nextcloud service name only appears once a task
passes its healthcheck, and the NFS-backed first-run init held the lock
~25s, so every proxy task that started in that window crashed with
'host not found in upstream', swarmkit's recent-failure penalty pinned
all replacements onto the manager, and the worker-placement gate failed.
Drop the static upstream and resolve nextcloud:9000 through a variable
(set $php_handler; fastcgi_pass $php_handler) using the http-level
resolver already emitted, mirroring the whiteboard/talk runtime
resolution in the same file: nginx now starts and returns 502 until the
app is healthy instead of crash-looping, so replicas keep their worker
placement. Validated with nginx -t.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 17:05:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cb52a72f2e fix(dr-drill): drill node-local workloads instead of skipping them
The drill entry gate treated 'deploys no swarm service' as 'has nothing
to restore', but the backup chain operates on NFS volumes and deployed
backup units, not on scheduler objects; bigbluebutton's seven
NFS-backed volumes went entirely unproven. Gate the drill solely on the
NFS-flagged volume and make only the full-disaster step
workload-aware: stack roles keep docker stack rm, node-local roles get
their compose project (label com.docker.compose.project=<entity>, the
compose handler's --project convention) stopped on every node so no
writer races the export wipe and restore. Declare
workload: node-local on bigbluebutton so the phantom-service
converge/chaos gates skip while play, e2e and now the drill keep
running; svc-runner stays skipped via its missing NFS volume. Document
the nine drill steps host-by-host in the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 16:07:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 131231ee28 fix(loader): stop consumer compose_mode_force from leaking into lazy providers
web-app-matrix forces compose mode for its own ansible flavor; when it
was the first consumer to lazy-load the object-store provider, the
include_role inherited that override and web-app-seaweedfs came up via
compose up instead of docker stack deploy, so seaweedfs_seaweedfs never
existed and the next consumer's bucket provisioning died on
resolve-container-id. Neutralize compose_mode_force on the lazy
provider includes of sys-svc-objstore and sys-svc-engine, matching the
neutralization sys-svc-rdbms already carries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 16:07:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0275fe502b fix(rdbms): point dedicated db sidecars at the pullable upstream image
The dedicated database branch reused the provider's <db>_custom tag,
which only exists where the central db stack happened to build it; a
cold runner deploying a central-db-disabled variant (nextcloud 1) hit
'pull access denied for mariadb_custom' and the aborted pull dragged
the remaining service pulls down with it. Pin the sidecar to the
upstream reference via the image= override: the CI mirror overrides
rewrite exactly that config path, so CI pulls the ghcr mirror and local
deploys pull the hub. mariadb's custom image is a bare passthrough and
postgres' baked healthcheck is shadowed by the compose-level
healthcheck, so the sidecar behavior is unchanged; the postgres
extensions branch keeps its local build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 16:07:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ee07cd96b2 fix(ci): cap variants per compose job per role instead of globally
The nextcloud compose job bundled variants 2,3 into one runner and
needed ~332min of deploy time against the 300min step budget inside
GitHub's hard 360min job cap; raising the timeout leaves no headroom
for the measured runtime variance and starves the post-steps that
upload artifacts. Add a variant_bundle_size override in meta/tests.yml,
consumed by the variant bundle packer and the complexity report alike,
and set it to 1 for web-app-nextcloud only: eight ~190min jobs with
over 100min of headroom while every other app keeps the global
bundling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:23 +02:00
kevinveenbirkenbachandClaude Opus 4.8 768e210c27 fix(compose): keep compose warnings out of pull.py's parsed output
run_cmd merged stderr into stdout, so compose warnings ('variable is
not set', ...) leaked into everything pull.py parses: the yaml config
for pull targets, the config --images fallback list (turning warning
lines into phantom image names), and the pull --help probe. A project
with an unset optional variable then failed the pull step even though
every image was present locally. Capture the streams separately, parse
stdout only, and forward stderr to the log.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:23 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f5ddd58b5f fix(matrix): move the inner docker daemon off the 172.17 default bridge
Since the inner mdad dockerd actually starts (storage-driver fix), its
default docker0 claims 172.17.0.1 inside the container netns and
shadows exactly the resolver address the swarm inventory hands to every
container via networks.internet.dns, so the bootstrap's ansible-galaxy
died on 'Could not resolve host: github.com' one second after boot.

Declare the nested-docker bridge ranges as a network SPOT
(NETWORK_NESTED_DOCKER_BIP / _ADDRESS_POOLS, deliberately outside
172.16.0.0/12 and away from the 192.168.244.0/24 lab net) and render
the runner's daemon.json from it at staging time instead of shipping a
static file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f544845591 fix(swarm-ci): stop the heavy variants from exhausting the runner disk
The DR drill ran in every variant job because its gate checked the
job-local plan index, which is 0 in each of the single-variant CI jobs;
gate it on the absolute round index so the destructive drill and the
backup host deploy run once per app, in the variant-0 job. Guard the
matrix steps with a disk watchdog that aborts visibly below 6G free
instead of letting the Actions Worker die silently on ENOSPC with no
artifacts. Cap node journald (256M) and DiD container logs (10m x 2),
which were unbounded across five nodes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c41343e8e2 fix(dr-drill): restore the volume subtree it proves and stop exhausting the disk
The marker discovery matched any .dr-drill-marker under files/, so
SRC_REL could resolve to a parent tree and step 8 restored the wrong
subtree into the live export as root, leaving n8n_data owned root:root
and n8n (uid 1000) crash-looping on EACCES. Scope both repo layouts to
the primary volume's own marker and declare uid/gid 1000 on the n8n
data volume so nfs_prep enforces ownership on every deploy instead of
relying on Docker's copy-on-first-use.

The drill also stacked four copies of the backup tree on shared space
(live export, pulled tree, fully-allocated LUKS image, final restore
into a tmpfs /tmp) and ENOSPCed on large apps. Restore to /var/tmp,
drop the pulled tree before the device recover, make the loop image
sparse and free it after the marker verify, and stage only the volume
generation instead of the whole machine tree.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 76508c946a fix(svc-runner): declare the node-local swarm workload for the CI gates
The swarm convergence gate derived runner_runner-1 from the services
metadata and polled a stack service the role never creates: the runner
fleet deploys as node-local compose projects on every node because each
runner needs its own node's docker socket and identity. Declare
workload: node-local on the runner services entry and let the gate
context map it to HAS_SWARM_SERVICE=false, so the stack-service gates
skip themselves while the swarm deploy, the registry distribution chain
and the CLI e2e keep running. Roles without the attribute keep the
image/template heuristic; workload: stack forces the gates on. Document
the runner architecture for both deploy modes in the role README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 13:46:22 +02:00
Alejandro Roman cb3382f0d5 add test, fix SPOT 2026-07-15 12:53:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 236117f6f1 fix(matrix): let the inner docker daemon start in nested overlayfs setups
The mdad runner pinned storage-driver overlay2, but on swarm test nodes
the container's /var/lib/docker sits on overlayfs and the kernel rejects
overlay-on-overlay upperdirs, so the inner dockerd exited, the bootstrap
oneshot failed by dependency and Synapse never came up (proven by the
rescue-diagnostics journal of run 29356373746). Drop the pin so dockerd
auto-detects and ship fuse-overlayfs as the nested fallback; real hosts
keep overlay2.

Replace the 30min blind HTTP poll with a bootstrap terminal-state wait:
fail fast with diagnostics when the inner docker daemon or the bootstrap
playbook fails, probe Synapse only after the oneshot completed, and run
the journal dump on every failure path instead of behind MODE_DEBUG.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07363ca3ba fix(hlth-container): judge swarm services by each slot's newest task
The manager-only convergence check classified a service by the
CurrentState of every task docker service ps returns, so a completed
run-once job whose first attempt failed before the database was ready
kept tripping the health gate on historical Failed entries, failing the
terminator for the full retry window while workers passed. Classify by
DesiredState and CurrentState of only the newest task per slot: stale
history no longer votes, a newest failed task still flags, and live
non-converged services keep flagging. stack_ready.sh gets the same
classifier so the deploy gate and the health gate agree.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6dcc564911 fix(nextcloud): make the Zammad OAuth coupling swarm-capable
The partner probe and both provisioning execs addressed the bare
compose container name zammad-railsserver, which does not resolve in
swarm; the probe silently skipped the whole coupling and the connect
control never rendered. Route them through the container_address
resolver with the shell module (unquoted, subshell expansion), gate the
provision include on IS_STACK_HOST, and retry the rails provision to
absorb Zammad convergence lag. Compose behavior is unchanged: the
lookup returns the bare name there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 05c12b5140 chore(backup): bump baudolo to 3.1.0
Picks up the --empty pre-clean fix (per-statement \gexec drops and
non-public schema removal) needed by the discourse and gitlab restore
drills.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Fable 5 15e707e9b2 fix(svc-runner): resolve empty entity name breaking swarm stack naming
get_entity_name('svc-runner') returned '' because the exact-match branch short-circuited on the category path 'svc-runner' before the shorter 'svc-' prefix could strip. The empty entity produced an empty swarm stack name (docker stack deploy "") and SERVICE_NAME '_' in the swarm convergence probe ('no such service: _' -> timeout). Record the exact match as a fallback and keep scanning so a shorter category prefix wins; simulated across all 254 roles, only svc-runner changes ('' -> 'runner').

Re-key roles/svc-runner/meta/services.yml from 'svc-runner' to the entity name 'runner' (services.yml is keyed by primary entity) and its 12 vars lookups, and add proxy.service_key: runner-1 so the swarm probe targets the real service (runner_runner-1) instead of the non-existent runner_runner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Fable 5 a786c8fcd7 fix(backup): make the DR-drill pull survive rsync version drift and stalls
The backup pull ssh forced-command wrapper exact-matched a fixed rsync --server flag string. rsync 3.4.x renegotiates that flag block (and our own --numeric-ids was added), so the match fell through to the reject echo, which polluted the rsync protocol stream and failed every pull with 'protocol incompatibility (code 2)'. Glob the version-dependent flags while pinning mode and destination, and reject shell metacharacters so the relaxation cannot be abused.

The pull script carried no ssh/rsync timeouts and the oneshot unit inherits the 24h default start timeout, so a stalled provider hung the whole job budget. Add ssh BatchMode/ConnectTimeout/ServerAlive and rsync --timeout so a dead provider fails in seconds and the retry loop reports it.

Add an integration test that routes a real rsync client through the rendered wrapper (the --numeric-ids path) and asserts the pull works and injection is rejected, so future rsync version drift fails in the suite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 04:25:50 +02:00
kevinveenbirkenbachandClaude Fable 5 d37520bb69 fix(nfs): anchor services.yml SPOT paths to PROJECT_ROOT
utils.storage.nfs read the nfs-server/client services.yml SPOTs via a CWD-relative path. The recover drill imports these inside the deploy container where the CWD is not the repo root, so get_export_base() raised FileNotFoundError. Anchor both paths to utils.PROJECT_ROOT (matching utils.paths), making the read CWD-independent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 01:12:29 +02:00
kevinveenbirkenbachandClaude Fable 5 e272ae2d04 fix(espocrm): correct sed delimiter and ampersand escape in entrypoint patch
The Dockerfile RUN that appends '|| true' to the upstream create-admin-user line used '|' as the sed delimiter, which collides with the '||' in the replacement (sed: unknown option to s), and left the '&' in '2>&1' unescaped (sed expands it to the whole match). Use '#' as the delimiter and escape the ampersand as '\&'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 01:12:29 +02:00
kevinveenbirkenbachandClaude Fable 5 ce3c515eb3 fix(deploy): neutralize apt-daily so it cannot hold the dpkg lock
apt-daily/apt-daily-upgrade services triggered at container boot hold the dpkg frontend lock, so the ansible apt tasks (module default lock_timeout 60s) fail with 'Unable to acquire the dpkg frontend lock'. Stopping only the timers left an already-running service holding it. Stop the timers and services, then mask all four units so nothing can re-trigger background apt during the deploy. The apt-readiness logic moves into tasks/utils/apt_ready.yml, included from the constructor under a single Debian-and-container guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 01:12:29 +02:00
kevinveenbirkenbachandClaude Fable 5 df6bf3096d chore(make): add quality-high gate (quality then lint) + i8quhi alias
quality-high runs the quality gate (autoformat + test) and then every lint check as ordered prerequisites, giving one full pre-push gate. Add the i8quhi operator alias so every make target keeps alias coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc0799883f refactor(structure): move prefix-grouped files into named subfolders
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:

- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
  nested a second level); registry imports and the four handler-dir
  globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
  select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
  includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
  filters() dict keys, so Jinja usage is unaffected; ansible discovers
  the one-level-deep subdir)

Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 eaf53257cf test(lint): fail on sibling files sharing a name prefix
Scan git-tracked files per directory and fail when three or more
siblings share a delimiter-bounded name prefix (the longest common
token run); a shared prefix is a folder in disguise, so the group
belongs in a subfolder named after the prefix with the prefix dropped
from the file names. Test-prefixed files, dotfiles/dunders and purely
numeric ordering prefixes are exempt, and directories whose flat layout
an external contract enforces (.github/workflows, ansible lookup dirs,
addon dirs) are skipped structurally rather than via a grandfather list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbach db2c005ddb Autoformat 2026-07-14 19:35:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6d44e0cd0a fix(gitlab): kernel-safe container hostname on onion nodes
On an onion node the gitlab canonical (gitlab.<56-char-onion>.onion,
69 bytes) exceeds the kernel's 64-byte UTS cap, so runc dies with
'sethostname: invalid argument' and the container never starts. Use the
container_hostname lookup, which falls back to the entity name when the
domain exceeds 63 bytes — byte-identical on clearnet, same pattern as
the sibling roles adopted in c8a432047. external_url is untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:35:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 da1d846310 fix(openwebui): source CANONICAL_DOMAIN from onion-aware domain lookup
The applications lookup returns the raw domains map that never receives
onion injection, so on an onion node CANONICAL_DOMAIN stayed clearnet
while the app itself rode onion (SOCKS proxy set). isOnionTarget() then
returned false: onion skips never fired and timeout scaling stayed at
clearnet levels. lookup('domain') resolves through the injected map and
is byte-identical on clearnet (single-entry canonical).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:35:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6f9142027d fix(e2e): skip seaweedfs filer check on onion targets
When an app rides onion, Playwright routes through the node Tor SOCKS
proxy, which cannot resolve the clearnet-internal SeaweedFS filer UI
domain (net::ERR_SOCKS_CONNECTION_FAILED). Skip the filer storage check
on onion targets in all consumer specs — same pattern already proven in
the nextcloud spec. Clearnet runs are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:34:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 89c6d43b77 revert(bbb): deploy clearnet vhost on onion nodes
The onion guard on the 01_core include skipped the whole bbb deploy on
onion nodes, so no vhost existed and the E2E readiness probe against
bbb's clearnet canonical failed with TLSV1_UNRECOGNIZED_NAME. bbb pins
tor.enabled false, so its canonical stays clearnet and the role deploys
byte-identically on onion and clearnet nodes.

Drop the nextcloud onion app:disable for cloud_bbb accordingly: with
bbb deployed again its api.url is a valid clearnet URL and the addon
spec already skips on onion targets.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 19:34:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4a36ef5885 chore: drop a narration comment and apply autoformat line wrapping
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c3edf7750b fix(swarm): bound registry sync subprocess calls at 10 minutes
A docker pull against ghcr wedged for 8+ minutes with zero progress
during the update pass and only the 1h task timeout would have
reaped it; give every pull/tag/push subprocess a 600s bound so the
handler's own retry loop takes over instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4527698014 fix(suitecrm): allow the SPA's inline and eval scripts in the CSP
With SSO disabled the public root serves the SuiteCRM 8 SPA directly
instead of redirecting to oauth2-proxy, and the Angular shell plus
bundled legacy code use inline and eval scripts, so hlth-csp failed the
deploy on every SSO-off variant; the SSO variant only passed because
csp-checker skips redirecting pages. Same allowance pattern as the
espocrm SPA. Deliberate XSS-hardening trade-off on this vhost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:28 +02:00