Commit Graph
64 Commits
Author SHA1 Message Date
kevinveenbirkenbach fb004b88a0 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/web-app-mattermost/meta/services.yml
2026-11-04 19:43:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1c561a1e7d test(install): keep role-file paths out of the stage fixtures
tests/lint/ansible/roles/test_file_spot.py rejects a hard-coded
"roles/a/tasks/main.yml" in test_stage.py and asks for the
ROLE_FILE_TASKS_MAIN constant. The fixture only needs some nested file,
not a role, so it uses pkg/a/data.yml and pkg/a/.git instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:09 +01:00
kevinveenbirkenbachandClaude Opus 5.5 330b85f9e7 fix(apparmor): lift the unprivileged-unconfined restriction in teardown
On runner image 20261101 kernel.apparmor_restrict_unprivileged_unconfined
is on, so the kernel turns runc's change_onexec into the stack
docker-default//&unconfined. docker-default grants no signal to an
unconfined peer, and every in-container kill() is denied; run 41743807934
lost four package builds that way (journal: apparmor="DENIED"
operation="signal" comm="rsync" signal=usr2).

teardown.sh now writes 0 to the sysctl and records the value it found
under /run/infinito; restore.sh writes that value back and drops the
record. A missing sysctl or one already at 0 is left alone, and a second
teardown keeps the first record. CI runs teardown before every deploy,
workspace and dns job; dev hosts through make environment-bootstrap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:08 +01:00
kevinveenbirkenbachandClaude Opus 5.5 fd05f85d44 fix(install): stage the package build workspace with tar, not rsync
Run 41743807934 lost Matrix#2, Workspace (debian:13), Gitea#0 and Wazuh#0
to the full job budget right after "Preparing build workspace". On runner
image 20261101 the kernel stacks every container onto
docker-default//&unconfined, which refuses every in-container kill(); the
host journal logs apparmor="DENIED" operation="signal" comm="rsync"
signal=usr2. rsync's local copy waits for exactly that SIGUSR2 and sleeps
until the job is killed.

scripts/install/stage.sh copies the tree through a tar pipe, which sends
no signal; --format=posix keeps nanosecond mtimes. package.sh uses it on
the arch and debian paths and installs tar instead of rsync.

Copying this repository both ways gives identical content, modes, types,
link targets and mtimes on all 11306 entries. test_stage.py runs stage.sh
under a seccomp filter that fails every signal syscall with EACCES; the
old rsync copy hangs past its timeout under the same filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:08 +01:00
kevinveenbirkenbach 9f4be23164 Merge remote-tracking branch 'origin/main' into feature/store-i18n 2026-11-03 22:56:04 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c3c0e9db77 fix(swarm): test service membership without a pipe into grep -q
_context.sh runs under set -euo pipefail and tested membership with
`printf '%s\n' "${_names}" | grep -qxF <name>`. grep -q exits on the first
match; when printf still has lines to write it dies of SIGPIPE (141), and
pipefail turns the matched test into false. resolve_db_service then fell
through to the central <dep>_<dep> name or printed nothing, and
has_swarm_service reported a deployed service as missing, so
skip_if_no_swarm_service let the converge and chaos gates exit 0 untested.
The race grows with the stack's service count and with CPU load (store-i18n
gate pass 11: 'mariadb_mariadb' != 'magento_database').

The three tests read the listing from a here-string, which is no pipeline.
Two regression tests put the match first in a 10000-name listing and fail
on every run against the old helper. Root cause found by i18n 2.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 22:49:39 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1ec96181b1 Merge remote-tracking branch 'origin/main' into feature/store-i18n
Conflicts resolved by combining both sides: utils/tests/swarm/run.py keeps store-i18n's re-export of run_watched from watchdog.py, and the #1330 storage roots (lowest_free_mb over / and /var/lib/containerd, the fullest root in the abort banner and the sample line, df over every root) move into watchdog.py; test_run_storage_roots.py now targets watchdog and expects the process-group kill. The seaweedfs 4.52 bump lands in roles/web-svc-seaweedfs, where store-i18n moved the engine; roles/web-app-seaweedfs keeps only the console.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 13:53:44 +01:00
kevinveenbirkenbachandClaude Opus 5.5 2d7bf740b0 fix(ci): put containerd, Docker and swap on the runner's empty spare disk
The small GitHub runner shape (AWS, 72G root) ran out of disk in swarm and compose rows (#1329: WordPress#2 of run 41261531965, Nextcloud#6 and #7 of run 41262286962) while an empty instance-store disk nvme1n1 sat unused. mount_spare_disk.sh formats and mounts it at /mnt, but only on a GitHub-hosted runner outside act, and only a whole unpartitioned disk without any signature, mount or swap use of at least 75G; every decline leaves a notice. relocate_docker_dataroot.sh then bind-mounts /var/lib/containerd from the spare disk and moves the Docker data-root to /mnt/docker-host (not /mnt/docker, which is the nested daemon's volume that dedicated.sh wipes). It refuses before stopping anything when images, volumes, build cache, foreign containerd namespaces or shims exist or daemon.json is unreadable, and rolls back the bind mount and daemon.json on any later failure or cancel. Swap goes on / once containerd lives on /mnt, dump_state.sh logs disk size, model and free space per storage root, and the swarm watchdog samples / and /var/lib/containerd every 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:31:02 +01:00
kevinveenbirkenbach 69e0d967a0 Merge branch 'feature/swarm-inventory-extra-vars' into feature/store-i18n
# Conflicts:
#	Makefile
2026-10-31 17:19:25 +01:00
kevinveenbirkenbach 620fafa6d1 feat(swarm): pass INFINITO_INVENTORY_EXTRA_VARS into the swarm-zombie inventory 2026-10-31 16:38:30 +01:00
kevinveenbirkenbachandClaude Opus 5.5 34506554f7 fix(cache): download pacman packages per node and publish them into the shared cache
The shared pacman cache was the first writable CacheDir on every arch runner and lab node. Our arch image sets DisableSandbox, so libalpm downloads straight into that directory (util.c _alpm_download_dir_setup only makes a per-process temp dir with the sandbox on), and the XferCommand writes <pkg>.part there with curl -C -o %o; two nodes fetching the same package appended to the same .part file, and the second rename failed or left a corrupt package in the shared cache.

pacman.sh now gives each node its own download directory <shared>/pacman/node-XXXXXX/ (created once with mktemp, reused from pacman.conf on later runs) as the CacheDir between the read-only host layer and <shared>/pacman/, and writes a PostTransaction hook to pacman.d/hooks/infinito-shared-cache.hook that renames every finished package and signature into <shared>/pacman/; .part files stay for the next resume. Both directories sit in one bind mount, so each publish is an atomic rename(2). pacman.sh fails when pacman.conf sets HookDir, since pacman would then skip the hook silently. Requirement 065 and cache.md describe the three layers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:43:01 +01:00
kevinveenbirkenbachandClaude Opus 5.5 74d99201e0 feat(cache): share package and tool caches across stacks and release swarm clusters after rescue
Parallel compose slots, swarm lab clusters and GUI lanes each kept their own pip, npm and pacman downloads, and a failed swarm run left its cluster (about 25 GB of node docker data plus the rescue bundle in the act container) on the root filesystem until someone ran make swarm-down. INFINITO_CACHE_SHARED_HOST_PATH (default <DIR_CACHE>/shared) is now mounted read-write at the same path into every compose slot and every swarm lab node, with PIP_CACHE_DIR and npm_config_cache below it; empty, the renders stay byte-identical to before. On a host whose os-release ID is exactly arch running an arch stack, INFINITO_CACHE_HOST_PACMAN_PATH mounts the host pacman cache read-only and scripts/docker/cache/pacman.sh makes it the first CacheDir with <shared>/pacman/ second, so new downloads land in the shared folder and the host cache stays untouched; on Manjaro and every other host pacman uses the shared folder alone.

make swarm-zombie without keep=true now copies the act job's rescue diagnostics and Playwright artefacts to the host and then releases the lab nodes, the lab network, the NFS export volume and the act outer container, on success and on failure; when the copy fails the nodes still go, the act container stays and the run exits naming make swarm-down. make roundtrip passes keep to every swarm step, make clean-cache removes the shared folder, and the disc-space cleanup skips pacman cache directories it cannot write. apt and dnf keep their per-container caches behind the package-cache proxy, and no docker data root is shared between daemons. Requirement 065.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:04:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 33591ab688 fix(roundtrip): pin the swarm step to clearnet when Tor is disabled
The workspace swarm roundtrip for svc-db-postgres failed on every host
with "The lookup plugin 'tls' failed: NETWORK_MODE 'multi' needs
svc-net-tor with a node onion on this node" (CI run 40979252521, job
Workspace debian:13). roundtrip.sh (b2469ff948) pinned the swarm step to
multi whenever the validator accepts it and never looked at disable=,
while 03_roundtrip.sh passes a disable list that removes tor. The
hard-fail is the documented contract (network_modes.md: tor or multi on
a node without the provider fails the deploy); the request was wrong.

roundtrip.sh now pins clearnet when disable= holds the Tor key (read
from utils.github.variant.network.TOR_SERVICE), else multi when the role
takes it, else the rotation. 02_zombie.sh, which also disables tor,
passes network=clearnet instead of relying on the rotation drawing it.
TestRoundtripSwarmNetwork runs the real script against stubs; the old
pin logic fails three of its cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 08:49:50 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5e8890d208 Merge branch 'feature/updater' into feature/store-i18n
Brings in feature/updater accb22da75, which carries main 5995766f58.

Conflict resolutions:
- web-app-fider: main's tenant bootstrap (01_tenant.yml) runs first on
  the stack host, OIDC setup moves to 02_setup_oidc.yml, design to
  03_design.yml; flat includes so main's gate test reads them.
- sys-front-tls test_nginx_ssl_header: the tls fake maps the domain term
  store-i18n passes back to its app.
- group_vars 04_webserver.yml, sys-svc-proxy CSP header: store-i18n's
  webserver_resource lookup plus #1256's WEBSERVER_SECURITY_HEADERS and
  the security_headers.lua include after the CSP block.
- web-app-nextcloud: updater's decoupled Talk stack (Janus, signaling,
  NATS, recording builds) replaces aio-talk; store-i18n's per-network
  Talk backends move from the wrapper script into signaling.conf.j2 urls
  (NEXTCLOUD_HPB_SIGNALING_BACKEND_URLS from network_urls, canonical
  first); IS_COMPOSE_MODE/IS_SWARM_MODE booleans kept; the integration
  OpenProject digest stays delegated to STACK_HOST.
- users lookup docs, token-store write_many, cache README, meta info
  lint: both sides combined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:40:05 +01:00
kevinveenbirkenbachandClaude Opus 5.5 6d44e1e28a fix(ci): check the PR head against the main it was tested with
The Validation step "Assert the base branch is fully merged into the PR
head" ran main_merged.sh, which fetches the live tip of main when the job
starts. PR runs wait in the queue for hours while image-update PRs land
on main several times an hour, so nearly every queued run failed
Validation on start and had to be re-pushed after another main merge
(PR #1265: behind_by=0 at push, behind_by=9 while its run still queued).

main_merged.sh takes a new INFINITO_MAIN_COMMIT override: when set, it
fetches that full commit SHA from the remote and requires HEAD to
contain it instead of the live tip; a set but malformed or empty value
exits 2. The Validation step passes github.event.pull_request.base.sha,
the main tip at event time, which the step's pull_request-only guard
always provides. The run tests the merge against that base, so the gate
still guarantees the tested tree contains it.

The local pre-push hook (make test-main-merged) leaves the override
unset and keeps checking the live tip: a push happens right now, so
lagging the current main is the real defect there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:10:45 +01:00
kevinveenbirkenbachandClaude Opus 5.5 3346049481 test(rescue): signal the drill's stand-in group instead of relying on GNU timeout
91f25a35f2 drove the budget-kill drills through `timeout -k GRACE 0` and sent it SIGALRM at the deadline, relying on GNU timeout treating that signal as its own expiry: TERM to the stand-in's process group, KILL GRACE seconds later. The ubuntu-26.04 runner ships coreutils from uutils (`9.5-1ubuntu2+0.0.0~ubuntu25`), whose timeout does not handle SIGALRM that way, so the stand-in's `sleep 1000` outlived the driver, kept the captured pipes open and both drills hit their 120 s subprocess timeout (CI run 40869769501, test_window.py:191 and :204). Production code uses neither `kill -ALRM` nor `timeout … 0`; only the drill did.

The driver now starts the stand-in as its own process group (`set -m`), keeps the armed FIFO from 91f25a35f2, sends TERM to the group at the deadline and KILL after GRACE seconds if it is still alive, and exits 124 or 137 like GNU timeout did. It no longer depends on any timeout implementation; all eight drills pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 13:53:36 +01:00
kevinveenbirkenbachandClaude Opus 5.5 91f25a35f2 test(rescue): start the drill's kill clock only once the stand-in is armed
BudgetKillDrillTests drove the stand-in deploy with `timeout -k GRACE 3`.
Under load (gate pass 76 ran unit, integration and lint in parallel)
timeout's TERM arrived while bash was still starting, before the
stand-in's `trap cleanup EXIT` existed, so bash died on TERM without the
rescue: test_without_the_window_the_kill_lands_before_the_copy failed
with "124 != 137" and only "kill_epoch=…" on stdout. A 4 s stall before
the trap reproduces it on the old driver; nothing in the merge touched
the rescue code.

The stand-in now writes to an `armed` FIFO right after it installs the
trap. The driver runs `timeout -k GRACE 0 …` (duration 0 disables
timeout's own clock, its TERM-then-KILL handling stays), waits for
`armed`, sleeps until the deadline epoch and sends timeout the same ALRM
its own expiry raises, so TERM still lands at the deadline and KILL
GRACE seconds later, with the same 124/137 exit codes. `wait -n -p`
lets the failure-before-the-deadline drill return as soon as its deploy
ends. No timeout was raised and nothing retries; with the 4 s stall all
three drills pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 03:07:12 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a5de245bf9 fix(ci): let the ZFS data-root pools hand freed blocks back to their images
docker_dataroot.sh backs each node's ZFS pool with a sparse loop image
and created the pool without autotrim. ZFS never discarded freed blocks
to the loop device, so the image only grew: in WordPress#0 of run
40625679270 the runner still held 118G after the teardown had removed
every container ("Containers: 0") while only 47G of files remained, and
the pools held ~38G of live data. The step had already aborted at
"DISK EXHAUSTION IMMINENT (<6144M free on /)" with 139G of 145G used
(rc=75). The other ZFS swarm jobs of the run show the same df-minus-du
gap (wordpress1 73G, odoo1 73G, magento0 82G, weblate1 55G, gitlab2
44G); ext4 and btrfs jobs show none.

The pool is now created with -o autotrim=on, so ZFS discards freed
ranges to the loop device, which punches holes into the image file.

test_docker_dataroot runs the script's zfs path against stubs of every
host tool and asserts the single zpool create carries -o autotrim=on;
against the previous script that assertion fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:07:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 8a0e858f12 refactor(ci): group the rescue helpers under scripts/tests/deploy/utils/rescue
2bf329cd3b put rescue_window.sh next to rescue_index.sh, and
test_prefix_grouped_files rejects two siblings sharing the `rescue`
prefix (quality-high pass 67). Both move to utils/rescue/ as index.sh and
window.sh, their tests to the mirrored tests/.../utils/rescue/, and every
caller follows: dedicated.sh, the swarm teardown in 00_one.sh, the
workspace workflow, and the swarm and diagnostics READMEs.

The budget-kill drill gave the stand-in a 1 s budget; under the full
parallel unit run the SIGTERM arrived before its EXIT trap was installed,
so nothing was captured and the test failed. The two kill drills now use
KILL_BUDGET = 3 s; their assertions are unchanged. A full `make test-unit`
passes them under load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:01:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 2bf329cd3b fix(ci): fit the rescue capture into the kill grace of a budget kill
distros.sh stops a distro at its budget with `timeout -k 60`: SIGTERM at
the deadline, SIGKILL 60 s later. dedicated.sh's cleanup trap then ran
the in-container capture under `timeout 1500` (266 s in Baserow#0 of run
40625679270) and the host capture under `timeout 900`, so the SIGKILL
always landed mid-capture, the snapshot was never copied out and the
rescue-diagnostics artifact was lost. The inner `timeout` also sat in its
own process group and outlived the kill, holding the step open.

- distros.sh keeps the grace in one place, KILL_GRACE_SECONDS, and
  exports INFINITO_CI_DISTRO_KILL_EPOCH next to the deadline epoch.
- scripts/tests/deploy/utils/rescue_window.sh decides per trap step:
  past the deadline, the step gets half of the time left until the
  SIGKILL (at least 1 s) and an end epoch; before it, the nominal
  timeout and no end epoch. Inside the trap `$?` is 0 on a budget kill,
  so the deadline epoch is the only reliable signal.
- dedicated.sh runs both captures through `timeout --foreground` with
  that window, keeping them in the trap's process group, and passes the
  end epoch as RESCUE_DEADLINE.
- utils/diagnostics/container.py caps every command at RESCUE_DEADLINE,
  writes `[skipped: past RESCUE_DEADLINE]` for what no longer fits and
  hands the deadline to nested captures. Without the variable nothing
  changes, so the swarm and workspace callers behave as before.

-k 60, the distro budget and every timeout-minutes are unchanged. Drill
tests replay the trap with the real helper and container.py: a budget
kill ends with rc 124 and the snapshot copied out, the old shape ends
with rc 137 and nothing copied, an early failure keeps the full capture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 19:11:15 +01:00
kevinveenbirkenbachandClaude Opus 5 677a63931c Merge branch 'feature/network/multi-network-serving' into feature/store-i18n
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.

Where the two sides covered different ground, both are kept rather than one
picked:

- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
  issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
  an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
  without a provisioned node, now gated on their tor_serves_host; their
  node_onion_of returns an empty string there, which would have served every
  onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
  Their per-network branch omitted the issuer domain on the onion arm, which
  breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
  hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
  resolution; their side reduced the logo sources to the bare service
  provider values.

Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.

Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:

- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
  module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
  longer calls

Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 09:53:41 +02:00
kevinveenbirkenbach d0b45f8b2d Merge branch 'main' of github.com:infinito-nexus/core into feature/store-i18n
# Conflicts:
#	.github/workflows/call-images-build-ci.yml
#	.github/workflows/call-test-deploy.yml
#	.github/workflows/call-test-install-pkgmgr.yml
#	.github/workflows/call-test-workspace.yml
#	roles/svc-bkp-local-2-device/README.md
#	roles/svc-storage-nfs-client/README.md
#	roles/web-app-pixelfed/meta/services.yml
#	scripts/tests/deploy/act/workflow.sh
#	tests/unit/python/utils/recovery/test_base.py
2026-10-20 20:05:15 +02:00
kevinveenbirkenbach feb5630f19 Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving 2026-10-20 17:14:44 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a6c3c1cc91 fix(apparmor): keep docker-default loaded when tearing AppArmor down
`security-apparmor-teardown` ran `aa-teardown`, which unloads every
profile including dockerd's in-memory `docker-default`. dockerd only loads
that profile when it starts, so every container start afterwards that
requests it fails until dockerd restarts. Since #1067 runs the teardown on
the GitHub runners, BuildKit failed each RUN of the swarm node image three
times (run 39332343490, 39 of 40 swarm jobs):

    runc run failed: unable to start container process: error during
    container init: unable to apply apparmor profile: apparmor failed to
    apply profile: write .../attr/apparmor/exec: no such file or directory

The teardown now removes each loaded profile by name through securityfs
and skips `docker-default` and child profiles, which leave with their
parent. Dev hosts gain the same: no dockerd restart is needed after
`make environment-bootstrap`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 12:11:14 +02:00
kevinveenbirkenbachandClaude Opus 5 1c950d0c5e fix(ci): widen the headroom test margin again and shorten an overlong task name
The full gate failed test_a_distro_needs_the_slowest_run_plus_headroom_to_start on a quiet machine, so the load is the gate own parallel workers rather than anything beside it. The branch under test needs d < B < 2.15*d for a measured first duration d, the rule 040955ebf2 wrote down when it widened the same fixture from sleep 2 / 3s to sleep 4 / 7s. At 4 / 7 the window admits a first iteration of 4 to 6 seconds, three seconds of slack, sized then against the single second CI had overshot by. Under the gate it overshoots further, takes the budget exhausted branch and never prints the line the test greps for. At 10 / 19 the window admits 10 to 18 seconds, nine seconds of slack. The assertion and the branch are unchanged and the suite pays six more seconds.

The second hunk shortens the Odoo reload task name to 60 characters; at 83 it broke the 80 character convention the name lint enforces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 16:04:09 +02:00
kevinveenbirkenbach f8311d0780 Merge branch 'main' into feature/network/multi-network-serving 2026-10-16 22:36:07 +02:00
kevinveenbirkenbach de0c4e0e00 Merge branch 'feature/designer' into feature/store-i18n 2026-10-16 20:07:01 +02:00
kevinveenbirkenbachandClaude Opus 5.5 891d46d589 fix(ci): time the DNS53 sampler probes from /proc/uptime
The 🐳 Workspace (ubuntu:latest) job on main (run 38879556215, 1b36ef6120)
failed test-unit-python in the hook's second make test:

    AssertionError: Regex didn't match: '^ localhost=ok/\d+ms$'
    not found in ' localhost=ok/-1612931482679767779ms'

ubuntu:latest is Ubuntu 26.04, whose date is uutils coreutils. uutils
0.7.0 ignores the field width in %3N and prints all nine nanosecond
digits, so `date +%s%3N` yields a 19-digit value instead of epoch
milliseconds. Against main's sampler, uutils 0.7.0 date reports
localhost=ok/7081859ms, and an unpadded nanosecond value turns the
difference negative.

answer_once now takes both timestamps from /proc/uptime, which the
kernel prints with two decimals whatever date the image ships. The
sampler already reads /proc/net/udp, so it was Linux-only before. The
resolution is 10 ms; the same probe reports 10 to 20 ms with uutils
date on PATH.

The new test puts a date on PATH that does not understand %3N; main's
sampler dies on it, the fixed one does not call date for the latency.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 14:15:25 +02:00
kevinveenbirkenbachandClaude Opus 5 4a08d58e55 test(cache): point the CA test at the path the script was moved to
230d9a1e5e moved scripts/docker/cache/package-frontend/ca.sh to roles/svc-cache-package/files/shell/ca.sh as a pure rename. On origin/main the script stayed where it was, and 2756fd56f1 changed it there and added this test. The merge took the test without the move, so every case ran bash against a path that no longer exists and failed with exit 127.

Only the path changes. The script it exercises is the same file the rename produced, and every other PROJECT_ROOT path under tests/ was checked against git ls-files: registry-ca.sh is still at its old location and its test is correct, cert-gen.sh moved along but has no test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-16 00:22:13 +02:00
kevinveenbirkenbach 810c8bc413 Merge branch 'main' into feature/store-i18n 2026-10-15 23:51:34 +02:00
kevinveenbirkenbachandClaude Opus 5.5 2756fd56f1 fix(cache): install CA anchors where every update-ca-trust family reads them
The Fedora and CentOS Stream workspace jobs failed TLS to pypi.org,
registry.npmjs.org and repo.packagist.org (CERTIFICATE_VERIFY_FAILED,
curl error 7). package-frontend/ca.sh put the frontend CA into the Arch
anchor dir /etc/ca-certificates/trust-source/anchors, which p11-kit on
the RedHat family never reads.

Probing for the anchor dir does not fix it: with-ca-trust.sh creates
both /etc/pki/ca-trust/source/anchors and
/etc/ca-certificates/trust-source/anchors on every update-ca-trust host,
so the probe in registry-ca.sh already picks the RedHat dir on a reused
Arch runner. Both scripts now install into both dirs, rebuild only when
one of them changed, and take CA_ROOT so unit tests run them against a
fake root.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 23:45:53 +02:00
kevinveenbirkenbach 1da2cef95e Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
# Conflicts:
#	plugins/lookup/email.py
#	tests/unit/python/roles/web-app-prometheus/test_blackbox_targets.py
2026-10-15 16:45:18 +02:00
kevinveenbirkenbachandClaude Opus 5 1e9936a9b1 fix(tests): point the distro loop test at the function that composes the parent image
efaed0dbb9 dropped pkgmgr_image from utils/distros.py and left this test importing it at line 23 and calling it at line 91, so the module raises ImportError at collection. pytest reports that as '7581 passed, 1 error' while test-unit-python exits 1, which is why the count looks healthy and the target is red.

The test's intent outlived the refactor. INFINITO_PARENT_IMAGE still exists and default.env documents it as composed by utils/env/handlers/infinito/parent_image.py; only the oracle the assertion compared against went away.

base_image(distro, owner, tag) at utils/distros.py:116 is the successor with a compatible signature, and it is what that handler itself calls - parent_image.py:20 is 'return base_image(distro, owner=owner, tag=tag)'. The assertion now checks the value against the function that produces it instead of against a parallel copy that could drift.

Collection goes from ImportError to 11 tests collected. The body shells out through distros.sh for every distro, so it gets its real exercise in the next full gate rather than here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-15 14:57:02 +02:00
kevinveenbirkenbach 6d68498b96 Merge branch 'feature/designer' into feature/store-i18n 2026-10-15 13:29:03 +02:00
kevinveenbirkenbach b1d38f49d6 Merge pull request #934 from infinito-nexus/fix/act/scope-outer-teardown
fix(act): remove only the own worktree's act containers
2026-10-15 10:51:48 +02:00
kevinveenbirkenbach b637742f96 Merge remote-tracking branch 'origin/main' into fix/env/distro-derived-keys/794 2026-10-15 01:07:53 +02:00
kevinveenbirkenbachandClaude Opus 5.5 87c9a5f41b test(act): mark the docker stub log read as outside the file cache
The teardown test reads the call log its docker stub writes into a
temporary directory. test_no_uncached_file_reads flagged that read in
CI; the path-keyed read cache does not fit a file written per run, so
the read keeps Path.read_text with the cache-read opt-out, as the other
stub-log tests do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 01:07:50 +02:00
kevinveenbirkenbachandClaude Opus 5 ac28c0b378 fix(ci): keep the backfill floor on the path the architecture gate takes
The main merge joined two correct rules into one broken script. The
architecture gate republishes the highest version when :latest misses an
architecture, because the published install instructions pull the image
untagged and only a release of that version rewrites latest. The backfill
floor refuses any tag below v9.0.2, which cannot build from its own tree.

The gate returned before the floor was ever consulted, so an incomplete
:latest would have asked the backfill for a tag it cannot build. Both
paths now ask below_floor, which also replaces the comparison the loop
carried inline.

The picker's own test could not have caught it: its docker stub answers
tags without emitting a manifest, and architectures.sh reads an empty
manifest as every architecture missing, so the gate fired on every case
and printed the newest tag. The stub now serves the whole pool, which is
the state the floor cases assume anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-14 14:33:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 46373c1ba3 fix(act): remove only the own worktree's act containers
down_act_outer.sh removed every container named act-*, so starting or
releasing a swarm drill in one worktree killed the act container of a
drill another worktree had running.

workflow.sh now labels act's job containers with
infinito.worktree=${INFINITO_CONTAINER} through the default
ACT_CONTAINER_OPTIONS, and down_act_outer.sh removes only containers
carrying the own label. Both scripts refuse to run without
INFINITO_CONTAINER, so the filter never widens to every act container;
make exports BASH_ENV, which loads it from .env.

Worktrees without a custom.env share the default container name and
therefore still share one label among themselves.

Closes #933

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:50:07 +02:00
kevinveenbirkenbach e6b88662be Merge pull request #919 from infinito-nexus/fix/ci/release-backfill-floor
fix(ci): stop the release backfill below the oldest buildable tag
2026-10-14 10:02:04 +02:00
kevinveenbirkenbachandClaude Opus 5.5 fd9286b24f test(lint): mark the stub call log as written by the test
test_no_uncached_file_reads rejects Path.read_text in tests unless the file is
written in the test itself. The mermaid script test reads the call log its
stubs append to during the run, so that read carries `# nocheck: cache-read`.
The mermaid cli test is formatted with ruff.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:32:35 +02:00
kevinveenbirkenbachandClaude Opus 5.5 ee8d2bd983 test(env): check each distro's parent image against its own reference
The loop test only required three different parent images for three distros,
which a swap between two of them would still pass. Each iteration now also
records the owner and tag it sees, and its image must equal
pkgmgr_image(distro, owner, tag). With the loader of main the test fails on
arch with pkgmgr-debian:stable.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:32:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1d970fe5bd fix(env): keep shellcheck and the cache-read lint green on the distro switch
The loader copies INFINITO_DISTRO into a loader-local variable before it
compares the value with the one in .env. With INFINITO_DISTRO read directly in
that condition, `shellcheck -x scripts/meta/resolve/apps.sh` reported SC2153
for INFINITO_DISTROS, which apps.sh takes from the sourced .env; with the copy
it reports nothing, as on main.

The loader test checks that .env is unchanged with filecmp against the file it
was copied from, instead of reading it twice with Path.read_bytes, which
test_no_uncached_file_reads rejects.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:32:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d412b23409 fix(ci): stop the release backfill below the oldest buildable tag
The release backfill dispatches the highest version tag whose images are
missing. Every tag from v15.0.0 down to v9.0.2 has its images now, so since
2026-10-05 it picks v9.0.1 every day, and each of the nine runs failed in
"Build missing CI images / Image" at
`entry.sh: line 19: infinito: command not found`. The v9.0.1 tree cannot
build its own image; c680be12ea fixed that build in v9.0.2, and a tag cannot
take the fix.

highest_release_tag.sh now stops at v9.0.2: no tag below it is picked, so the
backfill reports nothing missing instead of retrying v9.0.1. v9.0.0 has the
same Dockerfile, Makefile and scripts/docker as v9.0.1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:32:05 +02:00
kevinveenbirkenbachandClaude Opus 5.5 c8cb7fda8c fix(lint): provision the mermaid browser in install-lint, not in the parallel lint run
make lint runs install-lint once and then dispatches its fifteen targets
in parallel. scripts/lint/mermaid.sh provisioned chrome-headless-shell
inside that parallel phase with `npx puppeteer browsers install
--install-deps`, whose `apt-get satisfy` carries no lock timeout. In a
fresh stack container another target held the apt lock, the shared
libraries stayed missing, the script only warned, and all 289 Markdown
files then failed on libnspr4.so (#795).

utils/install/lint/mermaid_cli.py now provisions the browser right after
mmdc, inside the serial install-lint step, as root with --install-deps
and as a user without it, because puppeteer refuses to install libraries
without root. A failed install deletes the chrome-headless-shell cache
folder and retries once under a 900 second timeout per attempt; a cut-off
download otherwise leaves a folder without its executable, and puppeteer
refuses every later install of that build. The install-lint stamp counts
as fresh only while the browser executable exists, so a recreated
container provisions it again.

mermaid.sh no longer calls apt-get or npx. It renders one minimal diagram
first and, when that fails, exits with mmdc's output instead of reporting
every Markdown file.

Verified: tests/unit/python/scripts/lint/test_mermaid.py stubs mmdc,
npx and apt-get. With a browser that cannot load libnspr4.so the script
exits 1 with that line, before any Markdown file renders, and mmdc is the
only tool it calls. Against main's mermaid.sh the same test fails: it
calls apt-get and npx and reports the files one by one. The installer and
stamp cases pass, shellcheck is clean.

Not verified: a full make lint in a recreated stack container. The race
reproduced in one of two runs in the issue.

feature/store-i18n changed the provisioning in mermaid.sh with 2f5f52bd1a
and dc67bce5f0 (timeout, deleting a cut-off browser). Both behaviours
moved into mermaid_cli.py here, so its merge takes this version of
mermaid.sh.

Closes #795

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:31:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1079f48feb fix(env): derive the image and container names from the distro under test
A compose deploy job labelled centos or arch built its stack image from
pkgmgr-debian and ran it as infinito_nexus_debian (#794). distros.sh
exports INFINITO_DISTRO per iteration and sources load.sh again, but the
loader returned early because INFINITO_ENV_LOADED and the .env mtime were
unchanged. A full reload would not have helped either: the debian values
were already exported, so the restore step kept them as caller intent.

load.sh now records the distro it loaded for. Another INFINITO_DISTRO
defeats the early return and drops INFINITO_CONTAINER,
INFINITO_PARENT_IMAGE and INFINITO_IMAGE, the three keys the env handlers
derive from the distro. When the requested distro also differs from the
one .env was generated for, the loader sources a set that
`python -m cli.meta.env --output` generates for it in a private temp
directory and leaves .env untouched, so a local multi-distro run does not
switch the checkout's distro. A caller value set for the loaded distro
still wins.

Verified: tests/unit/python/scripts/meta/env/test_load.py and the new
distros.sh case pass. Against main's loader both fail with the symptom of
the issue: INFINITO_DISTRO=centos next to infinito_nexus_debian and
pkgmgr-debian:stable, and distros.sh handing debian, arch and centos the
same debian parent. shellcheck is clean.

Consequence: CI jobs labelled arch, ubuntu, fedora or centos now build
and deploy that distro, so a failure that shows up there was masked
before.

feature/store-i18n carries 3f973eb137, which resolves the parent image
for scripts/image/push.sh on its own, and 05e4dbafcb, which inserts the
INFINITO_INSTANCE slot check into the block this commit restructures. The
merge keeps the slot check ahead of the distro check.

Closes #794

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:30:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7fa3400cc6 Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
Conflicts resolved by keeping both sides' intent:

- call-test-deploy.yml: main's re-enabled deploy concurrency group, keyed
  on the network axis instead of the removed tor axis.
- sys-svc-proxy: main's shared headers/forwarded.conf.j2 now carries the
  branch's per-vhost semantics (X-Forwarded-Host, TLS port and state read
  from the vhost domain), so html, ws, upload and the Prometheus probes
  all answer per network sibling.
- plugins/lookup/email.py: main's provider-agnostic mail lookup asks the
  network registry instead of testing the .onion suffix, as the branch
  does everywhere; container_extra_hosts.py likewise. The provider host's
  NETWORK_MODE now decides whether it is served onion-first, since a Tor
  node defaults to multi and keeps clearnet canonical.
- workspace common.sh: main's INFINITO_DOMAIN fallback without the
  duplicated no-op line.
- README.md and requirement 035: both sides' rows (Wazuh plus main's
  Pi-hole and Stalwart entries); adapter_candidate count 39.

Adapted to main where the branch's rules now reach main's code:

- six tests from main read the primary domain from default.env instead
  of writing it out, as the hardcoded-domain lint requires;
- the chutney worker-check test passes ONION_SUFFIX from the network
  registry, which the branch's svc-net-tor test script requires;
- web-app-wazuh moves its local http ports to 8096/8097 (8094/8095 now
  belong to web-app-stalwart) and sizes the indexer heap through
  service_memory_mb, the same fixes #414 carries;
- web-app-wazuh's README cosmos follows Stalwart as the mail provider.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:14:42 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e31b9557d2 fix(env): keep the host .env when the runner container runs make setup
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.

compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.

test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.

Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7c7b895ae9 fix(ci): match the privileged run of a fork PR by its number as well
The wait job picks the newest privileged run on the head commit of the
PR. Two fork PRs that share a head repository, branch and commit but
target different bases each start such a run, and the commit alone cannot
tell them apart. A failed run of the other PR would then end the wait of
this one while its images are still being built.

The run name of the workflow carries the PR number, so the match now
requires it next to the head commit. A run without that name matches
nothing, and the wait does what it does whenever no run is found: it
keeps polling for the images until they exist or the wait times out.

Raised by the CodeRabbit review of this PR.

Refs #784

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:07:14 +02:00
kevinveenbirkenbach 8e9fa797fe Merge remote-tracking branch 'origin/main' into fix/ci-fork-image-wait/784 2026-10-05 03:13:43 +02:00