Commit Graph
406 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 b5e299da1c test(install): fold the assertRaises into the patch context (SIM117)
🎯 Fix: 99 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 98
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:54:52 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7e48e7fe2c fix(install): reinstall node_modules when a declared devDependency is missing
install-lint only checked that @playwright/test existed, so node_modules installed before 1c5e5a4fe1 never received @axe-core/playwright and lint-playwright failed with Cannot find module. The guard now compares package.json's devDependencies with node_modules and runs npm ci when one is absent.

🎯 Fix: 93 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:47:07 +00:00
kevinveenbirkenbach fb004b88a0 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/web-app-mattermost/meta/services.yml
2026-11-04 19:43:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c068e44d1b fix(update): treat an incomplete Docker Hub response as unreadable
followed_tags caught OSError, ValueError, KeyError and RuntimeError, but
http.client.IncompleteRead, BadStatusLine and LineTooLong derive from
http.client.HTTPException only. One truncated digest response therefore
escaped to collect_updates (docker.py:344, no try) and aborted the whole
Docker update run instead of printing the follow warning for that image.
Every other fetch site of the updater already catches HTTPException
(docker.py, pip.py).

Reported by CodeRabbit on #1370.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:16:16 +01:00
kevinveenbirkenbachandClaude Opus 5.5 b5543b6d62 fix(update): say when follow cannot read Docker Hub, and retry a 429
Update run 41722395224 proposed no sys-os bump although fedora:latest
points at 45 and a local dry run proposes 44 -> 45. followed_tags turned
every failed digest lookup into "no candidate" without a word, so the run
cannot tell a rate limit from a pin that is current.

tag_digest now retries an HTTP 429 twice with a short backoff, as the tag
listing does, and followed_tags reports on stderr what it compared and
which tags it kept, or the error that made it propose nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:20:00 +01:00
kevinveenbirkenbachandClaude Opus 5.5 01e27bfddb fix(sys-front-inj-design): store and link the role stylesheet per network
The design inject renders a role's style.css once per vhost but copied every
render to one shared CDN path, so the last render won. xwiki's stylesheet
embeds its own origin; on a multi-network node the onion render landed last
and every clearnet page loaded onion URLs, failing network-leak.spec.js
whenever the pass picked clearnet.

The first network keeps style.css, every further network gets
<label>-style.css (onion-style.css), keyed by the same rule the tls lookup
uses for the render (serving_network), and each vhost links its own file.
Single-network roles keep the exact old path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 10:46:58 +01:00
kevinveenbirkenbach 8690590e40 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	cli/build/docs/integration_matrix/__main__.py
#	cli/build/docs/integration_matrix/data.py
#	docs/requirements/027-integration-matrix.md
2026-11-03 22:42:56 +01:00
kevinveenbirkenbachandClaude Opus 5.5 0ed50ae0a5 feat(sys-os): declare the workspace runtime images where the updater finds them
The runtime image pins lived in meta/distros.yml, which neither the docker
updater, the image mirror nor the vulnerability scan reads, so a pin would
only ever move by hand. The new role sys-os carries them in its
meta/services.yml under the distro id; utils/distros.py renders the
workspace matrix from there with the stdlib block reader, and
meta/distros.yml keeps the distro attributes only.

A new release now arrives as an updater pull request. follow: latest holds
debian, ubuntu and fedora to the release their latest tag points at, by
Docker Hub digest: Ubuntu moves between LTS releases only (26.10 shares the
digest of rolling, not latest) and Fedora skips rawhide (46 shares the
digest of rawhide, latest is 45). Arch stays rolling; CentOS stream tags
carry no number the updater can order.

dockerhub_repo and is_dockerhub move to utils/docker/image/ref.py beside
split_registry_and_name, so utils/update/follow.py can use them without an
import cycle and utils/update/docker.py stays under the size cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 19:55:32 +01:00
kevinveenbirkenbachandClaude Opus 5.5 0c54388951 fix(constructor): accept the legacy owners of networks and volumes in the pre-flight
Production hosts carry objects from before the compose project was named after the entity: galaxyserver's mariadb, openldap and postgres networks have no project label, and echoserver's openldap_data volume is labelled svc-db-openldap, the role id older deploys passed as -p. The pre-flight stopped both deploys although compose reuses such networks and volumes with a warning. A network named after its entity now passes without a label or with a declaring role id, still checked against its subnet; a volume passes with a declaring role id as well. A container does not, because compose cannot see a container of another project and fails on its name. A label naming another project still stops the deploy, and the volume message now says compose would hand the role another project's data instead of claiming it refuses the volume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 18:07:15 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a3cb439a68 fix(constructor): stop the disk pre-flight only when a full filesystem also lacks absolute room
The pre-flight stopped every deploy on a filesystem above SIZE_PERCENT_MAXIMUM_DISK_USAGE, which blocked a 1.8 TB workstation at 99 % with 30 GB free while a 145 GB runner was fine. A filesystem now fails only when it is above the percentage AND offers less than SIZE_GB_MINIMUM_DISK_FREE (20 GB): disk_usage_errors reads df's available column, the preflight_disk_usage filter passes the floor, and disk.yml runs df -P -k so the block unit stays KiB even under POSIXLY_CORRECT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 16:10:34 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1ec96181b1 Merge remote-tracking branch 'origin/main' into feature/store-i18n
Conflicts resolved by combining both sides: utils/tests/swarm/run.py keeps store-i18n's re-export of run_watched from watchdog.py, and the #1330 storage roots (lowest_free_mb over / and /var/lib/containerd, the fullest root in the abort banner and the sample line, df over every root) move into watchdog.py; test_run_storage_roots.py now targets watchdog and expects the process-group kill. The seaweedfs 4.52 bump lands in roles/web-svc-seaweedfs, where store-i18n moved the engine; roles/web-app-seaweedfs keeps only the console.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 13:53:44 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a67dd9b372 feat(constructor): stop a deploy in a pre-flight when the host cannot carry it
The constructor now runs tasks/utils/constructor/preflight.yml before the package install and fails with a named remedy when: the running kernel has no module index on disk or the clock is not synchronised (chrony leap status, else timedatectl, outside containers); / or the docker root is fuller than SIZE_PERCENT_MAXIMUM_DISK_USAGE (90); a docker network contradicts the entity that declares it (name, compose project label, subnet outside its block, overlap with another declared subnet); a declared container lacks or a declared volume carries a foreign entity label; a docker default address pool overlaps the mesh, Tor or a declared subnet; a host process outside docker holds a public port of the deploy; or, in assert mode, a clearnet domain of the deploy does not resolve.

A volume without any label passes, since compose adopts it and installs before this change created the Synapse volume with docker run. web-app-discourse declares its launcher-started containers through networks.local.launcher, and web-app-matrix labels a new Synapse volume with its compose project.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 13:38:15 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a394d4bf6b fix(svc-net-wireguard): dial every endpoint at the IPv4 address it resolves to
The mesh is IPv4 only (60-byte header, IPv4 pools), yet wg.conf handed
wg the endpoint name, and wg dials the first getaddrinfo answer, an AAAA
on a dual-stack name. A host with broken IPv6 then never handshook
(alphaserver, 2026-11-01). Resolve every dialled endpoint with getent
ahostsv4 on the dialling host, render that literal, measure the path MTU
against it, and fail before rendering when a name has no IPv4 address.
Also set the tunnel MTU on a live interface, since wg syncconf drops it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 16:53:01 +01:00
kevinveenbirkenbachandClaude Opus 5.5 e810319d00 fix(svc-net-wireguard): require a handshake from the last 180 s in the mesh gate
The gate counted any stamp above zero as handshaken. wg syncconf keeps
the old stamp while it points the endpoint at a dead path, so a deploy
passed the gate over a broken tunnel (alphaserver, 2026-11-01: the
endpoint resolved to an AAAA the host could not reach). Read the host's
epoch in the same shell call as latest-handshakes and accept a stamp
only within WireGuard's REJECT_AFTER_TIME of 180 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 15:58:26 +01:00
kevinveenbirkenbachandClaude Opus 5.5 2d7bf740b0 fix(ci): put containerd, Docker and swap on the runner's empty spare disk
The small GitHub runner shape (AWS, 72G root) ran out of disk in swarm and compose rows (#1329: WordPress#2 of run 41261531965, Nextcloud#6 and #7 of run 41262286962) while an empty instance-store disk nvme1n1 sat unused. mount_spare_disk.sh formats and mounts it at /mnt, but only on a GitHub-hosted runner outside act, and only a whole unpartitioned disk without any signature, mount or swap use of at least 75G; every decline leaves a notice. relocate_docker_dataroot.sh then bind-mounts /var/lib/containerd from the spare disk and moves the Docker data-root to /mnt/docker-host (not /mnt/docker, which is the nested daemon's volume that dedicated.sh wipes). It refuses before stopping anything when images, volumes, build cache, foreign containerd namespaces or shims exist or daemon.json is unreadable, and rolls back the bind mount and daemon.json on any later failure or cancel. Swap goes on / once containerd lives on /mnt, dump_state.sh logs disk size, model and free space per storage root, and the swarm watchdog samples / and /var/lib/containerd every 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 12:31:02 +01:00
kevinveenbirkenbach 7e3c290d51 Merge branch 'feature/stalwart/forward-platform-admin-mail' into feature/store-i18n 2026-11-01 09:42:13 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c1cd4325d6 fix(stalwart): harden the platform admin forward after the skeptic round
The users cache no longer demands a mailbox on every host (a workstation
inventory without mail broke every users lookup) and skips forwards that
are still unrendered Jinja; a non-mapping target names itself. Stalwart now
enforces the mailbox where it consumes the forward. Under SSO 16_forwards
skips ReloadSettings, which would open the OIDC issuer before Keycloak runs;
the OIDC restart applies the limits. The routing probe treats transient
IMAP/SMTP errors as not yet within the same budget and only checks forwards
in Stalwart's own rounds, so the 67 roles that deploy it as mail provider
keep skipping it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 23:55:09 +01:00
kevinveenbirkenbachandClaude Opus 5.5 bbd8ad2764 feat(stalwart): forward the platform administrator's mail to the administrator
System mail now goes to platformadmin@, a mailbox nobody reads. The users
cache already carries a per-user forward and Stalwart already writes a
redirect :copy Sieve script per forwarding account; stalwart_user_forwards
feeds the first into the second for every user Stalwart has an account
for, merged with inventory forwards. platform_administrator forwards to
administrator, keeping its own copy. The users cache now rejects forwards
to unknown users, to users without a mailbox, and forward loops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:43:00 +01:00
kevinveenbirkenbachandClaude Opus 5.5 89063972a6 perf(test-e2e-playwright): order Playwright passes by meta/tests.yml run_after
A role's playwright.run_after names the roles its pass runs after; the new
playwright_order lookup reads it through role_tests (which now renders
Jinja with the queried role as application_id) and orders the round with
a stable topological sort, so only constrained roles move. web-app-docs
runs after every other invokable role: in CI run 41206003651 (Discourse#2)
its pass waited 3126 s for the docs server's Sphinx build while stalwart,
bigbluebutton, jitsi and prometheus (~400 s) queued behind it. Cycles,
self references, unknown roles and non-list values fail loudly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 21:41:55 +01:00
kevinveenbirkenbach 69e0d967a0 Merge branch 'feature/swarm-inventory-extra-vars' into feature/store-i18n
# Conflicts:
#	Makefile
2026-10-31 17:19:25 +01:00
kevinveenbirkenbachandClaude Opus 5.5 50b4fa38b2 perf(ci): skip the local stack image build the deploy jobs never run
On real GitHub Actions INFINITO_BUILD passed through from default.env as 1, so every deploy job built the infinito image locally (~5 min) and then pulled and ran the published ci image anyway (pull_policy always). An env handler now defaults INFINITO_BUILD to 0 there; act and jobs that set it themselves keep building.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:03:37 +01:00
kevinveenbirkenbach 620fafa6d1 feat(swarm): pass INFINITO_INVENTORY_EXTRA_VARS into the swarm-zombie inventory 2026-10-31 16:38:30 +01:00
kevinveenbirkenbach c976f51504 Merge branch 'fix/web-app-weblate/swarm-static-mountpoint' into feature/store-i18n 2026-10-31 13:27:52 +01:00
kevinveenbirkenbach 5cb5acae6d Merge branch 'fix/lint/resource-budget-honours-pins' into feature/store-i18n 2026-10-31 13:27:42 +01:00
kevinveenbirkenbachandClaude Opus 5.5 76fc6992c4 fix(variants): apply a variant pin only while its carrier is in the round
Pins of every role applied at every round index, so a role the round did not deploy could pull a partner into a heavier variant (web-app-discourse#1 measured 35.46 GB against the 32 GB reservation cap). resolve_round now starts every role on its clamped index and recomputes the include and the pins in force until the assignment settles, raising VariantPinError on a cycle. The planner resolves each round once (39.1 s to 16.0 s for all rounds), affected judges every (role, index) with the round the planner deploys for it, and the resource budget lint measures the pin-aware rounds. The stale web-app-matomo pin above the primary-only docs entry is dropped, and test_variant_pin_fixed_point.py guards every planned round against unsettled pins and pins above a disabled entry. docs/contributing/artefact/files/role/variants.md explains the settle loop with diagrams.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 12:45:53 +01:00
kevinveenbirkenbachandClaude Opus 5.5 79e88c5ea4 fix(mirror): refresh a mirrored tag that upstream re-pushed
Fork CI run 41147980556 pgAdmin#0 failed in sys-ctl-hlth-runtime with
"stalwart_webmail: PHP 8.4.26 is below the security floor 8.4.27".
The deploy ran the fork's GHCR mirror of roundcube/roundcubemail:1.7.5-
apache, copied on 10-19 with PHP 8.4.26, while upstream re-pushed the
same tag on 10-27 with PHP 8.4.27. Branch CI runs the sync with
--only-missing, which skipped the tag because it existed ("destination
exists, skipping") without comparing digests, and the full refresh only
runs on the fork's default branch, which still pins 1.7.4.

- utils/docker/registry.py: manifest_digest() reads the
  Docker-Content-Digest of a reference with a HEAD request, which does
  not count against Docker Hub's pull allowance.
- GHCRProvider.is_stale() reports a mirrored tag as stale only when both
  registries answer with a digest and the two differ.
- Under --only-missing an existing tag is re-copied when it is stale; a
  failed re-copy is reported on stderr with the image and keeps the
  existing copy, so one refresh cannot block every deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 09:50:14 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ae40a573c2 fix(web-app-weblate): give the static tmpfs mountpoint its mode on swarm NFS volumes
CI run 40979252521 Weblate#1 (swarm) never converged: every task after
the first died in collectstatic with "PermissionError: [Errno 13]
Permission denied: '/app/cache/static/weblate_fonts'". The tmpfs at
/app/cache/static sits inside the shared weblate_cache volume; the first
task's runc creates the mountpoint as root 0755, and runc gives a tmpfs
on an existing mountpoint that directory's mode. 99a7d28067 declared
mode 1777, which compose honours, but docker stack deploy keeps only the
tmpfs size (docker/cli 29.9.2 convert/volume.go), so swarm still got a
root 0755 tmpfs.

- utils/storage/nfs.py: nested_tmpfs_mountpoints() derives every tmpfs
  mountpoint whose innermost enclosing volume of the same service is
  NFS-backed, with its declared mode; a nested tmpfs without a mode, or
  one mountpoint declared with two modes, raises.
- plugins/lookup/nfs_tmpfs_mountpoints.py feeds it the rendered compose
  volumes, so compose_volumes stays the source of which volumes live on
  NFS.
- sys-svc-compose nfs_prep.yml creates those mountpoints on the NFS
  server with the declared mode before every stack deploy, which also
  repairs a mountpoint an earlier deploy left at 0755.
- web-app-weblate: the cache volume declares nfs uid 1000, gid 0,
  mode 0770; a pre-created mountpoint makes the volume non-empty, so
  Docker no longer copies the image's /app/cache root in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 02:05:59 +01:00
kevinveenbirkenbach 3504f17557 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	.github/workflows/call-test-deploy.yml
#	roles/sys-svc-compose/tasks/main.yml
#	roles/web-app-keycloak/TODO.md
#	utils/docker/registry.py
2026-10-30 23:36:38 +01:00
kevinveenbirkenbach 166da8f8d2 Merge branch 'feature/shared-stack-caches' into feature/store-i18n 2026-10-30 23:31:00 +01:00
kevinveenbirkenbach b17fc884ba Merge branch 'triage/cursor-deb-hold' into feature/store-i18n 2026-10-30 23:30:57 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a803ffbba6 fix(packages): treat a pinned deb as a minimum version
Cursor#0 (CI run 40979252521, debian host) installed the pinned cursor_3.27.6 deb on pass 1; its postinst registers Cursor's apt repo, so pass 2's dist-upgrade moved cursor to 3.27.22, and the plan's ansible.builtin.apt deb=<3.27.6> then failed with 'A later version is already installed': the apt module skips only an equal version and otherwise fails or, with allow_downgrade, downgrades. package_install now asks the target before such a call: dpkg-deb --field reads Package and Version from the checksum-verified deb, dpkg-query reads the fully installed version, and dpkg --compare-versions decides; at or above the pin the call is skipped with changed false, so the deploy-time upgrade may move vendor packages forward and the pin never pulls them back. Any query that cannot be read falls through to the apt install, which decides loudly. The dnf path already skips an older local rpm, so both families now behave alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 23:05:10 +01:00
kevinveenbirkenbach b0bacd072f Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/swarm-smtp-listener 2026-10-30 23:02:44 +01:00
kevinveenbirkenbachandClaude Opus 5.5 eb7056dafd fix(storage): keep the NFS client off the ports the platform publishes
Main CI run 40976399687, socialhome#0 (swarm, tor, centos): the CLI tests
of svc-net-tor (onion:25 "socks CONNECT failed with reply 0x01") and
web-app-stalwart (probe_mail relay: ConnectionRefusedError on port 25)
failed. dockerd on the manager had aborted the whole ingress program of
the mail stack:

  Failed to add ingress: failed to program ingress ports:
  failed to bind host port 0.0.0.0:993/tcp: address already in use

/proc/net/tcp named the squatter: 192.168.244.10:993 -> :2049, the
node's own NFS client. The mount carries no noresvport, so the kernel
drew a source port from the 665-1023 reserved window and landed on
imaps. None of 25/465/993/995/4190/8095 got a DNAT rule or a
docker-proxy listener while every other published port did, and
stalwart still reported healthy because its healthcheck runs inside the
container. 993 and 995 are the only published ports inside that window,
so this is luck of the draw on any NFS-backed swarm host that runs the
mail role.

mount_opts now emits noresvport. insecure on the kernel-flavor export is
the other half: kernel nfsd defaults to secure and answers a mount from
an unprivileged source port with EACCES. It reaches the kernel flavor
only; ganesha already accepts unprivileged clients.

What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state; a host-side job
holding that path at that instant fails loudly. A plain remount may keep
a running mount on its old reserved port until the next real mount.

Port of f46c4d0bab from feature/store-i18n, applied identically so the
later store-i18n -> main merge stays conflict-free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 23:02:41 +01:00
kevinveenbirkenbachandClaude Opus 5.5 334b643911 fix(svc-ai-ollama): let the package-cache probe check /v2/ on registry.ollama.ai
The svc-cache-package CLI probe GETs every declared route path through the frontend and fails on 502/503/504. registry.ollama.ai declared no paths, so the probe asked for /, which upstream now answers with 503; Nexus relays an upstream 5xx as its own 503, so package#0 (CI run 40979252521) failed with 'registry.ollama.ai/ on port 443 answered 503'. The host now declares /v2/, the registry API root every ollama pull goes through (manifests and blobs); upstream answers it with 404, which Nexus relays as 404 and the probe accepts. On a TLS host the frontend still proxies the whole host through location /, because upstreams.conf.j2 reads routes only for plain-HTTP listeners: the rendered upstreams.conf is byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:29:35 +01:00
kevinveenbirkenbachandClaude Opus 5.5 74d99201e0 feat(cache): share package and tool caches across stacks and release swarm clusters after rescue
Parallel compose slots, swarm lab clusters and GUI lanes each kept their own pip, npm and pacman downloads, and a failed swarm run left its cluster (about 25 GB of node docker data plus the rescue bundle in the act container) on the root filesystem until someone ran make swarm-down. INFINITO_CACHE_SHARED_HOST_PATH (default <DIR_CACHE>/shared) is now mounted read-write at the same path into every compose slot and every swarm lab node, with PIP_CACHE_DIR and npm_config_cache below it; empty, the renders stay byte-identical to before. On a host whose os-release ID is exactly arch running an arch stack, INFINITO_CACHE_HOST_PACMAN_PATH mounts the host pacman cache read-only and scripts/docker/cache/pacman.sh makes it the first CacheDir with <shared>/pacman/ second, so new downloads land in the shared folder and the host cache stays untouched; on Manjaro and every other host pacman uses the shared folder alone.

make swarm-zombie without keep=true now copies the act job's rescue diagnostics and Playwright artefacts to the host and then releases the lab nodes, the lab network, the NFS export volume and the act outer container, on success and on failure; when the copy fails the nodes still go, the act container stays and the run exits naming make swarm-down. make roundtrip passes keep to every swarm step, make clean-cache removes the shared folder, and the disc-space cleanup skips pacman cache directories it cannot write. apt and dnf keep their per-container caches behind the package-cache proxy, and no docker data root is shared between daemons. Requirement 065.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:04:38 +01:00
kevinveenbirkenbach f27d8369d7 Merge branch 'main' of github.com:infinito-nexus/core into integration/store-i18n-merge
# Conflicts:
#	roles/web-app-akaunting/TODO.md
#	roles/web-app-funkwhale/meta/services.yml
#	roles/web-app-openproject/tasks/01_manager_ops.yml
2026-10-30 19:37:15 +01:00
kevinveenbirkenbach bad992a537 Merge remote-tracking branch 'origin/main' into feature/sys-ctl-hlth-runtime/vulnerable-runtimes/1127 2026-10-30 18:47:55 +01:00
kevinveenbirkenbachandClaude Opus 5.5 b894db0a75 fix(svc-bkp-volume-2-local): restore into the resolved volume mountpoint
rsync 3.5.0 (Debian security update 3.5.0+ds1-0+deb13u1) refuses a
destination symlink owned by neither root nor the calling user.
svc-opt-ssd-hdd turns a volume's `_data` into a symlink to /mnt/hdd or
/mnt/ssd. The restore drill empties the volume and runs `compose create`;
docker repopulates it and lchowns the `_data` symlink to the image
directory's owner (e.g. www-data). The restore then failed with
"refusing to follow a symlink owned by an untrusted user" and
"change_dir#1 .../_data/ failed: Too many levels of symbolic links",
the drill left its subjects down, and the dependent checks timed out
(CI run 40979252521, ssd-hdd#0, on feature/store-i18n; recover.py is
identical on main).

VolumeRecovery now resolves the mountpoint before handing it to rsync:
os.path.realpath locally, `ssh <host> realpath -e -- <mountpoint>` for
--docker-host, so rsync writes into the real directory and never meets
the symlink. `--insecure-links` was not used because older rsync builds
on other distributions lack it. Two new unit tests (local symlinked
mountpoint, remote resolution); both fail against the previous code.

Root cause and fix by the i18n session (feature/store-i18n triage of
ssd-hdd#0), ported to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:51:44 +01:00
kevinveenbirkenbachandClaude Opus 5.5 024259e89d feat(design): recolour single-colour logos per slot and preview every app's assets
A logo whose solid pixels share one colour (alpha >= 128, every channel within 24) is detected once per file and tinted to the text colour of each slot: frame_fill on frame slots, fill elsewhere, plus a prefers-color-scheme: dark variant in the SVG of page slots and the page text colour for the favicon. Edge pixels below half opacity are ignored so antialiasing does not count as a second colour. Multi-colour logos render byte-identical to before.

make design-logos (alias i8delo) renders every role with a design entry from one logo and base colour into an HTML page with day and night tiles, without a deploy. Slots render in four threads and roles in one process per CPU: the full preview went from 287.6s to 33.2s with identical output (diff -rq over 1229 PNGs). A GPU does not pay off for these PIL resizes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:06:30 +01:00
kevinveenbirkenbachandClaude Opus 5.5 284b0873dd fix(sys-ctl-hlth-runtime): harden LAM CLI, catch apk options, drop vars comment
Addresses three CodeRabbit findings on PR #1265.

1. LAM hardening invisible to the probe (thread 4413764929): the runtime
   check probes PHP with `php -r`, the CLI SAPI. web-app-lam mounted the
   hardening ini only into /etc/php/8.4/apache2/conf.d, which the Debian
   CLI does not scan, so the probe read unhardened CLI settings and the
   check failed for LAM. The php_hardening entry now also mounts the ini
   into /etc/php/8.4/cli/conf.d. No other role mounts it into a Debian
   per-SAPI directory; the rest use the shared scan dir of the official
   image or of Alpine. The TODO entry now names the remaining gap: the
   probe reads only the CLI copy.

2. Explanatory comment above HEALTH_RUNTIME_FLOORS (thread 4413764949):
   removed. The README already states the end-of-life rule and now also
   defines a floor as the lowest patched release of each line.

3. `apk --no-cache add` missed (thread 4415018662): PACKAGE_INSTALL_PATTERN
   required `add` right after `apk`, so `RUN apk --no-cache add php7`
   passed as an image without PHP. The apk branch now accepts options
   before `add`, like the apt branch. New tests cover the option form and
   guard that `apk` in a path or a comment is no package install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 08:57:21 +01:00
kevinveenbirkenbachandClaude Opus 5.5 44ff41c875 style: rewrap three test lines past ruff's line length
make autoformat output for three lines the feature/updater merge brought
in over 88 characters; matches feature/updater-lockdown-probe 88d138ba1c.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:48:41 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5e8890d208 Merge branch 'feature/updater' into feature/store-i18n
Brings in feature/updater accb22da75, which carries main 5995766f58.

Conflict resolutions:
- web-app-fider: main's tenant bootstrap (01_tenant.yml) runs first on
  the stack host, OIDC setup moves to 02_setup_oidc.yml, design to
  03_design.yml; flat includes so main's gate test reads them.
- sys-front-tls test_nginx_ssl_header: the tls fake maps the domain term
  store-i18n passes back to its app.
- group_vars 04_webserver.yml, sys-svc-proxy CSP header: store-i18n's
  webserver_resource lookup plus #1256's WEBSERVER_SECURITY_HEADERS and
  the security_headers.lua include after the CSP block.
- web-app-nextcloud: updater's decoupled Talk stack (Janus, signaling,
  NATS, recording builds) replaces aio-talk; store-i18n's per-network
  Talk backends move from the wrapper script into signaling.conf.j2 urls
  (NEXTCLOUD_HPB_SIGNALING_BACKEND_URLS from network_urls, canonical
  first); IS_COMPOSE_MODE/IS_SWARM_MODE booleans kept; the integration
  OpenProject digest stays delegated to STACK_HOST.
- users lookup docs, token-store write_many, cache README, meta info
  lint: both sides combined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:40:05 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ded61d53ee fix(cache): leave tracked files deleted in the worktree out of repository_files
repository_files() listed every path `git ls-files` returns, including tracked files the worktree has already deleted. Every content scan that reads the listing then reported "[Errno 2] No such file or directory" as a finding: in a worktree that removed roles/web-app-nextcloud/filter_plugins/nextcloud_janus.py and its test but had not committed the removal yet, test_no_rsync_backup_flag failed on exactly those two paths and turned the whole lint target red. No scan can read such a file until the deletion is committed.

The listing now subtracts `git ls-files --deleted`. A clean checkout, as in CI, has no deleted entries, so the listing there is unchanged. The rsync lint is green again on the same worktree. The unit tests stub the extra git call and cover a file deleted in the worktree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 22:55:45 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c855efc30a refactor(web-app-dashboard): hide a tile through services.dashboard.enabled instead of display
services.dashboard.enabled already says whether a role takes part in the dashboard: roles_with_service lists only roles with enabled and shared as tile-reachability targets, the CSP filter adds the dashboard to frame-ancestors only when it is set, and docker_cards uses it as the card's iframe flag. A separate display key duplicated that switch and left a role with enabled false still rendering a card. docker_cards now skips the card when services.dashboard.enabled is false (a role without a dashboard binding keeps its card, and a value that is not a boolean fails loudly), so display, its validator exception and its roles_with_service filter go again; base.md documents enabled as the per-host switch.

An inventory hides a tile with applications.<role>.services.dashboard.enabled: false, which also drops the dashboard framing of that role. Unit tests cover false, true, a missing binding, a rendered string and a non-boolean; forcing the helper to always show the card fails four of them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 19:00:46 +01:00
kevinveenbirkenbachandClaude Opus 5.5 3ef937a9d8 feat(web-app-dashboard): move the tile switch to services.dashboard.display
The dashboard read `display` from the role's meta/info.yml, which no inventory can override, so a host could not hide a tile. docker_cards now skips a card when `services.dashboard.display` in the merged applications is false, and absent still means shown; a non-bool value fails loudly. roles_with_service drops the same role from the dashboard tile-reachability targets so the Playwright spec does not wait for a hidden tile, and the inventory validator accepts `services.dashboard.display` for every role that declares `services.dashboard`, since no role default carries it. `display` leaves the meta/info.yml allowlist, web-app-jellyfin drops its redundant `display: true`, and base.md and layout.md document the new switch.

Real lookups with web-app-gitea set to display false render the Dashboard and Jellyfin cards and only jellyfin as a tile target, and all three without the override; the nine tests for the new behaviour fail against the old code.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 17:46:03 +01:00
kevinveenbirkenbachandClaude Opus 5.5 2c6c7e1464 fix(web-app-nextcloud): trust the real reverse-proxy hop so Nextcloud sees client IPs
trusted_proxies held the host's public addresses, but OpenResty reaches Nextcloud through the published port, which Docker forwards from the gateway of one of the container's networks (192.168.201.1, the central MariaDB network, on production). Nextcloud therefore ignored X-Forwarded-For, attributed every request to that gateway and brute-force throttled it. The TRUSTED_PROXIES env var was joined with commas while the image's reverse-proxy.config.php splits it on spaces, which produced the single malformed entry the admin overview rejects, and the occ value reached config:system:set as a Python repr instead of JSON, so config.php got null.

Add the proxy_hops lookup next to resolve_upstream in utils/networks/proxy.py: under compose it returns the gateway of every network the application joins, under swarm the subnets of the overlays it shares with svc-prx-openresty, both derived from the service registry and role_subnets. NEXTCLOUD_TRUSTED_PROXIES now uses it, env.j2 joins the list with spaces and system.yml passes it through to_json.

Rendered with the real roles and group_vars for the galaxyserver layout: compose trusts 192.168.41.1, 192.168.201.1 and 192.168.202.1, swarm 192.168.107.0/24; unit and integration tests cover both modes, and two mutations of proxy_hops fail them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 17:07:57 +01:00
kevinveenbirkenbachandClaude Opus 5.5 96ebefb9db perf(spot): re-render only the users SPOT after a token store write
A token store write re-rendered the whole applications SPOT: merged_applications_cache_key carried the token store digest, so the applications_carrier park in sys-token-store/tasks/write_many.yml missed its key and rendered all 269 applications again (stalwart compose roundtrip 18.3 s and 10.2 s per pass, swarm pass 1 12.2 s and 12.6 s).

Rendered from the real roles tree under an empty, a realistic and an exhaustive token store (298 users x 269 applications), the applications tree differs only under <app>.users.<key>.tokens (28783 leaves in 46 applications, 0 elsewhere); the 69 password leaves that also differ change between two renders of the same store. No code reads a token through the applications tree: every reader goes through lookup('users'), lookup('config', <app>, 'users.<key>...') (served from get_merged_users), lookup('email') or the sys_token_store_users fact, and the users carrier keys on the store.

The token digest therefore moves from merged_applications_cache_key into merged_users_cache_key, whose value stays the same. A store write keeps the applications carrier and the write_many park re-renders the users alone; the task is renamed to match. applications.md now says that user records inside the payload can predate every store write and that application config must not derive a value from a token.

Real ansible-playbook harness on 2.20.6 with the real roles, group_vars and write_many.yml: the re-park after a first mint and after a rotation drops from 4.0 s to 0.55 s on one host (applications renders 1 -> 0), and every token probe (users, config users path, email password, stalwart_users) equals the old code in compose and swarm mode. Dropping the digest from the users key as well leaves those probes stale, which the harness and two unit tests catch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 15:56:08 +01:00
kevinveenbirkenbachandClaude Opus 5.5 91f29d332e perf(spot): carry the rendered users next to the applications
Ansible forks a fresh worker per task, so every lookup('users') in a forked worker re-rendered the whole user catalogue (298 users, 0.3 to 0.5 s locally and about 1 s on the runner, 1 s locally once a users password is vaulted): _MERGED_USERS_CACHE lives in the process and had no carrier fact.

applications_carrier now also parks lookup('users', carrier=True) in _INFINITO_USERS_RENDERED at the same park points, and clear drops both facts. get_merged_users serves the fact to templated reads when its key matches: the applications carrier key (roles dir, applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN, group_names, token store digest) plus inventory_hostname, so a carrier a run_once park broadcasts never answers another host with its become password. A park whose users carrier still matches leaves it in place instead of returning 119 KB again, so parks cost what they cost before.

Real ansible-playbook harness on 2.20.6 and 2.21.5 with the real roles tree: users renders per users-reading task after a park 1 -> 0, task 0.56 -> 0.08 s (vaulted 1.21 -> 0.17 s); a rotated token, a rotated inventory password, a cleared carrier and another host's carrier all re-render; 32 of 32 probe digests equal HEAD's. Removing the carrier check fails three unit tests. The stalwart swarm roundtrip pass 1 and its DR drill ran green with it on all nodes, with no 401 from JMAP, msmtp or LDAP.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 15:53:34 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d116dde659 fix(sys-ctl-hlth-runtime): bound the engine calls and catch hidden PHP runtimes
Three CodeRabbit findings on PR #1265:

- script.py listed the running containers with `container ps` and
  `container inspect` without a timeout, so a hung engine kept the daily
  unit active forever and blocked the next timer run. Both calls now use
  EXEC_TIMEOUT_SECONDS; main turns a timeout or engine error into a
  one-line failure and exit 1 instead of a traceback.

- runtime.py's PHP_PACKAGE_PATTERN requires a digit after "php", so an
  image installing an unversioned interpreter package (`php-cli`) and
  setting no PHP_VERSION was reported as PHP-free. The interpreter
  packages php, php-cli, php-fpm, php-cgi, php-phpdbg and
  libapache2-mod-php are now recorded as PhpRuntime.packages and reported
  with an unknown patch level; extension and library packages
  (php-mysql, php-pear, php-common, phpmyadmin) and paths do not count.

- php_finding returned early for a patched PHP_VERSION, so an end-of-life
  package line installed next to it (e.g. php7) was never reported. The
  version only returns a finding now; otherwise the lines are judged too.

Every finding has unit tests that fail on the previous code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 15:32:57 +01:00
kevinveenbirkenbachandClaude Opus 5.5 21e05b9d42 feat(env): switch the GPU-only tests with INFINITO_TEST_GPU
The two tests that only answer on a GPU host, the logout catalogue translation check and the complete-catalogs integration test, were gated on accelerated() alone: a contributor with an NVIDIA runtime had no way to keep them out of a local gate while the GPU translation run is paused, so every gate on such a host stayed red on them.

INFINITO_TEST_GPU is a new env key. Its handler derives the default from INFINITO_GPU_COUNT: true where the compose services reserve a GPU, false for 0 or none. custom.env is applied after every handler, so `INFINITO_TEST_GPU=false` there switches the tests off on a GPU host; an exported value wins as well. tests/utils/gpu.py reads the flag for both tests, fails on any value other than true or false, and falls back to the same GPU detection only where no .env is loaded, such as a direct pytest run.

The handler joins gpu_count in utils/env/handlers/infinito/gpu/ (count.py, only_tests.py), because two handlers sharing the gpu prefix are a folder by the prefix-grouping lint. The new module avoids a test* name so unittest discovery never collects it. customize.md lists the key among the common overrides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 11:50:26 +01:00