Commit Graph
1228 Commits
Author SHA1 Message Date
kevinveenbirkenbach 166da8f8d2 Merge branch 'feature/shared-stack-caches' into feature/store-i18n 2026-10-30 23:31:00 +01:00
kevinveenbirkenbach b17fc884ba Merge branch 'triage/cursor-deb-hold' into feature/store-i18n 2026-10-30 23:30:57 +01:00
kevinveenbirkenbach 7d8feb6386 Merge branch 'fix/svc-ai-ollama/cache-probe-path' into feature/store-i18n 2026-10-30 23:30:50 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a803ffbba6 fix(packages): treat a pinned deb as a minimum version
Cursor#0 (CI run 40979252521, debian host) installed the pinned cursor_3.27.6 deb on pass 1; its postinst registers Cursor's apt repo, so pass 2's dist-upgrade moved cursor to 3.27.22, and the plan's ansible.builtin.apt deb=<3.27.6> then failed with 'A later version is already installed': the apt module skips only an equal version and otherwise fails or, with allow_downgrade, downgrades. package_install now asks the target before such a call: dpkg-deb --field reads Package and Version from the checksum-verified deb, dpkg-query reads the fully installed version, and dpkg --compare-versions decides; at or above the pin the call is skipped with changed false, so the deploy-time upgrade may move vendor packages forward and the pin never pulls them back. Any query that cannot be read falls through to the apt install, which decides loudly. The dnf path already skips an older local rpm, so both families now behave alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 23:05:10 +01:00
kevinveenbirkenbachandClaude Opus 5.5 34506554f7 fix(cache): download pacman packages per node and publish them into the shared cache
The shared pacman cache was the first writable CacheDir on every arch runner and lab node. Our arch image sets DisableSandbox, so libalpm downloads straight into that directory (util.c _alpm_download_dir_setup only makes a per-process temp dir with the sandbox on), and the XferCommand writes <pkg>.part there with curl -C -o %o; two nodes fetching the same package appended to the same .part file, and the second rename failed or left a corrupt package in the shared cache.

pacman.sh now gives each node its own download directory <shared>/pacman/node-XXXXXX/ (created once with mktemp, reused from pacman.conf on later runs) as the CacheDir between the read-only host layer and <shared>/pacman/, and writes a PostTransaction hook to pacman.d/hooks/infinito-shared-cache.hook that renames every finished package and signature into <shared>/pacman/; .part files stay for the next resume. Both directories sit in one bind mount, so each publish is an atomic rename(2). pacman.sh fails when pacman.conf sets HookDir, since pacman would then skip the hook silently. Requirement 065 and cache.md describe the three layers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:43:01 +01:00
kevinveenbirkenbachandClaude Opus 5.5 334b643911 fix(svc-ai-ollama): let the package-cache probe check /v2/ on registry.ollama.ai
The svc-cache-package CLI probe GETs every declared route path through the frontend and fails on 502/503/504. registry.ollama.ai declared no paths, so the probe asked for /, which upstream now answers with 503; Nexus relays an upstream 5xx as its own 503, so package#0 (CI run 40979252521) failed with 'registry.ollama.ai/ on port 443 answered 503'. The host now declares /v2/, the registry API root every ollama pull goes through (manifests and blobs); upstream answers it with 404, which Nexus relays as 404 and the probe accepts. On a TLS host the frontend still proxies the whole host through location /, because upstreams.conf.j2 reads routes only for plain-HTTP listeners: the rendered upstreams.conf is byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:29:35 +01:00
kevinveenbirkenbach 80814daf7c Merge branch 'triage/objstore-onion-assets' into integration/store-i18n-merge 2026-10-30 22:23:38 +01:00
kevinveenbirkenbach b75f55da0c Merge branch 'feature/logout-sibling-handoff' into integration/store-i18n-merge 2026-10-30 22:23:35 +01:00
kevinveenbirkenbachandClaude Opus 5.5 637acbaf2a test(web-svc-logout): parse the rendered compose file through load_yaml_str
test_compose_siblings.py called yaml.safe_load on the rendered compose file, which TestNoDirectYamlCalls rejects in favour of the utils.cache.yaml helpers (quality-high run 6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:23:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 22614627d5 test(objstore): read web-app-wordpress meta through ROLE_FILE_META_SERVICES
test_s3_public_url.py built roles/web-app-wordpress/meta/services.yml with a pathlib chain, which TestRoleFileSpot rejects in favour of the utils.roles.mapping constant (quality-high run 6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:19:46 +01:00
kevinveenbirkenbachandClaude Opus 5.5 74d99201e0 feat(cache): share package and tool caches across stacks and release swarm clusters after rescue
Parallel compose slots, swarm lab clusters and GUI lanes each kept their own pip, npm and pacman downloads, and a failed swarm run left its cluster (about 25 GB of node docker data plus the rescue bundle in the act container) on the root filesystem until someone ran make swarm-down. INFINITO_CACHE_SHARED_HOST_PATH (default <DIR_CACHE>/shared) is now mounted read-write at the same path into every compose slot and every swarm lab node, with PIP_CACHE_DIR and npm_config_cache below it; empty, the renders stay byte-identical to before. On a host whose os-release ID is exactly arch running an arch stack, INFINITO_CACHE_HOST_PACMAN_PATH mounts the host pacman cache read-only and scripts/docker/cache/pacman.sh makes it the first CacheDir with <shared>/pacman/ second, so new downloads land in the shared folder and the host cache stays untouched; on Manjaro and every other host pacman uses the shared folder alone.

make swarm-zombie without keep=true now copies the act job's rescue diagnostics and Playwright artefacts to the host and then releases the lab nodes, the lab network, the NFS export volume and the act outer container, on success and on failure; when the copy fails the nodes still go, the act container stays and the run exits naming make swarm-down. make roundtrip passes keep to every swarm step, make clean-cache removes the shared folder, and the disc-space cleanup skips pacman cache directories it cannot write. apt and dnf keep their per-container caches behind the package-cache proxy, and no docker data root is shared between daemons. Requirement 065.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 22:04:38 +01:00
kevinveenbirkenbach 2df447821d Merge branch 'triage/objstore-onion-assets' into integration/store-i18n-merge 2026-10-30 21:57:50 +01:00
kevinveenbirkenbach 227c861e34 Merge branch 'feature/logout-sibling-handoff' into integration/store-i18n-merge 2026-10-30 21:57:48 +01:00
kevinveenbirkenbachandClaude Opus 5.5 f287322065 fix(objstore): serve public bucket files same-origin on every vhost of the consumer
WordPress put objstore.public_url, the clearnet api.seaweedfs.s3 host, into S3_UPLOADS_BUCKET_URL, so the onion pass of WordPress#0 (CI run 40979252521) loaded the site logo from clearnet and the leak spec failed; the /objstore rebase only fired on onion-only consumers and never on a multi-network node. lookup('objstore', app, 'relative_url') now returns /_infinito/objstore/<bucket> for a shared consumer with a public bucket and an API domain, and locations.conf.j2 serves that path on every vhost of the consumer, clearnet and onion alike, by proxying over the loopback to the provider's own vhost.

The route passes GET and HEAD only, answers 404 for a path ending in / (no ListObjects), drops the query string (no S3 subresources or presigned queries) and the Authorization and Cookie headers, so only the public bucket's anonymous read applies, and adds nosniff plus a sandbox CSP; private and embedded buckets get no route. Only web-app-wordpress switches to relative_url: mastodon, pixelfed, baserow, bookwyrm, snipe-it, listmonk, mediawiki, shopware and peertube need absolute URLs for federation, mail or server-side rendering, and decidim and funkwhale sign URLs, which the route deliberately refuses. The prefix moves from /objstore to /_infinito/objstore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 21:52:35 +01:00
kevinveenbirkenbachandClaude Opus 5.5 e0179a0893 feat(logout): hand the browser on to the conductor of the next network
A node serving several networks runs one universal-logout conductor per network, and a page served over HTTPS cannot fetch the HTTP onion hosts, so one conductor cannot clear the other network's sessions. compose.yml.j2 now gives every conductor the url.origin of the other networks' conductors in LOGOUT_SIBLINGS (empty on a single-network node, which disables the hand-off), and the Keycloak logout panel that frames the conductor follows its handoff report to the next conductor once the sweep is done, accepting only an http or https URL.

The pinned image v1.4.2 ignores LOGOUT_SIBLINGS and never reports handoff, so behaviour changes only with the bump to the universal-logout release that carries the reachability probe. test_compose_siblings.py renders single- and multi-network nodes; logout-panel.test.js covers the handoff branch and rejects non-web URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 21:46:00 +01:00
kevinveenbirkenbach f6779b9f8d Merge branch 'fix/web-app-nextcloud/swarm-local-scratch-volume' into integration/store-i18n-merge 2026-10-30 21:39:25 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a23500a270 fix(compose-volumes): keep the meta nfs flag on an injected volume of the same key
compose_volumes merged extra_volumes first and then skipped every meta/volumes.yml entry whose key was already present, so the meta nfs value was lost with it. Nextcloud injects talk_recording_tmp through nextcloud_extra_volumes without an nfs key, and 5728502a19 marked that volume nfs: false. On swarm with the NFS backend the stack therefore bound it to /var/lib/infinito/nextcloud_talk_recording_tmp, while nfs_prep.yml renders the same lookup without extras, saw nfs: false and never created the subdir; every node rejected the task with 'no such file or directory'. The meta nfs value now carries over onto the injected or derived spec unless that spec sets nfs itself, so both renders agree and the scratch volume stays node-local.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 21:31:59 +01:00
kevinveenbirkenbach e22cffe201 Merge branch 'triage/docs-cli-disc-space' into integration/store-i18n-merge 2026-10-30 21:09:51 +01:00
kevinveenbirkenbach 8b42651c5a Merge branch 'fix/web-app-erpnext/server-scripts-script-manager' into integration/store-i18n-merge 2026-10-30 21:09:47 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c68b9833bf fix(web-app-erpnext): grant the API bot Script Manager when server scripts are on
Frappe v16 limits the Server Script doctype to the Script Manager role and ServerScript.validate calls frappe.only_for('Script Manager'), so services.erpnext.server_scripts: true still left every integration on erpnext-api-bot with 403 Insufficient Permission for Server Script. The same switch now grants the bot Script Manager when true and revokes it when false; System Manager stays, the save runs only on a change, and an undeclared value fails before the site is touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 21:07:26 +01:00
kevinveenbirkenbachandClaude Opus 5.5 4e71a6c724 fix(merge): align main's swarm seeder wait and worker-limit test with store-i18n guards
Main's OpenProject seeder wait (#1302) compared DEPLOYMENT_MODE == 'swarm', which store-i18n's test_deployment_mode_booleans forbids; it now uses IS_SWARM_MODE. Main's test_webserver_worker_limits (#1297) rendered without host facts, so store-i18n's resource lookup refused to measure the controller; the fixture now carries processor_vcpus and memtotal_mb as every play does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:47:07 +01:00
kevinveenbirkenbachandClaude Opus 5.5 4f725d6fa9 fix(web-app-docs): replay the guide under the stack host's disc-space limit
The CLI test replays the published Production block in a throwaway target that shares the runner's disk. The replayed deploy knew only the default SIZE_PERCENT_CLEANUP_DISC_SPACE of 85 while the stack host runs with its own inventory value, so sys-ctl-hlth-disc-space failed in the target at 87% (Wazuh#1, CI run 40979252521). disc_space.sh writes the stack host's value into the block's provision --vars and fails on a non-integer value or a block without --vars; the published instructions stay unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:34:27 +01:00
kevinveenbirkenbach f27d8369d7 Merge branch 'main' of github.com:infinito-nexus/core into integration/store-i18n-merge
# Conflicts:
#	roles/web-app-akaunting/TODO.md
#	roles/web-app-funkwhale/meta/services.yml
#	roles/web-app-openproject/tasks/01_manager_ops.yml
2026-10-30 19:37:15 +01:00
kevinveenbirkenbach 5e39b946c9 Merge branch 'fix/sys-svc-compose/pull-test-cold-start' into integration/store-i18n-merge 2026-10-30 19:33:56 +01:00
kevinveenbirkenbach 9c7b615743 Merge branch 'triage/wordpress1' into integration/store-i18n-merge 2026-10-30 19:33:38 +01:00
kevinveenbirkenbach a785f6d956 Merge branch 'triage/wordpress0-editor-canvas' into integration/store-i18n-merge 2026-10-30 19:33:32 +01:00
kevinveenbirkenbach c5a0e5445e Merge branch 'feature/037-onion-network-aligned-config' into integration/store-i18n-merge 2026-10-30 19:33:27 +01:00
kevinveenbirkenbachandClaude Opus 5.5 b5736bda44 feat(web-app-wordpress): serve each multisite site on its own network sibling
CI run 40979252521 WordPress#1 (multisite, multi network): a request on a network sibling resolved to the clearnet site, because WordPress picks the site by Host before any mu-plugin runs and the network kept only clearnet base URLs per site. Operator decision: a real multisite per network with real child domains (option B).

infinito-network-origins.php is now also installed as wp-content/sunrise.php (SUNRISE=true), so the sibling host resolves to its own site before mu-plugins load; a guard constant stops the second load from mu-plugins. WORDPRESS_SITE_DOMAINS holds the canonical domains in the first domain's network, WORDPRESS_NETWORK_BASE_URLS carries every sibling of each site's base URL (sites ';', siblings ','), and INFINITO_NETWORK_BASE_URLS takes that list. Child sites are inserted by files/php/multisite/sites.php only where no site exists on the domain and path, replacing wp site list plus wp site create. The multisite variant's child domains move from server.domains to domains.canonical.

Covered by 29 Python tests and two PHPUnit suites; PHPUnit and a variant 1 deploy with an onion pass still have to run (store-i18n gate and deploy).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:49:33 +01:00
kevinveenbirkenbachandClaude Opus 5.5 7b6800493b test(sys-svc-compose): give the streamed-pull test a silence window above a loaded cold start
test_run_streamed_outlives_a_deadline_while_it_keeps_talking patched RUN_SILENCE_SECONDS to 1.0 while the chatty child prints its first line only after the interpreter has started. run_streamed measures silence from process start, which is right for production (a command that never speaks must time out), so under xdist load on a full disk the cold start alone exceeded the window and the run returned 124 (reported from IAM's make quality).

The window is now 2.0 s against a 0.1 s cadence over 40 lines (4 s, still twice the window, so the test keeps proving it outlives a deadline while talking), and the child runs with -I -S to skip the site import. With a simulated 1.5 s cold start the old parameters return 124 and the new ones 0. pull.py is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:43:39 +01:00
kevinveenbirkenbachandClaude Opus 5.5 f12d328fee fix(sys-front-tls): strip Secure from upstream cookies on every plain-HTTP vhost
On an onion vhost the browser talks plain HTTP, so an upstream that marks its session cookie Secure (the app still believes in its canonical https origin) loses that cookie and the login loops. n8n carried its own proxy_cookie_flags n8n-auth nosecure for this; every other app on the onion pass was exposed to the same failure.

The plain-HTTP branch of nginx_ssl_header.j2 now emits proxy_cookie_flags ~ nosecure, which matches every cookie; TLS vhosts are unchanged. n8n's per-cookie line goes. Taiga and Semaphore keep their tested location-level flags, which replace the server rule for their location, and network_modes.md says so. test_nginx_ssl_header.py asserts the rule on a plain-HTTP vhost and its absence on a TLS one; without the template line the plain-HTTP case fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:10:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 55604ca19c fix(web-app-wordpress): load the editor canvas design tokens from the page's own origin
CI run 40979252521 WordPress#0 (swarm, variant 0, onion pick): the block editor canvas stayed transparent (rgba(0, 0, 0, 0) instead of --design-surface-1). The trace shows the canvas iframe requesting https://cdn.main.infinito.test/_shared/css/default.css and the onion page's CSP (style-src-elem 'self' http://cdn.<onion> ...) blocking it; the clearnet pass of the same spec was green. design.php writes WORDPRESS_DESIGN_CONFIG.tokens into the infinito_design option once for every network sibling, so an absolute CDN host pins the onion editor to clearnet.

The token sheet now uses the cdn lookup's relative_urls, rendering /_infinito/cdn/_shared/css/default.css. WordPress prefixes a relative style source with site_url(), which infinito-network-origins derives from the request host, and every generic vhost serves /_infinito/cdn/ (basic.conf.j2 includes cdn.conf.j2), so the canvas loads from the page's origin on both networks. Same approach as the Joomla TinyMCE fix 175e00218f. test_design_tokens.py renders the tokens through the real cdn lookup and fails on the old URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:04:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 41ea801fd9 feat(network): point logout and peer allowlists at every network a vhost serves
CI run 40979252521 Nextcloud#0: on the onion pass the biber OIDC logout cleared only the clearnet session, because Keycloak holds one front-channel URL per client and the injected logout sent no post-logout target for the onion network; and Collabora's aliasgroup1 listed only the clearnet WOPI host.

On a vhost outside the application's canonical network (and without an external issuer) the injected logout one-liner now adds client_id and post_logout_redirect_uri set to that network's logout conductor, and Keycloak registers every conductor sibling as post-logout redirect URI next to '+'. Collabora's aliasgroup1 lists the Nextcloud WOPI host of every network (one instance, union allowlist). network_modes.md gains a Network-Aligned Values table, and test_network_allowlist_sinks.py fails when a template names an allowlist sink without the sibling union.

Rendering is covered by 14 tests; the full gate and the runtime acceptance of the '##' post-logout list by Keycloak and the comma aliasgroup by Collabora still need a gate on store-i18n and an onion deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:00:24 +01:00
kevinveenbirkenbach 85febdf3d7 Merge pull request #1297 from infinito-nexus/fix/sys-svc-webserver-core/proxy-worker-limits
fix(sys-svc-webserver-core): size proxy workers from the proxy alone
2026-10-30 17:40:49 +01:00
kevinveenbirkenbach d9a65873de Merge commit '455d06552a' into feature/store-i18n 2026-10-30 17:29:51 +01:00
kevinveenbirkenbachandClaude Opus 5.5 455d06552a fix(web-app-baserow): keep the idle database connection alive across the swarm VIP
CI run 40979252521, Baserow#1 (swarm, variant 1): the PASS 2 MCP probe got a 500 and the deploy refused the endpoint. In replica 3, worker 244 served MCP at 11:20:40 and then sat idle for 3824 s; the next request at 12:24:24 failed with "OperationalError: server closed the connection unexpectedly", and every later one, including the 13:09:10 probe, with "InterfaceError: connection already closed". Postgres stayed up throughout. The swarm service VIP (IPVS) drops TCP idle for 900 s.

BASEROW_DATABASE_URL now carries libpq keepalives (keepalives_idle=60, interval 10, count 6, tcp_user_timeout 120000), the same set as the OpenProject swarm fix. test_database_keepalive.py parses the rendered URL with libpq's parse_dsn and fails on the old URL.

Baserow 2.4.1 mounts its MCP Starlette app before Django's handler, so close_old_connections never runs there and a dropped connection stays dead until the worker restarts; that half is upstream. A server-side keepalive on svc-db-postgres that replaces the per-role options is tracked in #1298.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:52:50 +01:00
kevinveenbirkenbachandClaude Opus 5.5 b894db0a75 fix(svc-bkp-volume-2-local): restore into the resolved volume mountpoint
rsync 3.5.0 (Debian security update 3.5.0+ds1-0+deb13u1) refuses a
destination symlink owned by neither root nor the calling user.
svc-opt-ssd-hdd turns a volume's `_data` into a symlink to /mnt/hdd or
/mnt/ssd. The restore drill empties the volume and runs `compose create`;
docker repopulates it and lchowns the `_data` symlink to the image
directory's owner (e.g. www-data). The restore then failed with
"refusing to follow a symlink owned by an untrusted user" and
"change_dir#1 .../_data/ failed: Too many levels of symbolic links",
the drill left its subjects down, and the dependent checks timed out
(CI run 40979252521, ssd-hdd#0, on feature/store-i18n; recover.py is
identical on main).

VolumeRecovery now resolves the mountpoint before handing it to rsync:
os.path.realpath locally, `ssh <host> realpath -e -- <mountpoint>` for
--docker-host, so rsync writes into the real directory and never meets
the symlink. `--insecure-links` was not used because older rsync builds
on other distributions lack it. Two new unit tests (local symlinked
mountpoint, remote resolution); both fail against the previous code.

Root cause and fix by the i18n session (feature/store-i18n triage of
ssd-hdd#0), ported to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:51:44 +01:00
kevinveenbirkenbach dcd6b37812 Merge commit '024259e89d'; commit 'b40c83a694'; commit '470d970a9a' into feature/store-i18n 2026-10-30 16:07:05 +01:00
kevinveenbirkenbachandClaude Opus 5.5 024259e89d feat(design): recolour single-colour logos per slot and preview every app's assets
A logo whose solid pixels share one colour (alpha >= 128, every channel within 24) is detected once per file and tinted to the text colour of each slot: frame_fill on frame slots, fill elsewhere, plus a prefers-color-scheme: dark variant in the SVG of page slots and the page text colour for the favicon. Edge pixels below half opacity are ignored so antialiasing does not count as a second colour. Multi-colour logos render byte-identical to before.

make design-logos (alias i8delo) renders every role with a design entry from one logo and base colour into an HTML page with day and night tiles, without a deploy. Slots render in four threads and roles in one process per CPU: the full preview went from 287.6s to 33.2s with identical output (diff -rq over 1229 PNGs). A GPU does not pay off for these PIL resizes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:06:30 +01:00
kevinveenbirkenbachandClaude Opus 5.5 3c13d3e0ca fix(web-app-nextcloud): judge the Talk TURN assert on the stack host only
Swarm run 2 of the 061 Nextcloud step failed on every worker with "Assert Talk has the TURN server its calls relay through". The TURN server comes from web-svc-coturn's group membership (meta/services.yml standalone_enabled), which only the stack host carries; workers never run Talk, so the assert judged hosts it has nothing to say about. It came in with 5728502a19 and exists only on this line, not on main.

The assert now runs where IS_STACK_HOST holds: every compose host and the swarm manager, unchanged. test_talk_turn_assert.py evaluates the block and task guards plus the assertion with Jinja; against the unfixed file only the worker case fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 16:02:31 +01:00
kevinveenbirkenbachandClaude Opus 5.5 69a0dfa9b9 fix(sys-svc-webserver-core): size proxy workers from the proxy alone
The shared OpenResty nginx.conf took worker_processes and worker_connections
from WEBSERVER_WORKER_*, which are the min() of svc-prx-openresty's limits and
the current play's application_id. The file renders once, in the scope of
whichever role loads the proxy first, so that role's limits capped every vhost:
on feature/store-i18n web-svc-mirror (pids_limit 64) loaded it first, every
vhost got worker_connections 64, and openresty raised 555 "64
worker_connections are not enough while connecting to upstream" alerts while a
single openwebui page fires ~150-170 chunk requests (CI run 40979252521,
homeassistant#1). Main carries the same wiring.

Add WEBSERVER_PROXY_WORKER_PROCESSES and WEBSERVER_PROXY_WORKER_CONNECTIONS,
computed from svc-prx-openresty's own openresty service only (same
resource_filter fallbacks, at least one worker), and render nginx.conf from
them. WEBSERVER_WORKER_* keep pairing the proxy with the play for app-internal
nginx such as Nextcloud's. The values can only stay equal or rise.

Found by the i18n session's triage of homeassistant#1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 14:53:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 11308259b3 test(sys-svc-container): read the role vars through ROLE_FILE_VARS_MAIN
Gate pass 5 flagged test_gpu_detection.py:31 in test_file_spot: the
`ROLE / "vars" / "main.yml"` chain hard-codes a role-file path the
utils.roles.mapping SPOT already names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 13:43:02 +01:00
kevinveenbirkenbachandClaude Opus 5.5 137afc60cc fix(sys-svc-container): require device and driver for a GPU host, refuse a driverless bare host
Follow-up to ae259a9f58 after an independent review:

- The 136-character task name broke the 80-character name lint
  (tests/lint/repository/test_name_emoji_and_length.py). Names are short
  again; the device/driver/host matrix lives in the role README.
- Both stats now always run and lookup('gpu') requires both registers,
  instead of reading a skipped device stat as "no device".
- A bare host that shows /dev/nvidia0 without nvidia-smi stops the
  deploy with a message naming the fix. It used to die on the missing
  binary anyway; ae259a9f58 would have run its GPU services on the CPU
  silently. A container in that state (a swarm lab node, the dev
  container without its GPU override) still runs on the CPU.
- svc-ai-s1 decided its CUDA build from its own stat of /dev/nvidia0 and
  nvidia-container-runtime while its runtime came from lookup('gpu'), so
  a host with the toolkit but no nvidia-smi would have built a CUDA image
  that gets no GPU. S1_GPU now reads lookup('gpu'), the one detection.
- The libretranslate-engine README and the s1 TODO describe the new
  detection.

test_gpu_detection walks the real tasks over device, driver and
container; against the original 00_core.yml it fails in the
driverless-container, driverless-bare and device-and-driver cells.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 11:57:24 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ae259a9f58 fix(sys-svc-container): count a host as GPU host only beside the NVIDIA driver
The local 061 Nextcloud swarm step failed on every lab node in
"Ask the driver which CUDA devices this host carries" with
"[Errno 2] No such file or directory: b'nvidia-smi'". The lab nodes are
privileged containers (compose/swarm/compose.yml), so on a workstation
with an NVIDIA card they see the host's /dev/nvidia0, but they carry no
driver. The device node alone made them GPU hosts, for the GPU include
and for lookup('gpu') alike, which would also have picked a -cuda image
and runtime: nvidia for a GPU service there.

The role now stats SYS_SVC_CONTAINER_GPU_DRIVER (/usr/bin/nvidia-smi)
first and stats the device node only where the driver exists; the
skipped register reads as "no device" in lookup('gpu'), and the include
requires a device stat that ran and found the node. The query runs the
same binary the detection probed. Hosts with driver and device (bare
metal with the driver, the compose dev container through the GPU
override) and hosts without a device (CI) behave as before.

test_gpu_detection walks the real tasks over the four driver/device
cells and fails on the old 00_core.yml in exactly the device-without-
driver cell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 11:44:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 46962b5110 Merge branch 'feature/updater' into feature/store-i18n
Brings in feature/updater 8c14873225: lockdown-probe with the Bluesky
broker key-rotation fix (0a6b6f9130) and the occ-only trusted_proxies
(406998f43a), the ERPNext server_scripts switch, and main with the
hotfixes #1290 (BBB apps-akka READ_COMMITTED), #1291 (FusionDirectory
PHP_MEMORY_LIMIT) and #1293 (Matrix variant 1 force_verification false).

Conflict resolutions:
- web-app-erpnext: create_site.yml takes updater's include of
  restart_frappe.yml; that file's compose/swarm gates use
  IS_COMPOSE_MODE / IS_SWARM_MODE instead of DEPLOYMENT_MODE literals.
- web-app-mailu, web-app-stalwart vars: store-i18n's RUF from
  users.platform_administrator plus the new RUA vars; dmarc.rua_user is
  platform_administrator (on store-i18n `administrator` is the host
  account), so RUA and RUF share the app mailbox. READMEs follow.
- web-app-stalwart 09_dns-records.yml: store-i18n's block form with the
  alias-domain MX task, plus rua=mailto in the DMARC record.
- web-app-pretix: store-i18n's source build stays; the version bump
  lands as source_version v2026.9.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 10:16:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d5a7152575 test(web-app-erpnext): read the script through the shared cache in the server_scripts test
test_no_uncached_file_reads flagged four direct reads in test_server_scripts.py. The script comes from the project tree and is now read through utils.cache.files.read_text. The three reads of common_site_config.json stay direct and carry `# nocheck: cache-read tempdir fixture`: the file lives in a temporary directory and the script under test rewrites it within the same test, so a cached read would return stale content. Each read sits on its own line so ruff format keeps the marker beside it. Quality-high pass 16 ran green with this change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 09:43:47 +01:00
kevinveenbirkenbachandClaude Opus 5.5 131e8ea108 fix(web-app-suitecrm): keep the onion sibling's SAML login on its own host
CI run 40979252521 SuiteCRM#0 (swarm) picked the onion sibling and every
signed-in design spec failed on "the administrator owns a SuiteCRM
session" (100 times in the job log). The SAML ACS is
%env(SITE_URL)%/saml/acs and SITE_URL is the single canonical URL, so the
trace shows Keycloak posting the assertion to
https://suite.crm.main.infinito.test/saml/acs; the session landed on
clearnet and the onion host bounced back to /saml/login. Keycloak would
also have refused an onion ACS, because the SAML client listed only the
clearnet one.

- SuiteCRM: an auto_prepend_file (site_url.php, config-mounted) sets
  $_ENV['SITE_URL'] to the network sibling the request arrived on,
  matched only against the sibling URLs rendered at deploy time;
  Kernel::setSiteURLEnv keeps an existing value. The canonical host and
  CLI runs keep the installer's site_url. The mount target is read from
  the volume registry.
- Keycloak: the SAML client's redirectUris come from
  lookup('network_urls', <app>/saml/acs). Additive: clearnet stays first
  and a single-network node renders the same one-entry list.

Executing the rendered site_url.php in PHP sets the onion URL for the
onion host (with and without :80) and leaves SITE_URL unset for the
clearnet host, a forged host and the CLI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 09:32:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 8f5937d3a4 fix(web-app-espocrm): keep the onion sibling's OIDC callback on its own host
CI run 40979252521 EspoCRM#0 (swarm) picked the onion sibling and failed
the OIDC addon spec plus every signed-in design spec: EspoCRM builds its
redirect_uri as siteUrl + /oauth-callback.php from the single canonical
URL, so the onion login popup returned to
https://espo.crm.main.infinito.test/oauth-callback.php (12 times in the
job log), oidc.js could not read the code across origins and the onion
host never got a session.

A data/config-override.php, which Config::load reads on every request,
now answers siteUrl with the network sibling the request arrived on,
matched against the sibling URLs rendered at deploy time and nothing
else. The canonical host and CLI runs see no override. The manager ops
write it through container exec with a compare-and-rename that only
reports a change when the content moved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 09:12:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 99a7d28067 fix(web-app-weblate): let the uid-1000 containers write their static tmpfs
CI run 40979252521 Weblate#1 (swarm) never converged in its first pass:
every weblate, worker and beat task died on collectstatic with
"PermissionError: [Errno 13] Permission denied:
'/app/cache/static/weblate_fonts'". The per-container tmpfs from
a0bcd59150 sits on the weblate_cache volume, whose static mountpoint is
root-owned 0755 (nfs-volume-contents.txt: drwxr-xr-x 0:0), and runc gives
a tmpfs on an existing mountpoint that directory's mode, while Weblate
runs as uid 1000.

The three static tmpfs mounts now declare mode 1777, rendered as
tmpfs.mode 1023; test_static_root_per_container.py asserts it per
service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 09:08:51 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5faa21c458 fix(web-app-matrix): start Synapse on the rendered config before MDAD creates users
CI run 40979252521 Matrix#3 (ansible flavor) failed the MDAD bootstrap
in the pass after the credential rotation at "Ensure Synapse user
registered - baibot". store-i18n runs the playbook with
setup-all,ensure-matrix-users-created,start, so the user creator runs
before start. After the reconfigure restart systemd had already booted
matrix-synapse on the previous pass's homeserver.yaml, which failed
against the rotated database password ("password authentication failed
for user matrix" in the nested journal) and crash-looped. The user
creator's `state: started` returned ok for the still-activating unit,
skipped its pause, and the registration found no container.

bootstrap.sh now stops an installed matrix-synapse.service right before
the playbook, so the user creator starts it on the configuration the
playbook has just rendered and waits for it. A first install has no unit
yet and is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 09:06:28 +01:00
kevinveenbirkenbachandClaude Opus 5.5 7947e80f9d feat(web-app-erpnext): switch Server Scripts through services.erpnext.server_scripts
applications.web-app-erpnext.services.erpnext.server_scripts (bool, default false) now sets server_script_enabled on every deploy, so an inventory can turn ERPNext Server Scripts on and off again. The security check app needs them, and the validator aborted on its inventory with "Missing default for web-app-erpnext: services.erpnext.server_scripts" while core declared no such key.

Frappe v16 reads the key only from common_site_config.json, through a cached lookup (frappe/utils/safe_exec.py:86-88), and tests it with bool(). files/python/apply/server_scripts.py therefore writes the global file, never the site config, and stores the integer 1 or 0: a plain `bench set-config -g server_script_enabled 0` stores the string "0", which bool() reads as enabled. A value already stored as a string is normalised. The script reports changed only when the value differs, and only then are the Frappe processes restarted, so they drop the cached config.

The restart, the backend re-resolve and the gunicorn wait move from create_site.yml into 02_bench_bootstrap/restart_frappe.yml, unchanged apart from the task names, so the new site and the switch share one restart path for compose and swarm. Six unit tests cover enabling, disabling a string value, the string "0", an aligned value left untouched, the capitalised bool Ansible renders and a rejected value; storing strings instead fails three of them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 09:05:02 +01:00