The dedicated deploy launched whatever ansible-playbook PATH found. A stale pkgmgr venv without Pillow crashed the galaxyserver deploy after 4h23m at the first branding task. The runner now verifies every distribution of the pyproject deploy extra in the running interpreter before make setup, exits naming the missing ones with the install command, and runs the ansible-playbook next to that interpreter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A user key that no role declares, such as an inventory leftover, gets no username, firstname, lastname, uid or gid derived. openldap_user_entries then crashed with a bare KeyError 'username'. The filter now raises an AnsibleFilterError that names the user and the missing fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 41147980556 Nextcloud#0 failed both conductor design specs on a
multi-network node. e0179a0893 handed every conductor the other
network's conductor in LOGOUT_SIBLINGS. With that list set,
universal-logout v1.5.0 hides #iframe-warning on the stand-alone page
("'#iframe-warning' matches nothing visible"), and v1.5.1 fetches the
sibling after its sweep, which the conductor's per-network CSP blocks
("Connecting to 'https://logout.main.infinito.test/' violates ...
connect-src 'self' *.<onion>"). Requirement 037 limits each vhost's CSP
to its own network, so the browser hand-off could never complete in
either direction; its only effects were the two red specs.
Drop the LOGOUT_SIBLINGS wiring and its README bullet, so the compose
template and README match their state before e0179a0893. Each conductor
still logs out of every domain of its own network; ending the sessions
of the other network belongs on the server side, not in the browser.
The test now asserts that no conductor on a single-network, tor-only or
multi-network node is handed a conductor of another network.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A production redeploy of echoserver hung at the swap-forge task and the
host stopped answering SSH. swap-forge v1.0.1 recreates the swapfile
whenever its size differs from the requested MiB by any amount, starting
with swapoff on the live file. The request is the host's MemTotal, which
drifts by a few MiB across kernels, so a routine deploy paged the whole
swap back in, charged it to the containers' memory cgroups and starved
sshd until the OOM killer took swapoff.
Read the active swap files with swapon before swap-forge runs and skip
it when an active file is within 5 % of the requested size, reporting
the kept size against the request. The sizing formula is unchanged, so
no host is forced into a one-time recreate; a deliberate resize beyond
the tolerance still reaches swap-forge.
The unit test resolves the real vars through Ansible's Templar for MiB
drift both ways, no swap, zram and partitions, a halved size, an exact
pin, a lowercase unit and an escaped file name, and shows that the exact
MiB comparison swap-forge uses misses the drift.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fork CI run 41147980556 pgAdmin#0 failed in sys-ctl-hlth-runtime with
"stalwart_webmail: PHP 8.4.26 is below the security floor 8.4.27".
The deploy ran the fork's GHCR mirror of roundcube/roundcubemail:1.7.5-
apache, copied on 10-19 with PHP 8.4.26, while upstream re-pushed the
same tag on 10-27 with PHP 8.4.27. Branch CI runs the sync with
--only-missing, which skipped the tag because it existed ("destination
exists, skipping") without comparing digests, and the full refresh only
runs on the fork's default branch, which still pins 1.7.4.
- utils/docker/registry.py: manifest_digest() reads the
Docker-Content-Digest of a reference with a HEAD request, which does
not count against Docker Hub's pull allowance.
- GHCRProvider.is_stale() reports a mirrored tag as stale only when both
registries answer with a digest and the two differ.
- Under --only-missing an existing tag is re-copied when it is stale; a
failed re-copy is reported on stderr with the image and keeps the
existing copy, so one refresh cannot block every deploy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared proxy's nginx.conf derived worker_connections from the
openresty container's pids_limit. pids_limit caps processes and threads,
worker_connections caps simultaneous sockets per worker (client and
upstream), so with pids_limit 256 and one worker the whole host's proxy
held about 256 sockets, roughly 128 proxied requests in flight across
every vhost. Raising pids_limit to buy connections would loosen the
fork-bomb guard.
svc-prx-openresty now declares services.openresty.worker_connections
(4096 per worker, the common nginx production range, enough for several
chunk-heavy apps such as OpenWebUI at ~150-170 requests per page load).
WEBSERVER_PROXY_WORKER_CONNECTIONS reads it through lookup('config'),
without a default, so a missing key fails loudly. pids_limit stays 256.
WEBSERVER_PROXY_WORKER_RLIMIT_NOFILE is derived as 2x worker_connections
(8192), one descriptor for each client and upstream socket of a proxied
request, and nginx.conf renders it as worker_rlimit_nofile. The openresty
container pins its nofile ulimit (soft and hard) to the same value:
workers call setrlimit as root without CAP_SYS_RESOURCE, so they can only
lower the hard limit, and the inherited hard limit depends on the host
daemon (Docker >= 25 defaults to 1024:524288, DiD nests whatever the
outer container has). An explicit ulimit makes the rlimit succeed on
every host instead of logging a setrlimit alert.
The unit test pins worker_connections 4096, worker_rlimit_nofile 8192 and
the container ulimit, and fails when worker_connections follows
pids_limit again.
Live proof (compose deploy of web-svc-html, both plays failed=0): the
deployed nginx.conf renders worker_rlimit_nofile 8192 and
worker_connections 4096; openresty's master, worker and cache manager
run with "Max open files 8192 8192"; the openresty log carries no
setrlimit or "worker_connections are not enough" line.
Closes#1300
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_sso.py, test_sso_oidc_plugin.py and test_sso_proxy_consumers.py
evict utils, utils.cache, utils.cache.applications and the roles
packages from sys.modules to import their plugin against fresh modules,
and never put the originals back. Ansible's plugin loader keeps the
applications lookup it loaded earlier in sys.modules, so that lookup
stayed bound to the evicted utils.cache.applications, whose
_MERGED_APPLICATIONS_CACHE utils.cache._reset_cache_for_tests() no
longer reaches.
On main this turned test_webserver_worker_limits red whenever one xdist
worker ran a test that loads the applications lookup through Ansible,
then one of these sso tests, then the worker-limits test: the stale
cache served the cpus 0.5 payload of test_a_sub_cpu_proxy_still_runs_one_worker,
so the later methods rendered worker_processes 1 instead of 4 (main run
41139196554, Workspace centos and fedora, second make test on gw1).
_load_module now runs inside mock.patch.dict(sys.modules): the plugin
keeps the fresh modules it was built against, and sys.modules returns to
its prior state afterwards. This is the change store-i18n carries since
ec0fde6561, ported unchanged so both lines stay identical.
Evidence:
- Before: pytest -n 0 test_tor_ports.py test_sso_oidc_plugin.py
test_webserver_worker_limits.py -> "3 failed, 33 passed", the CI
failure set ('1' != '4' twice, worker_processes 1 vs 4).
- After: test_tor_ports.py, the four sso test files and
test_webserver_worker_limits.py -> "68 passed".
(cherry picked from commit ec0fde6561)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main's admin allowlist test renders sys-svc-proxy's location/html.conf.j2,
which calls the entity_name Jinja filter. The test stubbed it as
get_entity_name, the name store-i18n retired when it registered the
filter as entity_name (plugins/filter/get/entity_name.py), so all eleven
cases failed with "No filter named 'entity_name'" after the main merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cursor#0 (CI run 40979252521, debian host) installed the pinned cursor_3.27.6 deb on pass 1; its postinst registers Cursor's apt repo, so pass 2's dist-upgrade moved cursor to 3.27.22, and the plan's ansible.builtin.apt deb=<3.27.6> then failed with 'A later version is already installed': the apt module skips only an equal version and otherwise fails or, with allow_downgrade, downgrades. package_install now asks the target before such a call: dpkg-deb --field reads Package and Version from the checksum-verified deb, dpkg-query reads the fully installed version, and dpkg --compare-versions decides; at or above the pin the call is skipped with changed false, so the deploy-time upgrade may move vendor packages forward and the pin never pulls them back. Any query that cannot be read falls through to the apt install, which decides loudly. The dnf path already skips an older local rpm, so both families now behave alike.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main CI run 40976399687, socialhome#0 (swarm, tor, centos): the CLI tests
of svc-net-tor (onion:25 "socks CONNECT failed with reply 0x01") and
web-app-stalwart (probe_mail relay: ConnectionRefusedError on port 25)
failed. dockerd on the manager had aborted the whole ingress program of
the mail stack:
Failed to add ingress: failed to program ingress ports:
failed to bind host port 0.0.0.0:993/tcp: address already in use
/proc/net/tcp named the squatter: 192.168.244.10:993 -> :2049, the
node's own NFS client. The mount carries no noresvport, so the kernel
drew a source port from the 665-1023 reserved window and landed on
imaps. None of 25/465/993/995/4190/8095 got a DNAT rule or a
docker-proxy listener while every other published port did, and
stalwart still reported healthy because its healthcheck runs inside the
container. 993 and 995 are the only published ports inside that window,
so this is luck of the draw on any NFS-backed swarm host that runs the
mail role.
mount_opts now emits noresvport. insecure on the kernel-flavor export is
the other half: kernel nfsd defaults to secure and answers a mount from
an unprivileged source port with EACCES. It reaches the kernel flavor
only; ganesha already accepts unprivileged clients.
What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state; a host-side job
holding that path at that instant fails loudly. A plain remount may keep
a running mount on its old reserved port until the next real mount.
Port of f46c4d0bab from feature/store-i18n, applied identically so the
later store-i18n -> main merge stays conflict-free.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared pacman cache was the first writable CacheDir on every arch runner and lab node. Our arch image sets DisableSandbox, so libalpm downloads straight into that directory (util.c _alpm_download_dir_setup only makes a per-process temp dir with the sandbox on), and the XferCommand writes <pkg>.part there with curl -C -o %o; two nodes fetching the same package appended to the same .part file, and the second rename failed or left a corrupt package in the shared cache.
pacman.sh now gives each node its own download directory <shared>/pacman/node-XXXXXX/ (created once with mktemp, reused from pacman.conf on later runs) as the CacheDir between the read-only host layer and <shared>/pacman/, and writes a PostTransaction hook to pacman.d/hooks/infinito-shared-cache.hook that renames every finished package and signature into <shared>/pacman/; .part files stay for the next resume. Both directories sit in one bind mount, so each publish is an atomic rename(2). pacman.sh fails when pacman.conf sets HookDir, since pacman would then skip the hook silently. Requirement 065 and cache.md describe the three layers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The svc-cache-package CLI probe GETs every declared route path through the frontend and fails on 502/503/504. registry.ollama.ai declared no paths, so the probe asked for /, which upstream now answers with 503; Nexus relays an upstream 5xx as its own 503, so package#0 (CI run 40979252521) failed with 'registry.ollama.ai/ on port 443 answered 503'. The host now declares /v2/, the registry API root every ollama pull goes through (manifests and blobs); upstream answers it with 404, which Nexus relays as 404 and the probe accepts. On a TLS host the frontend still proxies the whole host through location /, because upstreams.conf.j2 reads routes only for plain-HTTP listeners: the rendered upstreams.conf is byte-identical before and after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_compose_siblings.py called yaml.safe_load on the rendered compose file, which TestNoDirectYamlCalls rejects in favour of the utils.cache.yaml helpers (quality-high run 6).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_s3_public_url.py built roles/web-app-wordpress/meta/services.yml with a pathlib chain, which TestRoleFileSpot rejects in favour of the utils.roles.mapping constant (quality-high run 6).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel compose slots, swarm lab clusters and GUI lanes each kept their own pip, npm and pacman downloads, and a failed swarm run left its cluster (about 25 GB of node docker data plus the rescue bundle in the act container) on the root filesystem until someone ran make swarm-down. INFINITO_CACHE_SHARED_HOST_PATH (default <DIR_CACHE>/shared) is now mounted read-write at the same path into every compose slot and every swarm lab node, with PIP_CACHE_DIR and npm_config_cache below it; empty, the renders stay byte-identical to before. On a host whose os-release ID is exactly arch running an arch stack, INFINITO_CACHE_HOST_PACMAN_PATH mounts the host pacman cache read-only and scripts/docker/cache/pacman.sh makes it the first CacheDir with <shared>/pacman/ second, so new downloads land in the shared folder and the host cache stays untouched; on Manjaro and every other host pacman uses the shared folder alone.
make swarm-zombie without keep=true now copies the act job's rescue diagnostics and Playwright artefacts to the host and then releases the lab nodes, the lab network, the NFS export volume and the act outer container, on success and on failure; when the copy fails the nodes still go, the act container stays and the run exits naming make swarm-down. make roundtrip passes keep to every swarm step, make clean-cache removes the shared folder, and the disc-space cleanup skips pacman cache directories it cannot write. apt and dnf keep their per-container caches behind the package-cache proxy, and no docker data root is shared between daemons. Requirement 065.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WordPress put objstore.public_url, the clearnet api.seaweedfs.s3 host, into S3_UPLOADS_BUCKET_URL, so the onion pass of WordPress#0 (CI run 40979252521) loaded the site logo from clearnet and the leak spec failed; the /objstore rebase only fired on onion-only consumers and never on a multi-network node. lookup('objstore', app, 'relative_url') now returns /_infinito/objstore/<bucket> for a shared consumer with a public bucket and an API domain, and locations.conf.j2 serves that path on every vhost of the consumer, clearnet and onion alike, by proxying over the loopback to the provider's own vhost.
The route passes GET and HEAD only, answers 404 for a path ending in / (no ListObjects), drops the query string (no S3 subresources or presigned queries) and the Authorization and Cookie headers, so only the public bucket's anonymous read applies, and adds nosniff plus a sandbox CSP; private and embedded buckets get no route. Only web-app-wordpress switches to relative_url: mastodon, pixelfed, baserow, bookwyrm, snipe-it, listmonk, mediawiki, shopware and peertube need absolute URLs for federation, mail or server-side rendering, and decidim and funkwhale sign URLs, which the route deliberately refuses. The prefix moves from /objstore to /_infinito/objstore.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node serving several networks runs one universal-logout conductor per network, and a page served over HTTPS cannot fetch the HTTP onion hosts, so one conductor cannot clear the other network's sessions. compose.yml.j2 now gives every conductor the url.origin of the other networks' conductors in LOGOUT_SIBLINGS (empty on a single-network node, which disables the hand-off), and the Keycloak logout panel that frames the conductor follows its handoff report to the next conductor once the sweep is done, accepting only an http or https URL.
The pinned image v1.4.2 ignores LOGOUT_SIBLINGS and never reports handoff, so behaviour changes only with the bump to the universal-logout release that carries the reachability probe. test_compose_siblings.py renders single- and multi-network nodes; logout-panel.test.js covers the handoff branch and rejects non-web URLs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compose_volumes merged extra_volumes first and then skipped every meta/volumes.yml entry whose key was already present, so the meta nfs value was lost with it. Nextcloud injects talk_recording_tmp through nextcloud_extra_volumes without an nfs key, and 5728502a19 marked that volume nfs: false. On swarm with the NFS backend the stack therefore bound it to /var/lib/infinito/nextcloud_talk_recording_tmp, while nfs_prep.yml renders the same lookup without extras, saw nfs: false and never created the subdir; every node rejected the task with 'no such file or directory'. The meta nfs value now carries over onto the injected or derived spec unless that spec sets nfs itself, so both renders agree and the scratch volume stays node-local.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL py/unused-global-variable flagged two test package inits
(code-scanning alerts #1068 and #1072):
- tests/unit/python/roles/web-app-fider/__init__.py exports PROJECT_ROOT
to test_tenant_bootstrap_gate.py through `from . import PROJECT_ROOT`
but declared no __all__. Declare it, as every other test package
that re-exports PROJECT_ROOT does.
- tests/unit/python/roles/sys-front-tls/__init__.py defined a
PROJECT_ROOT nothing imports: the only test lives in templates/, whose
own __init__.py provides it. Leave the role-level package init empty,
like the other role-level test packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frappe v16 limits the Server Script doctype to the Script Manager role and ServerScript.validate calls frappe.only_for('Script Manager'), so services.erpnext.server_scripts: true still left every integration on erpnext-api-bot with 403 Insufficient Permission for Server Script. The same switch now grants the bot Script Manager when true and revokes it when false; System Manager stays, the save runs only on a change, and an undeclared value fails before the site is touched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's OpenProject seeder wait (#1302) compared DEPLOYMENT_MODE == 'swarm', which store-i18n's test_deployment_mode_booleans forbids; it now uses IS_SWARM_MODE. Main's test_webserver_worker_limits (#1297) rendered without host facts, so store-i18n's resource lookup refused to measure the controller; the fixture now carries processor_vcpus and memtotal_mb as every play does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CLI test replays the published Production block in a throwaway target that shares the runner's disk. The replayed deploy knew only the default SIZE_PERCENT_CLEANUP_DISC_SPACE of 85 while the stack host runs with its own inventory value, so sys-ctl-hlth-disc-space failed in the target at 87% (Wazuh#1, CI run 40979252521). disc_space.sh writes the stack host's value into the block's provision --vars and fails on a non-integer value or a block without --vars; the published instructions stay unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 40979252521 WordPress#1 (multisite, multi network): a request on a network sibling resolved to the clearnet site, because WordPress picks the site by Host before any mu-plugin runs and the network kept only clearnet base URLs per site. Operator decision: a real multisite per network with real child domains (option B).
infinito-network-origins.php is now also installed as wp-content/sunrise.php (SUNRISE=true), so the sibling host resolves to its own site before mu-plugins load; a guard constant stops the second load from mu-plugins. WORDPRESS_SITE_DOMAINS holds the canonical domains in the first domain's network, WORDPRESS_NETWORK_BASE_URLS carries every sibling of each site's base URL (sites ';', siblings ','), and INFINITO_NETWORK_BASE_URLS takes that list. Child sites are inserted by files/php/multisite/sites.php only where no site exists on the domain and path, replacing wp site list plus wp site create. The multisite variant's child domains move from server.domains to domains.canonical.
Covered by 29 Python tests and two PHPUnit suites; PHPUnit and a variant 1 deploy with an onion pass still have to run (store-i18n gate and deploy).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_run_streamed_outlives_a_deadline_while_it_keeps_talking patched RUN_SILENCE_SECONDS to 1.0 while the chatty child prints its first line only after the interpreter has started. run_streamed measures silence from process start, which is right for production (a command that never speaks must time out), so under xdist load on a full disk the cold start alone exceeded the window and the run returned 124 (reported from IAM's make quality).
The window is now 2.0 s against a 0.1 s cadence over 40 lines (4 s, still twice the window, so the test keeps proving it outlives a deadline while talking), and the child runs with -I -S to skip the site import. With a simulated 1.5 s cold start the old parameters return 124 and the new ones 0. pull.py is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
