76adaeb9e8 wrote the regex_search group reference as '\1' inside the folded
task scalar. Jinja unescapes string literals, so the filter received \x01,
rejected it as an unknown argument and failed the dev-python task on every
remote host. The literal is now '\\1', and the unit test renders the task's
environment expression the way Ansible does instead of only checking that
the variable is named.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bf9c5be173 capped the bootstrap at 3.14 with constants inside install.sh, a
second copy of the range pyproject.toml already declares. requires-python is
now the single source (>=3.11,<3.15): install.sh derives its floor and cap
from it, in the repository or, on a remote host where the script runs alone,
from INFINITO_PYTHON_REQUIRES that the dev-python role reads from the same
file; any other form fails loudly. The Dockerfile copies pyproject.toml
before the bootstrap, and pip now refuses 3.15 for the package as well.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fedora 45 defaults to Python 3.15. ansible-core 2.21, which the unpinned
`ansible` extra still resolves to (ansible 14.5.0 pins ~=2.21.5), calls
dataclasses._is_type, which 3.15 removed, so the Workspace (fedora:latest)
job died before the first playbook (run 41491308729).
install.sh now picks and installs the highest Python between
PYTHON_MIN_MINOR and PYTHON_MAX_MINOR (3.11 to 3.14) on apt, rpm and
pick_python_bin, and fails loudly when only newer interpreters exist. On
Fedora 45 it installs python3.14. Raise PYTHON_MAX_MINOR once ansible ships
ansible-core 2.22.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The small GitHub runner shape (AWS, 72G root) ran out of disk in swarm and compose rows (#1329: WordPress#2 of run 41261531965, Nextcloud#6 and #7 of run 41262286962) while an empty instance-store disk nvme1n1 sat unused. mount_spare_disk.sh formats and mounts it at /mnt, but only on a GitHub-hosted runner outside act, and only a whole unpartitioned disk without any signature, mount or swap use of at least 75G; every decline leaves a notice. relocate_docker_dataroot.sh then bind-mounts /var/lib/containerd from the spare disk and moves the Docker data-root to /mnt/docker-host (not /mnt/docker, which is the nested daemon's volume that dedicated.sh wipes). It refuses before stopping anything when images, volumes, build cache, foreign containerd namespaces or shims exist or daemon.json is unreadable, and rolls back the bind mount and daemon.json on any later failure or cancel. Swap goes on / once containerd lives on /mnt, dump_state.sh logs disk size, model and free space per storage root, and the swarm watchdog samples / and /var/lib/containerd every 10 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestDotenvSmoke ran `python -m cli.meta.env` with cwd=PROJECT_ROOT and no
--output, so every make test rewrote the real .env with defaults: as root
inside the container, with a fresh INFINITO_CACHE_PACKAGE_ADMIN_PASSWORD
and possibly another INFINITO_DOMAIN.
The test now writes to --output <tempdir>/.env and asserts that the
repository .env keeps its inode and mtime. A marker line in a worktree .env
vanished on every run of the old test and survives the new one; without
--output the new assertion fails with "The smoke run replaced the
repository .env".
Fixes#1188
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The access-log map read $request_uri, which keeps percent-encoding and
repeated slashes. A request such as /%70assword/reset/<token>,
/password%2Freset/<token> or //password//reset/<token> missed the
token-route expression, fell through to the plain-path branch and was
logged with the token in plain text, while the backend still decoded it
to its reset handler.
The map now reads $uri: decoded, slash-merged (merge_slashes is not
overridden anywhere) and without the query string, so the existing
expressions match every spelling of a token route. Trade-off: after an
internal redirect the log shows the internal URI; with the generic
vhost's error_page (ws_generic.conf.j2) a 5xx line now carries
/500.html instead of the client's path, with the status unchanged.
Nothing in the repository parses the proxy access log.
The unit test resolves the map against the variable it reads, modelling
$uri as nginx builds it, and adds the three encoded spellings above.
Negative control: with the map back on $request_uri, exactly those three
cases fail (3 failed, 8 passed).
Reported by CodeRabbit on #1322.
Part of #1192
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The front proxy logged the full request line: production had no
access_log, so nginx fell back to `combined` ($request, Referer,
$remote_user), and the debug format logged $request and the Referer.
Query-string tokens (?access_token=, ?key=, ?token=) and token routes
such as /password/reset/{token} and /i/confirm-email/{user}/{random}
ended up in plain text in the container log.
nginx.conf.j2 now declares a map from $request_uri to
$infinito_access_log_path:
- the path is written without the query string;
- the segments after a known token route (password/reset,
password-reset, reset-password, confirm-email,
lostpassword/reset/form, email-login, activate-account; matched
case-insensitively at any depth) are written as [redacted], so the
request, its status and timings stay visible without the token.
Both modes write an explicit access_log to /dev/stdout:
- production uses the new `redacted` format, laid out like `combined`
(remote address, time, "method path protocol", status, bytes,
user agent) with Referer and basic-auth user written as "-", so
combined-style parsers keep working;
- debug keeps its fields (host, timings, upstream time, lengths,
X-Forwarded-For, scheme, server name) but logs
"method path protocol" instead of $request and drops the Referer.
The path comes from $request_uri instead of $uri so internal rewrites
(error_page, try_files) do not replace the path the client requested.
Nothing in the repository parses the proxy access log (Wazuh, the
health checks and the Prometheus log_by_lua metrics do not read it),
so no consumer needs to change. The error log still carries
`request:` and `referrer:` on error-level entries; that is outside
this change.
A unit test renders nginx.conf.j2 in both modes and asserts that every
access_log uses a format declared there, that no format references
$request, $request_uri, $args, $query_string, $is_args, $arg_*,
$http_referer or $remote_user, and, by resolving the map as nginx does,
that query strings are dropped, token routes are redacted and
lookalike paths stay unchanged.
Verified live with web-svc-html in production mode (deploy failed=0, so
openresty accepted the config). A probe request to
/password/reset/<probe>?token=<probe> was logged as
"GET /password/reset/[redacted] HTTP/1.1" 404 62 "-" "curl/8.14.1",
and /plain/page as "GET /plain/page HTTP/1.1". Negative control: the
same request against the previous config logged the probe path token
and the query token in plain text. The debug format with the fix is
covered by the unit test only.
Closes#1192
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared proxy's nginx.conf derived worker_connections from the
openresty container's pids_limit. pids_limit caps processes and threads,
worker_connections caps simultaneous sockets per worker (client and
upstream), so with pids_limit 256 and one worker the whole host's proxy
held about 256 sockets, roughly 128 proxied requests in flight across
every vhost. Raising pids_limit to buy connections would loosen the
fork-bomb guard.
svc-prx-openresty now declares services.openresty.worker_connections
(4096 per worker, the common nginx production range, enough for several
chunk-heavy apps such as OpenWebUI at ~150-170 requests per page load).
WEBSERVER_PROXY_WORKER_CONNECTIONS reads it through lookup('config'),
without a default, so a missing key fails loudly. pids_limit stays 256.
WEBSERVER_PROXY_WORKER_RLIMIT_NOFILE is derived as 2x worker_connections
(8192), one descriptor for each client and upstream socket of a proxied
request, and nginx.conf renders it as worker_rlimit_nofile. The openresty
container pins its nofile ulimit (soft and hard) to the same value:
workers call setrlimit as root without CAP_SYS_RESOURCE, so they can only
lower the hard limit, and the inherited hard limit depends on the host
daemon (Docker >= 25 defaults to 1024:524288, DiD nests whatever the
outer container has). An explicit ulimit makes the rlimit succeed on
every host instead of logging a setrlimit alert.
The unit test pins worker_connections 4096, worker_rlimit_nofile 8192 and
the container ulimit, and fails when worker_connections follows
pids_limit again.
Live proof (compose deploy of web-svc-html, both plays failed=0): the
deployed nginx.conf renders worker_rlimit_nofile 8192 and
worker_connections 4096; openresty's master, worker and cache manager
run with "Max open files 8192 8192"; the openresty log carries no
setrlimit or "worker_connections are not enough" line.
Closes#1300
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_sso.py, test_sso_oidc_plugin.py and test_sso_proxy_consumers.py
evict utils, utils.cache, utils.cache.applications and the roles
packages from sys.modules to import their plugin against fresh modules,
and never put the originals back. Ansible's plugin loader keeps the
applications lookup it loaded earlier in sys.modules, so that lookup
stayed bound to the evicted utils.cache.applications, whose
_MERGED_APPLICATIONS_CACHE utils.cache._reset_cache_for_tests() no
longer reaches.
On main this turned test_webserver_worker_limits red whenever one xdist
worker ran a test that loads the applications lookup through Ansible,
then one of these sso tests, then the worker-limits test: the stale
cache served the cpus 0.5 payload of test_a_sub_cpu_proxy_still_runs_one_worker,
so the later methods rendered worker_processes 1 instead of 4 (main run
41139196554, Workspace centos and fedora, second make test on gw1).
_load_module now runs inside mock.patch.dict(sys.modules): the plugin
keeps the fresh modules it was built against, and sys.modules returns to
its prior state afterwards. This is the change store-i18n carries since
ec0fde6561, ported unchanged so both lines stay identical.
Evidence:
- Before: pytest -n 0 test_tor_ports.py test_sso_oidc_plugin.py
test_webserver_worker_limits.py -> "3 failed, 33 passed", the CI
failure set ('1' != '4' twice, worker_processes 1 vs 4).
- After: test_tor_ports.py, the four sso test files and
test_webserver_worker_limits.py -> "68 passed".
(cherry picked from commit ec0fde6561)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main CI run 40976399687, socialhome#0 (swarm, tor, centos): the CLI tests
of svc-net-tor (onion:25 "socks CONNECT failed with reply 0x01") and
web-app-stalwart (probe_mail relay: ConnectionRefusedError on port 25)
failed. dockerd on the manager had aborted the whole ingress program of
the mail stack:
Failed to add ingress: failed to program ingress ports:
failed to bind host port 0.0.0.0:993/tcp: address already in use
/proc/net/tcp named the squatter: 192.168.244.10:993 -> :2049, the
node's own NFS client. The mount carries no noresvport, so the kernel
drew a source port from the 665-1023 reserved window and landed on
imaps. None of 25/465/993/995/4190/8095 got a DNAT rule or a
docker-proxy listener while every other published port did, and
stalwart still reported healthy because its healthcheck runs inside the
container. 993 and 995 are the only published ports inside that window,
so this is luck of the draw on any NFS-backed swarm host that runs the
mail role.
mount_opts now emits noresvport. insecure on the kernel-flavor export is
the other half: kernel nfsd defaults to secure and answers a mount from
an unprivileged source port with EACCES. It reaches the kernel flavor
only; ganesha already accepts unprivileged clients.
What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state; a host-side job
holding that path at that instant fails loudly. A plain remount may keep
a running mount on its old reserved port until the next real mount.
Port of f46c4d0bab from feature/store-i18n, applied identically so the
later store-i18n -> main merge stays conflict-free.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL py/unused-global-variable flagged two test package inits
(code-scanning alerts #1068 and #1072):
- tests/unit/python/roles/web-app-fider/__init__.py exports PROJECT_ROOT
to test_tenant_bootstrap_gate.py through `from . import PROJECT_ROOT`
but declared no __all__. Declare it, as every other test package
that re-exports PROJECT_ROOT does.
- tests/unit/python/roles/sys-front-tls/__init__.py defined a
PROJECT_ROOT nothing imports: the only test lives in templates/, whose
own __init__.py provides it. Leave the role-level package init empty,
like the other role-level test packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Addresses two CodeRabbit findings on PR #1261.
1. Scoped IPv6 (thread 4421596356): ipaddress.ip_network accepts a scope
ID, so `fe80::1%eth0` resolved to `fe80::1%eth0/128` and the vhost
would carry `allow fe80::1%eth0/128;`, which nginx refuses to load.
kc_admin_networks now raises AnsibleFilterError for any entry whose
network address carries a scope ID. test_admin_allowlist.py covers a
scoped address and a scoped CIDR; both fail against the previous
filter.
2. Module docstring (thread 4421596351): removed; it described the
module instead of documenting parameters or a trip-wire.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rsync 3.5.0 (Debian security update 3.5.0+ds1-0+deb13u1) refuses a
destination symlink owned by neither root nor the calling user.
svc-opt-ssd-hdd turns a volume's `_data` into a symlink to /mnt/hdd or
/mnt/ssd. The restore drill empties the volume and runs `compose create`;
docker repopulates it and lchowns the `_data` symlink to the image
directory's owner (e.g. www-data). The restore then failed with
"refusing to follow a symlink owned by an untrusted user" and
"change_dir#1 .../_data/ failed: Too many levels of symbolic links",
the drill left its subjects down, and the dependent checks timed out
(CI run 40979252521, ssd-hdd#0, on feature/store-i18n; recover.py is
identical on main).
VolumeRecovery now resolves the mountpoint before handing it to rsync:
os.path.realpath locally, `ssh <host> realpath -e -- <mountpoint>` for
--docker-host, so rsync writes into the real directory and never meets
the symlink. `--insecure-links` was not used because older rsync builds
on other distributions lack it. Two new unit tests (local symlinked
mountpoint, remote resolution); both fail against the previous code.
Root cause and fix by the i18n session (feature/store-i18n triage of
ssd-hdd#0), ported to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared OpenResty nginx.conf took worker_processes and worker_connections
from WEBSERVER_WORKER_*, which are the min() of svc-prx-openresty's limits and
the current play's application_id. The file renders once, in the scope of
whichever role loads the proxy first, so that role's limits capped every vhost:
on feature/store-i18n web-svc-mirror (pids_limit 64) loaded it first, every
vhost got worker_connections 64, and openresty raised 555 "64
worker_connections are not enough while connecting to upstream" alerts while a
single openwebui page fires ~150-170 chunk requests (CI run 40979252521,
homeassistant#1). Main carries the same wiring.
Add WEBSERVER_PROXY_WORKER_PROCESSES and WEBSERVER_PROXY_WORKER_CONNECTIONS,
computed from svc-prx-openresty's own openresty service only (same
resource_filter fallbacks, at least one worker), and render nginx.conf from
them. WEBSERVER_WORKER_* keep pairing the proxy with the play for app-internal
nginx such as Nextcloud's. The values can only stay equal or rise.
Found by the i18n session's triage of homeassistant#1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both deploy steps set `disable: ${{ matrix.disable || inputs.disable }}`.
A row carries tokens of its own whenever its network mode needs them (a
clearnet row disables Tor), so the caller's list never reached the
deploy. On feature/store-i18n (CI run 40979252521) the workspace swarm
zombie for svc-db-mariadb/postgres deployed 19 apps instead of 7 on its
clearnet row (fedora: 4h06 for one step, job cancelled at 345 min). main
carries the same expression; it bites as soon as a caller's disable list
meets a clearnet row.
The steps now pass format('{0} {1}', matrix.disable, inputs.disable);
every reader of `disable` splits on spaces and commas. Both input
descriptions follow. test_deploy_disable_merge.py reads the live
expressions; against the old fallback it fails two subtests.
Ported from feature/store-i18n 57e574ac2d (store-i18n's extra `network:`
step env stays there).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Addresses three CodeRabbit findings on PR #1265.
1. LAM hardening invisible to the probe (thread 4413764929): the runtime
check probes PHP with `php -r`, the CLI SAPI. web-app-lam mounted the
hardening ini only into /etc/php/8.4/apache2/conf.d, which the Debian
CLI does not scan, so the probe read unhardened CLI settings and the
check failed for LAM. The php_hardening entry now also mounts the ini
into /etc/php/8.4/cli/conf.d. No other role mounts it into a Debian
per-SAPI directory; the rest use the shared scan dir of the official
image or of Alpine. The TODO entry now names the remaining gap: the
probe reads only the CLI copy.
2. Explanatory comment above HEALTH_RUNTIME_FLOORS (thread 4413764949):
removed. The README already states the end-of-life rule and now also
defines a floor as the lowest patched release of each line.
3. `apk --no-cache add` missed (thread 4415018662): PACKAGE_INSTALL_PATTERN
required `add` right after `apk`, so `RUN apk --no-cache add php7`
passed as an image without PHP. The apk branch now accepts options
before `add`, like the apt branch. New tests cover the option form and
guard that `apk` in a path or a comment is no package install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tasks/00_core.yml built webserver_extra_configuration by concatenating the
helper iframes, the admin allowlist and the logout panel template lookups
with `~`. With the shipped empty `services.keycloak.admin.allowed_networks`
the allowlist template rendered no text, Ansible's template lookup returned
None for it, and `~` wrote the word "None" into the auth vhost. OpenResty
then refused `unknown directive "Nonelocation"` in front of the logout
panel location and every deploy with Keycloak went red.
The role now resolves the declared networks once in KEYCLOAK_ADMIN_NETWORKS
and the task appends the allowlist template only when that list is not
empty, the same guard the logout panel uses. The template renders the
restricted /admin location unconditionally from KEYCLOAK_ADMIN_NETWORKS.
With a list that did not contain the deploy host, the deploy still went
red: Keycloak answers `/` with a redirect to `/admin/`, so the CSP health
check and the front proxy probe, which request `/` from the host, got 403
and hlth-csp failed. While the allowlist is active, `location = /` now
redirects to the account console of the platform realm instead, and the
README no longer claims the deploy needs no entry for any reason.
test_admin_allowlist.py rendered the template with plain Jinja, which
yields '' for an empty template, so it could not see the None. It now
evaluates the task's own webserver_extra_configuration expression through
Ansible's Templar and real template lookup, which reproduces
"HELPER_IFRAMES\nNoneLOGOUT_PANEL" on the old task, and covers the root
redirect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/lint/filesystem/env/test_worktree_swarm_lab_subnet.py landed on
main with lines ruff format wraps, so every `make autoformat` rewrites it
and leaves an unstaged change behind. Formatter output only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
00_core.yml included 01_setup_oidc.yml only when services.sso.enabled
was true, but that file carried the whole tenant bootstrap: wait for
HTTP, count tenants, POST /_api/tenants, create the admin user and
activate the tenant. Only its last task configured the Keycloak
provider. With SSO off no tenant was ever created.
Fider's RequireTenant middleware then answers every tenant route,
/api/v1/tags included, with 307 to /signup, and /signup is open to any
visitor, who can claim the fresh installation and become its
administrator. The MCP read fider_list_tags got that 307 in the swarm
SSO-off matrix row.
The tenant bootstrap moves unchanged into tasks/01_tenant.yml, which
00_core.yml includes on the stack host regardless of SSO. The OIDC file
becomes tasks/02_setup_oidc.yml and keeps only the provider upsert,
still gated on services.sso.enabled. The README states which bootstrap
steps run without SSO.
A unit test evaluates the include and task guards for every SSO,
stack-host and deploy-mode state: a fresh stack host posts exactly one
tenant with and without SSO, an existing tenant is never posted again,
the Keycloak provider is written only with SSO, and a non-stack host
runs none of it. The compose matrix row with SSO off stayed green
because nothing there read a tenant route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/lint/filesystem/env/test_worktree_swarm_lab_subnet.py landed on
main with lines ruff format wraps, so every `make autoformat` rewrites it
and leaves an unstaged change behind. Formatter output only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Validation step "Assert the base branch is fully merged into the PR
head" ran main_merged.sh, which fetches the live tip of main when the job
starts. PR runs wait in the queue for hours while image-update PRs land
on main several times an hour, so nearly every queued run failed
Validation on start and had to be re-pushed after another main merge
(PR #1265: behind_by=0 at push, behind_by=9 while its run still queued).
main_merged.sh takes a new INFINITO_MAIN_COMMIT override: when set, it
fetches that full commit SHA from the remote and requires HEAD to
contain it instead of the live tip; a set but malformed or empty value
exits 2. The Validation step passes github.event.pull_request.base.sha,
the main tip at event time, which the step's pull_request-only guard
always provides. The run tests the merge against that base, so the gate
still guarantees the tested tree contains it.
The local pre-push hook (make test-main-merged) leaves the override
unset and keeps checking the live tip: a push happens right now, so
lagging the current main is the real defect there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three CodeRabbit findings on PR #1265:
- script.py listed the running containers with `container ps` and
`container inspect` without a timeout, so a hung engine kept the daily
unit active forever and blocked the next timer run. Both calls now use
EXEC_TIMEOUT_SECONDS; main turns a timeout or engine error into a
one-line failure and exit 1 instead of a traceback.
- runtime.py's PHP_PACKAGE_PATTERN requires a digit after "php", so an
image installing an unversioned interpreter package (`php-cli`) and
setting no PHP_VERSION was reported as PHP-free. The interpreter
packages php, php-cli, php-fpm, php-cgi, php-phpdbg and
libapache2-mod-php are now recorded as PhpRuntime.packages and reported
with an unknown patch level; extension and library packages
(php-mysql, php-pear, php-common, phpmyadmin) and paths do not count.
- php_finding returned early for a patched PHP_VERSION, so an end-of-life
package line installed next to it (e.g. php7) was never reported. The
version only returns a finding now; otherwise the lines are judged too.
Every finding has unit tests that fail on the previous code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two CI failures on PR #1265 (run 40834164415):
- Chunk 0 / SuiteCRM#2 (swarm): the post-load flush included
sys-ctl-hlth-runtime on every play host, the NFS server too, which never
gets sys-svc-container and therefore has no `container` CLI. The unit
died with FileNotFoundError: 'container'. The role entry now runs only
where a container runtime exists: in compose mode, or on swarm nodes
('svc-swarm-node' in group_names), the same role-entry host scoping that
sys-ctl-hlth-csp and sys-ctl-hlth-webserver use with IS_STACK_HOST.
Workers stay covered because PHP replicas can run there.
- Workspace (ubuntu): tests/external/repository/test_urls_reachable.py
probed the literal "https://hub.docker.com/v2" from
utils/docker/image/runtime.py:20 and got HTTP 404. The constant is gone;
the endpoint is written as one f-string at the call site, the way
utils/update/docker.py:104 and utils/update/source.py do, which the URL
test skips as a template. With the scheme now visible in the literal,
ruff reported the noqa: S310 on urllib.request.Request as unused.
Both test files that imported unittest twice (CodeQL) now use one
`from unittest import ...` form.
The run_once queue leak that spreads the stack host's post-load queue to
every host is fixed separately on the updater line (70ed8a2e98); when it
meets main, sys-ctl-hlth-runtime moves into sys-svc-container's queue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
