The access-log map read $request_uri, which keeps percent-encoding and
repeated slashes. A request such as /%70assword/reset/<token>,
/password%2Freset/<token> or //password//reset/<token> missed the
token-route expression, fell through to the plain-path branch and was
logged with the token in plain text, while the backend still decoded it
to its reset handler.
The map now reads $uri: decoded, slash-merged (merge_slashes is not
overridden anywhere) and without the query string, so the existing
expressions match every spelling of a token route. Trade-off: after an
internal redirect the log shows the internal URI; with the generic
vhost's error_page (ws_generic.conf.j2) a 5xx line now carries
/500.html instead of the client's path, with the status unchanged.
Nothing in the repository parses the proxy access log.
The unit test resolves the map against the variable it reads, modelling
$uri as nginx builds it, and adds the three encoded spellings above.
Negative control: with the map back on $request_uri, exactly those three
cases fail (3 failed, 8 passed).
Reported by CodeRabbit on #1322.
Part of #1192
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The front proxy logged the full request line: production had no
access_log, so nginx fell back to `combined` ($request, Referer,
$remote_user), and the debug format logged $request and the Referer.
Query-string tokens (?access_token=, ?key=, ?token=) and token routes
such as /password/reset/{token} and /i/confirm-email/{user}/{random}
ended up in plain text in the container log.
nginx.conf.j2 now declares a map from $request_uri to
$infinito_access_log_path:
- the path is written without the query string;
- the segments after a known token route (password/reset,
password-reset, reset-password, confirm-email,
lostpassword/reset/form, email-login, activate-account; matched
case-insensitively at any depth) are written as [redacted], so the
request, its status and timings stay visible without the token.
Both modes write an explicit access_log to /dev/stdout:
- production uses the new `redacted` format, laid out like `combined`
(remote address, time, "method path protocol", status, bytes,
user agent) with Referer and basic-auth user written as "-", so
combined-style parsers keep working;
- debug keeps its fields (host, timings, upstream time, lengths,
X-Forwarded-For, scheme, server name) but logs
"method path protocol" instead of $request and drops the Referer.
The path comes from $request_uri instead of $uri so internal rewrites
(error_page, try_files) do not replace the path the client requested.
Nothing in the repository parses the proxy access log (Wazuh, the
health checks and the Prometheus log_by_lua metrics do not read it),
so no consumer needs to change. The error log still carries
`request:` and `referrer:` on error-level entries; that is outside
this change.
A unit test renders nginx.conf.j2 in both modes and asserts that every
access_log uses a format declared there, that no format references
$request, $request_uri, $args, $query_string, $is_args, $arg_*,
$http_referer or $remote_user, and, by resolving the map as nginx does,
that query strings are dropped, token routes are redacted and
lookalike paths stay unchanged.
Verified live with web-svc-html in production mode (deploy failed=0, so
openresty accepted the config). A probe request to
/password/reset/<probe>?token=<probe> was logged as
"GET /password/reset/[redacted] HTTP/1.1" 404 62 "-" "curl/8.14.1",
and /plain/page as "GET /plain/page HTTP/1.1". Negative control: the
same request against the previous config logged the probe path token
and the query token in plain text. The debug format with the fix is
covered by the unit test only.
Closes#1192
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared proxy's nginx.conf derived worker_connections from the
openresty container's pids_limit. pids_limit caps processes and threads,
worker_connections caps simultaneous sockets per worker (client and
upstream), so with pids_limit 256 and one worker the whole host's proxy
held about 256 sockets, roughly 128 proxied requests in flight across
every vhost. Raising pids_limit to buy connections would loosen the
fork-bomb guard.
svc-prx-openresty now declares services.openresty.worker_connections
(4096 per worker, the common nginx production range, enough for several
chunk-heavy apps such as OpenWebUI at ~150-170 requests per page load).
WEBSERVER_PROXY_WORKER_CONNECTIONS reads it through lookup('config'),
without a default, so a missing key fails loudly. pids_limit stays 256.
WEBSERVER_PROXY_WORKER_RLIMIT_NOFILE is derived as 2x worker_connections
(8192), one descriptor for each client and upstream socket of a proxied
request, and nginx.conf renders it as worker_rlimit_nofile. The openresty
container pins its nofile ulimit (soft and hard) to the same value:
workers call setrlimit as root without CAP_SYS_RESOURCE, so they can only
lower the hard limit, and the inherited hard limit depends on the host
daemon (Docker >= 25 defaults to 1024:524288, DiD nests whatever the
outer container has). An explicit ulimit makes the rlimit succeed on
every host instead of logging a setrlimit alert.
The unit test pins worker_connections 4096, worker_rlimit_nofile 8192 and
the container ulimit, and fails when worker_connections follows
pids_limit again.
Live proof (compose deploy of web-svc-html, both plays failed=0): the
deployed nginx.conf renders worker_rlimit_nofile 8192 and
worker_connections 4096; openresty's master, worker and cache manager
run with "Max open files 8192 8192"; the openresty log carries no
setrlimit or "worker_connections are not enough" line.
Closes#1300
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared OpenResty nginx.conf took worker_processes and worker_connections
from WEBSERVER_WORKER_*, which are the min() of svc-prx-openresty's limits and
the current play's application_id. The file renders once, in the scope of
whichever role loads the proxy first, so that role's limits capped every vhost:
on feature/store-i18n web-svc-mirror (pids_limit 64) loaded it first, every
vhost got worker_connections 64, and openresty raised 555 "64
worker_connections are not enough while connecting to upstream" alerts while a
single openwebui page fires ~150-170 chunk requests (CI run 40979252521,
homeassistant#1). Main carries the same wiring.
Add WEBSERVER_PROXY_WORKER_PROCESSES and WEBSERVER_PROXY_WORKER_CONNECTIONS,
computed from svc-prx-openresty's own openresty service only (same
resource_filter fallbacks, at least one worker), and render nginx.conf from
them. WEBSERVER_WORKER_* keep pairing the proxy with the play for app-internal
nginx such as Nextcloud's. The values can only stay equal or rise.
Found by the i18n session's triage of homeassistant#1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Proxy:
- server_tokens off, X-Powered-By stripped from upstream responses,
HSTS max-age raised to one year.
- No global X-Content-Type-Options: under nosniff, ORB (step 3.4)
blocks no-cors cross-origin responses typed text/html, e.g. tracking
beacons answered with an empty 204.
PHP (CVE-2026-109618, CVE-2026-109619):
- New role sys-svc-compose-php installs a shared ini (expose_php Off,
session.upload_progress.enabled 0, fatal_error_backtraces 0 for
PHP 8.5, interim memory_limit -1) and a PHP-FPM [www] drop-in
(pm.max_requests = PHP_FPM_MAX_REQUESTS) once per stack host,
before 04_files renders any compose file.
- The host file names carry a sha1 prefix of the controller-side
source, so a content change alters every PHP app's compose.yml and
recreates it, also on remote deploys where compose_volumes cannot
read the stack host file and hashes the path instead.
- 20 PHP roles mount the ini as a compose/swarm config into their
image's scan dir; moodle, nextcloud and shopware also mount the FPM
drop-in. Magento keeps its own pool: its php-fpm.conf has no
include= and is rendered without notify.
- Joomla's php-upload.ini now loads as zzz-joomla-uploads.ini so its
own memory_limit wins: the media API compares Content-Length with
toBytes(memory_limit) and rejects every upload at -1.
TODOs record the memory_limit revert condition and accepted
trade-offs, Akaunting on PHP 8.1, FusionDirectory on PHP 7.4 and the
public Keycloak admin console.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
headers/buffers.conf.j2 lifts large_client_header_buffers to 32k for exactly one reason, stated in its own first line: Keycloak's tokens. It was included only inside each vhost server block, but nginx picks the buffer from the server it has selected while parsing, and that is the catch-all default server until the Host header arrives. Ruby's OIDC clients send Authorization first, so the /userinfo call carrying the access token hit nginx's built-in 8k instead.
Run 33698844864 shows the consequence twelve times per Decidim job: 'client sent too long header line: "Authorization: Bearer ..."', a 400 answered by nginx itself (UpstreamTime: -), omniauth reporting API Access Failed and redirecting to the app's own sign-in page, which is the URL the assertion reports. The same shape ends the mastodon and gitlab logout specs.
Including the partial at http level covers the catch-all, the 443 reject block and every server that forgets the include; the vhost include stays and sets identical values.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Penpot 0 of run 31543714113 hung every OIDC login for 900 s. The Penpot backend is a JVM app and JDK 26 offers `Upgrade: h2c` on a plaintext POST. The generic HTML location relayed that upgrade verbatim and hardcoded `Connection: upgrade`, so Keycloak answered 101 Switching Protocols and continued in HTTP/2 framing. nginx proxies HTTP/1.1 and cannot consume those frames: it emitted chunked response headers with status 400 and zero body bytes, then held the connection until proxy_read_timeout. The client died on "chunked transfer encoding, state: READING_LENGTH" at oidc.clj:459, and Playwright timed out at 150 s long before that.
Measured, not inferred. Against Keycloak 26.7.0 the same token POST, differing only in the three upgrade headers, returns 401 with a 93-byte body without the offer and 101 Switching Protocols followed by HTTP/2 frames with it. Through nginx carrying the pre-fix directives the h2c request reproduces the CI signature exactly - status 400, Transfer-Encoding chunked, zero body bytes, elapsed equal to proxy_read_timeout - while the same request through the patched config returns the complete 401. The non-h2c path is byte-identical across both configs at 379 header bytes and 93 body bytes, and real WebSockets take the default branch unchanged.
Two maps in http{} do the work: $proxy_upgrade blanks the Upgrade header for h2c so nginx omits it, and $connection_upgrade answers close. html.conf.j2 also stops hardcoding `Connection: upgrade` on every HTML request, which misstated the hop state on requests that carried no Upgrade header at all. ws.conf.j2 already paired `Upgrade $http_upgrade` with $connection_upgrade and moves to $proxy_upgrade so the pair stays coherent under the new h2c branch.
Refusing the upgrade also removes a proxy/backend desync surface rather than adding one.
The comment deletions are the diff-scoped comment lint un-grandfathering the touched files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.
- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
.onion-identity/hs (idempotent). The onion is a deploy-time input;
no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
installs the minted keys, torrc with one node HiddenService + a SOCKS
proxy restricted to loopback/DiD ranges. files/test.sh probes every
served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
skip the clearnet DNS sanity check for onion; raise nginx
server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
navigation/action timeouts so specs reach .onion over Tor and fail
fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.
Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Centralise the nginx proxy_pass directive in one source of truth (utils/networks/proxy.py + plugins/lookup/proxy_pass.py). Compose keeps the literal 127.0.0.1:<local> upstream; swarm resolves the service over the shared overlay at request time (variable upstream + http resolver) so a rescheduling backend never blocks openresty startup. Converts the html/media/upload/ws location partials, the keycloak oauth2 and login-iframe endpoints, and the logout endpoint to the lookup; adds the swarm resolver and the front-proxy service_key plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refines the explicit HTTPS default_server introduced in e9df1ae7b:
instead of aborting the TLS handshake (curl→000), complete TLS with
the primary-domain cert and return 421 Misdirected Request (RFC 9110
§15.5.20). When the cert SAN still covers the requested host but no
vhost is configured for it — the exact failure mode that
sys-svc-cln-domains produces between matrix-deploy variants — clients
now see a precise, RFC-correct signal instead of an opaque TLS error.
For hostnames not in any cert SAN (production with per-domain LE
certs and unrelated probes), curl still rejects the cert and reports
000, so the response is naturally dual-mode:
- SAN match + no vhost → 421 (deploy-incomplete signal)
- no SAN match → 000 (truly unknown host)
Updates scripts/tests/environment/08_deploy_performance.sh: the
matomo-only fresh-purged deploy keeps dashboard.infinito.example in
the shared CI cert's SAN list, so the assertion changes from
`assert_http_status 000` to `assert_http_status 421`. The matomo
assertion in 07_deploy_minimal.sh stays at 000 because matomo is
excluded from the cert SAN in that deploy's CA regen.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `listen 443 ssl default_server` block with `ssl_reject_handshake
on;` to default_server.conf.j2. Without this, when an SNI request hits
a hostname whose vhost has been purged (e.g. dashboard.infinito.example
after the fresh-purge sweep between matrix-deploy variants), openresty
silently falls onto whichever ssl-listening server happens to load
first and responds — bluesky-pds caught dashboard.infinito.example in
CI job 76714878217 and served its text/plain landing.
`ssl_reject_handshake on;` closes the TLS connection before any
certificate selection happens, so curl receives http_code "000" — the
exact contract the env-test `scripts/tests/environment/08_deploy_
performance.sh` already asserts after a matomo-only fresh-purged
deploy. Currently four devenv jobs (debian / centos / fedora /
manjaro) in workflow run 26089179898 fail with
`[FAIL] https://dashboard.infinito.example returned HTTP 301,
expected 000` because some downstream vhost answered instead.
Requires nginx >= 1.19.4 (openresty bundles 1.27 in this stack, so
the directive is available).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.
This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references
Behavior, lookup API, and semantics remain unchanged.
https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
Migrate all lookup() usages from the deprecated want= keyword argument to the
new strict positional want-path API. Update call order where required
(e.g. nginx_paths) and ensure consistency across vars, tasks, and templates.
https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
- switch CI DNS to CoreDNS-based *.localhost resolution (A/AAAA → loopback)
- set PRIMARY_DOMAIN to localhost for CI deploys
- add explicit DNS resolution assert for test.localhost
- deploy explicit default_server vhost returning 404 text/plain
- assert default vhost behavior via HTTP before app domains exist
https://chatgpt.com/share/6952cf2a-bbd4-800f-b2b8-c4baba66e301
- Add reusable Nginx include: roles/sys-svc-proxy/templates/headers/access_control_allow.conf.j2
(dynamic ACAO/credentials/methods/headers via role vars)
- Set global 'Vary: Origin' in nginx.conf.j2 to prevent cache poisoning
- CSP: allow Simple Icons via connect-src when feature is enabled
- Front proxy: rename vars to lowercase + flush handlers after config deploy
- Desktop: gate & load Simple Icons role; inject brand logos when enabled
- Bluesky + Logout: replace inline CORS with centralized include
- Simpleicons: public CORS (ACAO='*', no credentials), keep GET/OPTIONS, allow headers
- Taiga: adjust canonical domain to taiga.kanban.{{ PRIMARY_DOMAIN }}
- LibreTranslate: remove unused images/versions keys
Fixes: https://open.project.infinito.nexus/projects/cymais/work_packages/342/activity
Discussion: https://chatgpt.com/share/68da5e27-ffd4-800f-91a3-0ef103058d44
