Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port and
X-Forwarded-Ssl were written out separately in the html, ws and upload
locations and in the Prometheus health probes, and the copies had drifted:
ws and upload never sent Port and Ssl, and the probes sent only Host, which
is how the Shopware readiness probe came to see plain http.
headers/forwarded.conf.j2 now holds the set and every one of those locations
includes it. The html location renders the same headers as before. ws and
upload additionally send Port and Ssl with the values the html location
already gives the same backend. The ready and live probes gain X-Real-IP and
X-Forwarded-For, and the live probe the three X-Forwarded headers as well.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
b8759a363e closed this for html.conf.j2 and its message said 'every proxy
location', so the class read as done. Three proxying snippets never got
it: ws, upload and media.
They are not covered by inheritance. nginx inherits proxy_set_header as
an array, so a location declaring any of its own discards every one from
the enclosing block, and proxy_pass_request_headers defaults to on. A
grep for the strip found exactly one include site before this commit,
which means there was nothing above those locations to inherit either.
So every websocket, upload and image location forwarded whatever
X-Forwarded-User the client sent, to applications that authenticate by
believing that header. Reached through ws_generic (mastodon, espocrm,
odoo), nextcloud's signaling and whiteboard, peertube's websocket and
tracker, and collabora's /cool/.
The lint asks the file whether it proxies rather than keeping a list of
snippet names. It grants no exemption for an SSO branch: html.conf.j2
carries both, and an exemption keyed on the SSO reference would keep it
green after the strip was deleted from the other branch.
Penpot 0 of run 31543714113 hung every OIDC login for 900 s. The Penpot backend is a JVM app and JDK 26 offers `Upgrade: h2c` on a plaintext POST. The generic HTML location relayed that upgrade verbatim and hardcoded `Connection: upgrade`, so Keycloak answered 101 Switching Protocols and continued in HTTP/2 framing. nginx proxies HTTP/1.1 and cannot consume those frames: it emitted chunked response headers with status 400 and zero body bytes, then held the connection until proxy_read_timeout. The client died on "chunked transfer encoding, state: READING_LENGTH" at oidc.clj:459, and Playwright timed out at 150 s long before that.
Measured, not inferred. Against Keycloak 26.7.0 the same token POST, differing only in the three upgrade headers, returns 401 with a 93-byte body without the offer and 101 Switching Protocols followed by HTTP/2 frames with it. Through nginx carrying the pre-fix directives the h2c request reproduces the CI signature exactly - status 400, Transfer-Encoding chunked, zero body bytes, elapsed equal to proxy_read_timeout - while the same request through the patched config returns the complete 401. The non-h2c path is byte-identical across both configs at 379 header bytes and 93 body bytes, and real WebSockets take the default branch unchanged.
Two maps in http{} do the work: $proxy_upgrade blanks the Upgrade header for h2c so nginx omits it, and $connection_upgrade answers close. html.conf.j2 also stops hardcoding `Connection: upgrade` on every HTML request, which misstated the hop state on requests that carried no Upgrade header at all. ws.conf.j2 already paired `Upgrade $http_upgrade` with $connection_upgrade and moves to $proxy_upgrade so the pair stays coherent under the new h2c branch.
Refusing the upgrade also removes a proxy/backend desync surface rather than adding one.
The comment deletions are the diff-scoped comment lint un-grandfathering the touched files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenCloud's proxy mounts an unconditional https-redirect middleware: any
request carrying X-Forwarded-Proto other than https gets a 308, with no
config toggle. Onion vhosts are plaintext, so the app 308-looped on
onion nodes and the webserver health gate aborted the deploy.
Make X-Forwarded-Proto overridable in the shared proxy location
templates (default $scheme, byte-identical for every other role) and
have opencloud assert https towards its backend.
Verified on a fresh onion node: opencloud deploys, the health gate
passes, the SPA loads over the onion, and its well-known, OIDC env and
CSP all resolve to http://auth.<onion>. The SSO E2E still flakes on
transient Tor SOCKS errors mid-flow (client-side multi-hop OIDC), which
is a Tor path property, not an app defect.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The healthz, media, upload and ws location templates called proxy_pass
without internal_port, so in swarm they fell back to
services.<entity>.ports.internal.http. Apps whose entity service has no
internal http port (e.g. bluesky) hard-failed the nginx config render
with resolve_upstream: swarm upstream needs
services.<entity>.ports.internal.http. Forward internal_port (default '')
like html.conf.j2 already does; no-op where it is not passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralise the nginx proxy_pass directive in one source of truth (utils/networks/proxy.py + plugins/lookup/proxy_pass.py). Compose keeps the literal 127.0.0.1:<local> upstream; swarm resolves the service over the shared overlay at request time (variable upstream + http resolver) so a rescheduling backend never blocks openresty startup. Converts the html/media/upload/ws location partials, the keycloak oauth2 and login-iframe endpoints, and the logout endpoint to the lookup; adds the swarm resolver and the front-proxy service_key plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Browser vendors deprecated X-Frame-Options/X-XSS-Protection/X-Download-Options/
X-Permitted-Cross-Domain-Policies; CSP (rendered from each role's
meta/server.yml via build_csp_header) is the modern replacement. To make the
SPOT actually win against upstream apps (e.g. Keycloak/Quarkus still emits
X-Frame-Options), the proxy now hides those headers on every location type.
Includes a fail-loud lint that catches any future add_header / Header set/add
re-introduction in .conf / .conf.j2 / .htaccess / .j2 / .jinja / .tmpl files.
- sys-svc-proxy: new upstream_strips.conf.j2 partial bundles HSTS + the 4
deprecated header strips; included in html/media/ws/upload + web-svc-logout
+ web-app-oauth2-proxy locations (nginx array-replacement inheritance forces
per-location placement)
- web-app-keycloak: meta/server.yml frame-ancestors allows the primary-domain
wildcard with the dynamic TLS protocol so dashboard silent-check-sso iframe
loads cleanly
- web-app-dashboard/playwright: drop isOidcSilentCheckNoise — root cause is
now fixed, no noise filter needed
- web-app-dashboard/nginx + web-app-nextcloud/nginx: remove the deprecated
add_header lines themselves
- tests + docs: new lint registers `deprecated-http-header` suppression rule
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.
This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references
Behavior, lookup API, and semantics remain unchanged.
https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup
https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
with explicit per-application TLS resolution.
Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles
This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.
https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
Refactor proxy/webserver configuration variables to a consistent webserver_* naming scheme across roles. Replace legacy variables like proxy_extra_configuration, client_max_body_size, vhost_flavour, location_ws and ws_port with webserver_extra_configuration, webserver_client_max_body_size, webserver_vhost_flavour, webserver_websocket_location and webserver_websocket_port. Update NGINX vhost and location templates (html, upload, ws, basic, ws_generic) as well as callers (sys-front-inj-all, sys-stk-front-proxy, various web-app-* and web-svc-* roles) to use the new naming.
Tighten docker-compose Git repository handling by making docker_git_repository_pull depend on docker_git_repository_address being defined, a string and non-empty. This avoids accidental Git operations when the repository address is unset or of the wrong type.
Refactor the BigBlueButton role structure and fix deployment bugs: introduce 01_core.yml to orchestrate docker/proxy setup, database seeding, websocket map deployment, docker-compose overrides and admin/bootstrap logic in a single once-executed entrypoint. Rename supporting task files (02_docker-compose.yml, 03_administrator.yml, 04_dependencies.yml) and update tasks/main.yml to delegate via include_tasks with run_once_web_app_bigbluebutton. Improve Greenlight admin creation behavior by treating the 'Email has already been taken' error as a non-fatal, unchanged outcome and running user:set_admin_role as a fallback, both for the primary password and the OIDC starred-password path.
Also standardize vhost flavour selection for services like Mailu, Discourse, CDN, Collabora, Coturn, OnlyOffice, Simpleicons and web-svc-logout by explicitly passing webserver_vhost_flavour where needed and aligning client_max_body_size and websocket configuration with the new webserver_* variables.
Reference: ChatGPT conversation https://chatgpt.com/share/6931c530-bba8-800f-9997-dd61dc1d497b
