Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port and
X-Forwarded-Ssl were written out separately in the html, ws and upload
locations and in the Prometheus health probes, and the copies had drifted:
ws and upload never sent Port and Ssl, and the probes sent only Host, which
is how the Shopware readiness probe came to see plain http.
headers/forwarded.conf.j2 now holds the set and every one of those locations
includes it. The html location renders the same headers as before. ws and
upload additionally send Port and Ssl with the values the html location
already gives the same backend. The ready and live probes gain X-Real-IP and
X-Forwarded-For, and the live probe the three X-Forwarded headers as well.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Baserow's image mounts the MCP app on the ASGI root unconditionally, so
mcp.enabled only decides whether an endpoint record exists. The
application's own vhost kept routing /mcp/*: switched off it answered
401 Endpoint not found., and a refused credential is still a surface.
Switched on it is worse - the adapter narrows fourteen upstream tools to
four, and the publicly routed native path reaches all fourteen for
anyone holding the endpoint key. web-app-homeassistant had the same
shape, an exposure: internal surface reachable publicly. Both now 404
there and stay reachable on the container network.
The public surfaces get the edge limits the contract already declared.
Zones are written per role by the role that owns them, in the shape
web-app-prometheus uses, because one round-scoped file would drop another
application's zones on the next deploy. The rate is concurrent_requests
per second rather than an eighth limit key: a client allowed N in flight,
each bounded by timeout_seconds, cannot sustain more unless every call
returns instantly. A refusal is 429, not nginx's 503, because a client
that should back off must not read it as an outage. stream_seconds
finally binds something - the adapter holds no stream, the proxy does.
The shared location snippet's three timeouts become parameters defaulting
to the 900s they hard-coded, so no existing vhost changes.
Found by the disabled-state assertion 7d8e7e87ff added, which had never
run: its helper was never staged, so the two defects hid each other.
Penpot 0 of run 31543714113 hung every OIDC login for 900 s. The Penpot backend is a JVM app and JDK 26 offers `Upgrade: h2c` on a plaintext POST. The generic HTML location relayed that upgrade verbatim and hardcoded `Connection: upgrade`, so Keycloak answered 101 Switching Protocols and continued in HTTP/2 framing. nginx proxies HTTP/1.1 and cannot consume those frames: it emitted chunked response headers with status 400 and zero body bytes, then held the connection until proxy_read_timeout. The client died on "chunked transfer encoding, state: READING_LENGTH" at oidc.clj:459, and Playwright timed out at 150 s long before that.
Measured, not inferred. Against Keycloak 26.7.0 the same token POST, differing only in the three upgrade headers, returns 401 with a 93-byte body without the offer and 101 Switching Protocols followed by HTTP/2 frames with it. Through nginx carrying the pre-fix directives the h2c request reproduces the CI signature exactly - status 400, Transfer-Encoding chunked, zero body bytes, elapsed equal to proxy_read_timeout - while the same request through the patched config returns the complete 401. The non-h2c path is byte-identical across both configs at 379 header bytes and 93 body bytes, and real WebSockets take the default branch unchanged.
Two maps in http{} do the work: $proxy_upgrade blanks the Upgrade header for h2c so nginx omits it, and $connection_upgrade answers close. html.conf.j2 also stops hardcoding `Connection: upgrade` on every HTML request, which misstated the hop state on requests that carried no Upgrade header at all. ws.conf.j2 already paired `Upgrade $http_upgrade` with $connection_upgrade and moves to $proxy_upgrade so the pair stays coherent under the new h2c branch.
Refusing the upgrade also removes a proxy/backend desync surface rather than adding one.
The comment deletions are the diff-scoped comment lint un-grandfathering the touched files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenCloud's proxy mounts an unconditional https-redirect middleware: any
request carrying X-Forwarded-Proto other than https gets a 308, with no
config toggle. Onion vhosts are plaintext, so the app 308-looped on
onion nodes and the webserver health gate aborted the deploy.
Make X-Forwarded-Proto overridable in the shared proxy location
templates (default $scheme, byte-identical for every other role) and
have opencloud assert https towards its backend.
Verified on a fresh onion node: opencloud deploys, the health gate
passes, the SPA loads over the onion, and its well-known, OIDC env and
CSP all resolve to http://auth.<onion>. The SSO E2E still flakes on
transient Tor SOCKS errors mid-flow (client-side multi-hop OIDC), which
is a Tor path property, not an app defect.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).
- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
resolve client_max_body_size from the role's own server config
(lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
on the render tasks, so a leaked value from any other source is neutralized;
a role's own include params (#20) still override the reset (precedence-tested,
direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
(suppressable via '# nocheck: leaky-proxy-var').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The gated filer/master branch defaulted the proxy upstream to the entity, so in swarm it resolved to seaweedfs:8333 (the S3 API) instead of the admin oauth2-proxy. Set proxy_service_key=sso-proxy + proxy_internal_port (keycloak's services.sso_proxy.ports.internal.http) so swarm routes to seaweedfs_sso-proxy:4180. html.conf.j2 gains an optional internal_port kwarg (default empty = no-op for existing callers; compose ignores it and keeps local_port).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralise the nginx proxy_pass directive in one source of truth (utils/networks/proxy.py + plugins/lookup/proxy_pass.py). Compose keeps the literal 127.0.0.1:<local> upstream; swarm resolves the service over the shared overlay at request time (variable upstream + http resolver) so a rescheduling backend never blocks openresty startup. Converts the html/media/upload/ws location partials, the keycloak oauth2 and login-iframe endpoints, and the logout endpoint to the lookup; adds the swarm resolver and the front-proxy service_key plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).
proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.
Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).
proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.
Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Browser vendors deprecated X-Frame-Options/X-XSS-Protection/X-Download-Options/
X-Permitted-Cross-Domain-Policies; CSP (rendered from each role's
meta/server.yml via build_csp_header) is the modern replacement. To make the
SPOT actually win against upstream apps (e.g. Keycloak/Quarkus still emits
X-Frame-Options), the proxy now hides those headers on every location type.
Includes a fail-loud lint that catches any future add_header / Header set/add
re-introduction in .conf / .conf.j2 / .htaccess / .j2 / .jinja / .tmpl files.
- sys-svc-proxy: new upstream_strips.conf.j2 partial bundles HSTS + the 4
deprecated header strips; included in html/media/ws/upload + web-svc-logout
+ web-app-oauth2-proxy locations (nginx array-replacement inheritance forces
per-location placement)
- web-app-keycloak: meta/server.yml frame-ancestors allows the primary-domain
wildcard with the dynamic TLS protocol so dashboard silent-check-sso iframe
loads cleanly
- web-app-dashboard/playwright: drop isOidcSilentCheckNoise — root cause is
now fixed, no noise filter needed
- web-app-dashboard/nginx + web-app-nextcloud/nginx: remove the deprecated
add_header lines themselves
- tests + docs: new lint registers `deprecated-http-header` suppression rule
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related cleanups extracted from the in-flight req-013 work because
they touch shared config layers (filter plugins, group_vars/all,
unrelated role templates) and deserve their own atomic commit so the
req-013 PR stays focused on the joomla/bluesky integration.
1. `on_off` filter (plugins/filter/on_off.py + unit tests)
Replaces the verbose `{{ x | bool | ternary('on', 'off') }}` idiom
that nginx, msmtp, unbound etc. need for their config syntax.
Coercion mirrors Ansible's `bool` filter; unknown strings raise
so templates can't silently mis-render. Migrated call sites:
- roles/sys-svc-mail-msmtp/templates/msmtprc.conf.j2 (4x)
- roles/sys-svc-proxy/templates/location/html.conf.j2
2. `NETWORK_PUBLIC_DNS_RESOLVERS` SPOT (group_vars/all/08_networks.yml)
New canonical list of public DNS resolvers used wherever a role
needs runtime external-hostname lookups (nginx `resolver`
directives, unbound `forward-addr`, etc.). Default
`[1.1.1.1, 8.8.8.8]`. Migrated call sites:
- roles/web-app-mailu/templates/unbound.conf.j2 (forward-zone loop)
The IPv6 toggle in any nginx `resolver` directive comes from the
existing `NETWORK_IPV6_ENABLED` flag in the same file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.
This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references
Behavior, lookup API, and semantics remain unchanged.
https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup
https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
with explicit per-application TLS resolution.
Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles
This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.
https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
- Set X-Forwarded-Ssl dynamically based on TLS_ENABLED
- Enable KC_PROXY_HEADERS=xforwarded for proper reverse proxy support
- Ensure correct protocol handling for HTTP and HTTPS environments
https://chatgpt.com/share/696bbbed-5514-800f-8281-b8485d77cac7
Refactor proxy/webserver configuration variables to a consistent webserver_* naming scheme across roles. Replace legacy variables like proxy_extra_configuration, client_max_body_size, vhost_flavour, location_ws and ws_port with webserver_extra_configuration, webserver_client_max_body_size, webserver_vhost_flavour, webserver_websocket_location and webserver_websocket_port. Update NGINX vhost and location templates (html, upload, ws, basic, ws_generic) as well as callers (sys-front-inj-all, sys-stk-front-proxy, various web-app-* and web-svc-* roles) to use the new naming.
Tighten docker-compose Git repository handling by making docker_git_repository_pull depend on docker_git_repository_address being defined, a string and non-empty. This avoids accidental Git operations when the repository address is unset or of the wrong type.
Refactor the BigBlueButton role structure and fix deployment bugs: introduce 01_core.yml to orchestrate docker/proxy setup, database seeding, websocket map deployment, docker-compose overrides and admin/bootstrap logic in a single once-executed entrypoint. Rename supporting task files (02_docker-compose.yml, 03_administrator.yml, 04_dependencies.yml) and update tasks/main.yml to delegate via include_tasks with run_once_web_app_bigbluebutton. Improve Greenlight admin creation behavior by treating the 'Email has already been taken' error as a non-fatal, unchanged outcome and running user:set_admin_role as a fallback, both for the primary password and the OIDC starred-password path.
Also standardize vhost flavour selection for services like Mailu, Discourse, CDN, Collabora, Coturn, OnlyOffice, Simpleicons and web-svc-logout by explicitly passing webserver_vhost_flavour where needed and aligning client_max_body_size and websocket configuration with the new webserver_* variables.
Reference: ChatGPT conversation https://chatgpt.com/share/6931c530-bba8-800f-9997-dd61dc1d497b
This commit introduces a unified mechanism to configure client_max_body_size
for both HTML and upload locations in the sys-svc-proxy role. The directive
is now injected early in html.conf.j2 and moved to a dedicated block in
upload.conf.j2 to ensure consistent behavior for large file uploads such as
OpenProject attachments.
Additionally:
- Added client_max_body_size variable override from web-app-openproject (set to 30m).
- Reordered header includes to avoid duplicate injection.
- Improved comments and structure for better clarity.
Reference: https://chatgpt.com/share/691d873e-9b50-800f-ae70-baf8bf1e5454
- added new file roles/web-app-shopware/files/framework.yaml defining trusted_proxies and trusted_headers for Symfony
- mounted framework.yaml into /var/www/html/config/packages/ in docker-compose
- exposed new role vars SHOPWARE_FRAMEWORK_HOST/DOCKER for mounting path
- rendered framework.yaml via Ansible copy task with proper permissions
- adjusted env.j2 to set TRUSTED_PROXIES and TRUSTED_HOSTS dynamically from domains and networks
- added SHOPWARE_DOMAIN var to vars/main.yml
- removed inline framework.yaml creation from Dockerfile (now managed via mount)
- updated proxy template (html.conf.j2) to include X-Forwarded-Ssl header
- improved init.sh permission handling for shared volumes
See ChatGPT conversation for implementation details and rationale:
https://chatgpt.com/share/690d4fe7-2830-800f-8b6d-b868e7fe0e97
