Commit Graph
36 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 eda3bdbc35 refactor(sys-svc-proxy): set the upstream request headers in one snippet
Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port and
X-Forwarded-Ssl were written out separately in the html, ws and upload
locations and in the Prometheus health probes, and the copies had drifted:
ws and upload never sent Port and Ssl, and the probes sent only Host, which
is how the Shopware readiness probe came to see plain http.

headers/forwarded.conf.j2 now holds the set and every one of those locations
includes it. The html location renders the same headers as before. ws and
upload additionally send Port and Ssl with the values the html location
already gives the same backend. The ready and live probes gain X-Real-IP and
X-Forwarded-For, and the live probe the three X-Forwarded headers as well.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 17:15:22 +02:00
kevinveenbirkenbach 211d98aa88 fix(sys-svc-proxy): finish stripping forgeable identity headers
b8759a363e closed this for html.conf.j2 and its message said 'every proxy
location', so the class read as done. Three proxying snippets never got
it: ws, upload and media.

They are not covered by inheritance. nginx inherits proxy_set_header as
an array, so a location declaring any of its own discards every one from
the enclosing block, and proxy_pass_request_headers defaults to on. A
grep for the strip found exactly one include site before this commit,
which means there was nothing above those locations to inherit either.

So every websocket, upload and image location forwarded whatever
X-Forwarded-User the client sent, to applications that authenticate by
believing that header. Reached through ws_generic (mastodon, espocrm,
odoo), nextcloud's signaling and whiteboard, peertube's websocket and
tracker, and collabora's /cool/.

The lint asks the file whether it proxies rather than keeping a list of
snippet names. It grants no exemption for an SSO branch: html.conf.j2
carries both, and an exemption keyed on the SSO reference would keep it
green after the strip was deleted from the other branch.
2026-08-29 15:54:29 +02:00
kevinveenbirkenbach f5272fa85f feat(proxy): serve an MCP path only where it is meant to be public
Baserow's image mounts the MCP app on the ASGI root unconditionally, so
mcp.enabled only decides whether an endpoint record exists. The
application's own vhost kept routing /mcp/*: switched off it answered
401 Endpoint not found., and a refused credential is still a surface.
Switched on it is worse - the adapter narrows fourteen upstream tools to
four, and the publicly routed native path reaches all fourteen for
anyone holding the endpoint key. web-app-homeassistant had the same
shape, an exposure: internal surface reachable publicly. Both now 404
there and stay reachable on the container network.

The public surfaces get the edge limits the contract already declared.
Zones are written per role by the role that owns them, in the shape
web-app-prometheus uses, because one round-scoped file would drop another
application's zones on the next deploy. The rate is concurrent_requests
per second rather than an eighth limit key: a client allowed N in flight,
each bounded by timeout_seconds, cannot sustain more unless every call
returns instantly. A refusal is 429, not nginx's 503, because a client
that should back off must not read it as an outage. stream_seconds
finally binds something - the adapter holds no stream, the proxy does.

The shared location snippet's three timeouts become parameters defaulting
to the 900s they hard-coded, so no existing vhost changes.

Found by the disabled-state assertion 7d8e7e87ff added, which had never
run: its helper was never staged, so the two defects hid each other.
2026-08-29 11:05:09 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 a98b94045d fix(proxy): refuse an h2c upgrade instead of relaying it
Penpot 0 of run 31543714113 hung every OIDC login for 900 s. The Penpot backend is a JVM app and JDK 26 offers `Upgrade: h2c` on a plaintext POST. The generic HTML location relayed that upgrade verbatim and hardcoded `Connection: upgrade`, so Keycloak answered 101 Switching Protocols and continued in HTTP/2 framing. nginx proxies HTTP/1.1 and cannot consume those frames: it emitted chunked response headers with status 400 and zero body bytes, then held the connection until proxy_read_timeout. The client died on "chunked transfer encoding, state: READING_LENGTH" at oidc.clj:459, and Playwright timed out at 150 s long before that.

Measured, not inferred. Against Keycloak 26.7.0 the same token POST, differing only in the three upgrade headers, returns 401 with a 93-byte body without the offer and 101 Switching Protocols followed by HTTP/2 frames with it. Through nginx carrying the pre-fix directives the h2c request reproduces the CI signature exactly - status 400, Transfer-Encoding chunked, zero body bytes, elapsed equal to proxy_read_timeout - while the same request through the patched config returns the complete 401. The non-h2c path is byte-identical across both configs at 379 header bytes and 93 body bytes, and real WebSockets take the default branch unchanged.

Two maps in http{} do the work: $proxy_upgrade blanks the Upgrade header for h2c so nginx omits it, and $connection_upgrade answers close. html.conf.j2 also stops hardcoding `Connection: upgrade` on every HTML request, which misstated the hop state on requests that carried no Upgrade header at all. ws.conf.j2 already paired `Upgrade $http_upgrade` with $connection_upgrade and moves to $proxy_upgrade so the pair stays coherent under the new h2c branch.

Refusing the upgrade also removes a proxy/backend desync surface rather than adding one.

The comment deletions are the diff-scoped comment lint un-grandfathering the touched files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:22:55 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d492cc038e fix(opencloud): ride the node onion via an overridable X-Forwarded-Proto
OpenCloud's proxy mounts an unconditional https-redirect middleware: any
request carrying X-Forwarded-Proto other than https gets a 308, with no
config toggle. Onion vhosts are plaintext, so the app 308-looped on
onion nodes and the webserver health gate aborted the deploy.

Make X-Forwarded-Proto overridable in the shared proxy location
templates (default $scheme, byte-identical for every other role) and
have opencloud assert https towards its backend.

Verified on a fresh onion node: opencloud deploys, the health gate
passes, the SPA loads over the onion, and its well-known, OIDC env and
CSP all resolve to http://auth.<onion>. The SSO E2E still flakes on
transient Tor SOCKS errors mid-flow (client-side multi-hop OIDC), which
is a Tor path property, not an app defect.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 20:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f424ac1007 fix(swarm/proxy): forward internal_port in shared location templates
The healthz, media, upload and ws location templates called proxy_pass
without internal_port, so in swarm they fell back to
services.<entity>.ports.internal.http. Apps whose entity service has no
internal http port (e.g. bluesky) hard-failed the nginx config render
with resolve_upstream: swarm upstream needs
services.<entity>.ports.internal.http. Forward internal_port (default '')
like html.conf.j2 already does; no-op where it is not passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36ebf34b64 Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:13:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ffac55c1d5 fix(proxy): stop cross-role webserver_* var leak (config fields + task-scoping + front-proxy reset + lint)
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).

- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
  resolve client_max_body_size from the role's own server config
  (lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
  include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
  on the render tasks, so a leaked value from any other source is neutralized;
  a role's own include params (#20) still override the reset (precedence-tested,
  direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
  (suppressable via '# nocheck: leaky-proxy-var').

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d1c25b3edc fix(seaweedfs): route filer/master admin UIs to the sso-proxy sidecar in swarm
The gated filer/master branch defaulted the proxy upstream to the entity, so in swarm it resolved to seaweedfs:8333 (the S3 API) instead of the admin oauth2-proxy. Set proxy_service_key=sso-proxy + proxy_internal_port (keycloak's services.sso_proxy.ports.internal.http) so swarm routes to seaweedfs_sso-proxy:4180. html.conf.j2 gains an optional internal_port kwarg (default empty = no-op for existing callers; compose ignores it and keeps local_port).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c3f01a8640 feat(proxy): mode-aware proxy_pass SPOT lookup
Centralise the nginx proxy_pass directive in one source of truth (utils/networks/proxy.py + plugins/lookup/proxy_pass.py). Compose keeps the literal 127.0.0.1:<local> upstream; swarm resolves the service over the shared overlay at request time (variable upstream + http resolver) so a rescheduling backend never blocks openresty startup. Converts the html/media/upload/ws location partials, the keycloak oauth2 and login-iframe endpoints, and the logout endpoint to the lookup; adds the swarm resolver and the front-proxy service_key plumbing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbach b8759a363e fix(sys-svc-proxy): strip forgeable identity headers on every proxy location to close trusted-header injection (snipe-it) 2026-06-17 11:30:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc9d8b3b54 feat(web-app-baserow): trusted-header SSO bridge, drop LDAP
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).

proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.

Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:30:35 +02:00
kevinveenbirkenbach ed86a3d633 fix(sys-svc-proxy): strip forgeable identity headers on every proxy location to close trusted-header injection (snipe-it) 2026-06-14 13:29:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b48ab523c7 feat(web-app-baserow): trusted-header SSO bridge, drop LDAP
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).

proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.

Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 12:47:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e746368c20 fix(sys-svc-proxy): CSP is SPOT — strip deprecated security headers + add regression lint
Browser vendors deprecated X-Frame-Options/X-XSS-Protection/X-Download-Options/
X-Permitted-Cross-Domain-Policies; CSP (rendered from each role's
meta/server.yml via build_csp_header) is the modern replacement. To make the
SPOT actually win against upstream apps (e.g. Keycloak/Quarkus still emits
X-Frame-Options), the proxy now hides those headers on every location type.
Includes a fail-loud lint that catches any future add_header / Header set/add
re-introduction in .conf / .conf.j2 / .htaccess / .j2 / .jinja / .tmpl files.

- sys-svc-proxy: new upstream_strips.conf.j2 partial bundles HSTS + the 4
  deprecated header strips; included in html/media/ws/upload + web-svc-logout
  + web-app-oauth2-proxy locations (nginx array-replacement inheritance forces
  per-location placement)
- web-app-keycloak: meta/server.yml frame-ancestors allows the primary-domain
  wildcard with the dynamic TLS protocol so dashboard silent-check-sso iframe
  loads cleanly
- web-app-dashboard/playwright: drop isOidcSilentCheckNoise — root cause is
  now fixed, no noise filter needed
- web-app-dashboard/nginx + web-app-nextcloud/nginx: remove the deprecated
  add_header lines themselves
- tests + docs: new lint registers `deprecated-http-header` suppression rule

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 22:36:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 79409ebf10 test(tooling): integrate markdownlint-cli2, ansible-lint, galaxy-importer + fix the resulting baseline
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
  configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
  ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
  ensure_* functions (pip / npm with sudo / --user / --break-system-packages
  fallbacks). install_ansible_tools now also pulls ansible-lint and
  galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
  syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
  galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
  role, and treats `bare-name dependency` errors as success since our
  roles/ tree is not a Galaxy collection layout. All other Galaxy schema
  errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
  `make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
  now installs ansible-core + ansible-lint; ci-orchestrator wires both
  new lint workflows into lint-gate.

Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
  two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
  stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
  meta/main.yml plus _CANONICAL_COMPANY in
  tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.

Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
  across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
  TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
  list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
  renamed to "## User data resolution" (D3 self-label rule).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:46:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c8207a7635 refactor(filter,dns): add on_off filter and NETWORK_PUBLIC_DNS_RESOLVERS SPOT
Two related cleanups extracted from the in-flight req-013 work because
they touch shared config layers (filter plugins, group_vars/all,
unrelated role templates) and deserve their own atomic commit so the
req-013 PR stays focused on the joomla/bluesky integration.

1. `on_off` filter (plugins/filter/on_off.py + unit tests)
   Replaces the verbose `{{ x | bool | ternary('on', 'off') }}` idiom
   that nginx, msmtp, unbound etc. need for their config syntax.
   Coercion mirrors Ansible's `bool` filter; unknown strings raise
   so templates can't silently mis-render. Migrated call sites:
   - roles/sys-svc-mail-msmtp/templates/msmtprc.conf.j2 (4x)
   - roles/sys-svc-proxy/templates/location/html.conf.j2

2. `NETWORK_PUBLIC_DNS_RESOLVERS` SPOT (group_vars/all/08_networks.yml)
   New canonical list of public DNS resolvers used wherever a role
   needs runtime external-hostname lookups (nginx `resolver`
   directives, unbound `forward-addr`, etc.). Default
   `[1.1.1.1, 8.8.8.8]`. Migrated call sites:
   - roles/web-app-mailu/templates/unbound.conf.j2 (forward-zone loop)
   The IPv6 toggle in any nginx `resolver` directive comes from the
   existing `NETWORK_IPV6_ENABLED` flag in the same file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 17:11:30 +02:00
kevinveenbirkenbach ce64265cca Optimized docu in preparation for deploy script 2026-02-07 23:17:17 +01:00
kevinveenbirkenbach 3bb8c0ce30 refactor(tls): rename tls_resolve lookup to tls
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.

This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references

Behavior, lookup API, and semantics remain unchanged.

https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
2026-02-06 14:01:32 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 6af75fc8ce Renamed webserver_websocket_port to websocket_port 2026-02-05 17:02:17 +01:00
kevinveenbirkenbach 0b9f71463c refactor(tls): replace legacy tls filters with strict tls_resolve lookup
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup

https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
2026-01-23 00:36:46 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach f52f49118a Disable 'proxy_hide_header Strict-Transport-Security;' when TLS disabled 2026-01-22 16:26:37 +01:00
kevinveenbirkenbach 5fc951dc88 fix(proxy,keycloak): make X-Forwarded-Ssl conditional and enable proxy headers
- Set X-Forwarded-Ssl dynamically based on TLS_ENABLED
- Enable KC_PROXY_HEADERS=xforwarded for proper reverse proxy support
- Ensure correct protocol handling for HTTP and HTTPS environments

https://chatgpt.com/share/696bbbed-5514-800f-8281-b8485d77cac7
2026-01-17 17:42:14 +01:00
kevinveenbirkenbach 986f959696 Refactor webserver proxy variables and fix BigBlueButton deployment behavior
Refactor proxy/webserver configuration variables to a consistent webserver_* naming scheme across roles. Replace legacy variables like proxy_extra_configuration, client_max_body_size, vhost_flavour, location_ws and ws_port with webserver_extra_configuration, webserver_client_max_body_size, webserver_vhost_flavour, webserver_websocket_location and webserver_websocket_port. Update NGINX vhost and location templates (html, upload, ws, basic, ws_generic) as well as callers (sys-front-inj-all, sys-stk-front-proxy, various web-app-* and web-svc-* roles) to use the new naming.

Tighten docker-compose Git repository handling by making docker_git_repository_pull depend on docker_git_repository_address being defined, a string and non-empty. This avoids accidental Git operations when the repository address is unset or of the wrong type.

Refactor the BigBlueButton role structure and fix deployment bugs: introduce 01_core.yml to orchestrate docker/proxy setup, database seeding, websocket map deployment, docker-compose overrides and admin/bootstrap logic in a single once-executed entrypoint. Rename supporting task files (02_docker-compose.yml, 03_administrator.yml, 04_dependencies.yml) and update tasks/main.yml to delegate via include_tasks with run_once_web_app_bigbluebutton. Improve Greenlight admin creation behavior by treating the 'Email has already been taken' error as a non-fatal, unchanged outcome and running user:set_admin_role as a fallback, both for the primary password and the OIDC starred-password path.

Also standardize vhost flavour selection for services like Mailu, Discourse, CDN, Collabora, Coturn, OnlyOffice, Simpleicons and web-svc-logout by explicitly passing webserver_vhost_flavour where needed and aligning client_max_body_size and websocket configuration with the new webserver_* variables.

Reference: ChatGPT conversation https://chatgpt.com/share/6931c530-bba8-800f-9997-dd61dc1d497b
2025-12-04 18:31:09 +01:00
kevinveenbirkenbach 002f8de3ec feat(proxy): add configurable client_max_body_size for HTML and upload locations
This commit introduces a unified mechanism to configure client_max_body_size
for both HTML and upload locations in the sys-svc-proxy role. The directive
is now injected early in html.conf.j2 and moved to a dedicated block in
upload.conf.j2 to ensure consistent behavior for large file uploads such as
OpenProject attachments.

Additionally:
- Added client_max_body_size variable override from web-app-openproject (set to 30m).
- Reordered header includes to avoid duplicate injection.
- Improved comments and structure for better clarity.

Reference: https://chatgpt.com/share/691d873e-9b50-800f-ae70-baf8bf1e5454
2025-11-19 10:02:33 +01:00
kevinveenbirkenbach 493d5bbbda refactor(web-app-shopware): externalize trusted proxy and host configuration with mounted framework.yaml
- added new file roles/web-app-shopware/files/framework.yaml defining trusted_proxies and trusted_headers for Symfony
 - mounted framework.yaml into /var/www/html/config/packages/ in docker-compose
 - exposed new role vars SHOPWARE_FRAMEWORK_HOST/DOCKER for mounting path
 - rendered framework.yaml via Ansible copy task with proper permissions
 - adjusted env.j2 to set TRUSTED_PROXIES and TRUSTED_HOSTS dynamically from domains and networks
 - added SHOPWARE_DOMAIN var to vars/main.yml
 - removed inline framework.yaml creation from Dockerfile (now managed via mount)
 - updated proxy template (html.conf.j2) to include X-Forwarded-Ssl header
 - improved init.sh permission handling for shared volumes

See ChatGPT conversation for implementation details and rationale:
https://chatgpt.com/share/690d4fe7-2830-800f-8b6d-b868e7fe0e97
2025-11-07 02:48:49 +01:00
kevinveenbirkenbach 09a4c243d7 Add centralized include for Access-Control-Allow headers across proxy/service Nginx templates and align ACA vars for simpleicons task.
Ref: https://chatgpt.com/share/68dbf59c-f424-800f-aa27-787db52e260f
2025-09-30 17:22:28 +02:00
kevinveenbirkenbach 26a1992d84 Nextcloud/Talk: add Janus config & fix WebSocket proxying
Nginx: define 'map $http_upgrade $connection_upgrade' once in http{} and reuse; drop duplicate map from ws_generic vhost; tidy ws location headers/spacing. Nextcloud: add WS location for standalone signaling; render & mount Janus config (NAT 1:1, ICE enforce/ignore lists, libnice hardening); extend CSP (connect-src/frame-src for cloud & collabora, worker-src blob:); disable keeporsweep app; replace nginx reload handler with compose up; add NEXTCLOUD_HOST_JANUS_CONF_PATH and related vars.

Context: https://chatgpt.com/share/68db9f41-16ec-800f-9cdf-7530862f89aa
2025-09-30 11:14:15 +02:00
kevinveenbirkenbach f4cf55b3c8 Open WebUI OIDC & proxy fixes + Ollama preload + async-safe pull
- svc-ai-ollama:
  - Add preload_models (llama3, mistral, nomic-embed-text)
  - Pre-pull task: loop_var=model, async-safe changed_when/failed_when

- sys-svc-proxy (OpenResty):
  - Forward Authorization header
  - Ensure proxy_pass_request_headers on

- web-app-openwebui:
  - ADMIN_EMAIL from users.administrator.email
  - Request RBAC group scope in OAUTH_SCOPES

Ref: ChatGPT support (2025-09-23) — https://chatgpt.com/share/68d20588-2584-800f-aed4-26ce710c69c4
2025-09-23 04:27:46 +02:00
kevinveenbirkenbach b02d88adc0 Refactored server roles for better readability 2025-09-01 18:08:35 +02:00