Commit Graph
29 Commits
Author SHA1 Message Date
Alejandro Roman e6596d51f2 fix(stalwart): add the fix PR into main PR 2026-09-09 00:21:58 +04:00
kevinveenbirkenbachandClaude Opus 5 acc9e25c0f refactor(volumes): one canonical mount surface, context-wired lookups
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.

As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).

The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.

Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).

Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.

Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:16 +02:00
kevinveenbirkenbachandClaude Opus 5 b9669fcaa5 feat(healthcheck): give every container a probe, or state why it cannot have one
A container without a healthcheck is accepted the moment its process starts. The restore drill asks the host's health service for its verdict, so a service that came back empty passes as long as it is running - which is how an empty volume survives a drill unnoticed.

Twenty-one services get a probe and thirteen get a marker naming why none is possible: a one-shot client that exits, a build image with no service port, a media bridge on UDP. The flavor comes from each image's own layer history, read from the registry through the mirror, so no image had to be pulled to find out whether it ships curl, wget or a shell.

Two entries carry more weight than the rest. The central Postgres, which every replay of the recover chain lands in, had no probe at all; it gets the pg_isready line that sys-svc-rdbms already runs for the per-app engine. And the matrix runner starts systemd within a second while its inner daemon is still pulling every image of the deployment, so it counted as healthy long before anything served - it now waits for Synapse to answer.

No probe replaces one: nineteen of the twenty-one images ship no HEALTHCHECK, and the two that do already carried a declaration before this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:33 +02:00
kevinveenbirkenbachandClaude Opus 5 ed0f5af9db feat(playwright): make a blocked persona state why
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.

The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.

Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.

Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.

Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.

45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 20:36:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 945fbccc1e refactor(swarm): move service_replicas into services.yml
Replace the per-role service_replicas Jinja variable with a per-service
replicas: field in meta/services.yml. deploy.yml.j2 resolves it via
resource_filter (with entity fallback), the same path already used for
cpus/mem, so services.yml is the single source of truth for replica
overrides.

- 13 web roles migrated to per-service (or entity-level) replicas: 1
- matomo bootstrap -> replicas: 0, mastodon migrate -> replicas: 1
- minio/seaweedfs provider entries -> replicas: 1, which also pins the
  central single-writer object-store providers instead of relying on the
  topology default
- service_replicas removed entirely; dropped its dead reference from the
  compose-structure lint allowlist and updated requirement 023 wording

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d4ee9a85ba refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 02:36:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d337b37e81 fix(web-app-opencloud): pin to the manager and one replica for the embedded state
OpenCloud is a single binary with embedded NATS, decomposedfs metadata and search index: mmap/lock-based stores that corrupt over NFS and lose state when swarm reschedules a node-local volume to another node. default_placement: manager opts the volumes out of the NFS driver and makes the node deterministic; one replica because the stores are single-writer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:21 +02:00
kevinveenbirkenbachandClaude Opus 4.8 594109018a refactor(compose): publish every container port through the container_ports lookup
Take the host IP as an optional third element of each term
([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole
ports: block from declared ports.local/public (host) + ports.internal
(container). Convert every compose template across 84 roles, folding the
remaining raw var/numeric/specific-ip mappings (including mailu's per-interface
mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line.

Add the compose-ports-must-use-container-ports lint: it forbids any raw
"<host>:<published>:<container>" publish item under a ports: block and requires
every term to declare ports.internal. Remove
tests/lint/roles/test_swarm_internal_ports.py, now subsumed.

Fix a container_depends_on rendering regression. The lookup self-indented
(default indent=4), so the added 4-space placement + | indent(4) triple-indented
the block. Flip the lookup default to column 0 (consistent with
container_networks), make every call use the 4-space + | indent(4) form, and add
a lint enforcing it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 03:30:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 25f286d2bf feat(swarm): per-service internal ports and entity-named services
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cb27ff3756 refactor(networks): replace Jinja includes with compose_networks/container_networks lookup plugins
Convert the two registry-driven networks templates into pure-Python
rendering functions plus thin Ansible lookup plugins:

  roles/sys-svc-compose/templates/networks.yml.j2  -> {{ lookup('compose_networks') }}
  roles/sys-svc-container/templates/networks.yml.j2 -> {{ lookup('container_networks') | indent(4) }}

The rendering logic lives in utils/networks/render.py as pure-Python
functions that take a registry + closure-style config/database lookups,
making the logic directly unit-testable without a Jinja stub harness.
The lookup plugins are thin wrappers that resolve application_id,
DEPLOYMENT_MODE, and swarm.network.encryption from the templar, build
the service_registry, and forward to the render functions.

Migrate 233 include lines across 85 production templates. Convert
web-app-matrix's hand-coded MDAD networks block (default + conditional
postgres + conditional openldap) to the generic lookup; this adds an
openresty attachment when services.sso.enabled (harmless extra
interface, pinned in test_matrix_mdad_consumer_attaches_postgres_ldap_sso).

Mark local-DB / redis sidecars (sys-svc-rdbms/templates/services/,
svc-db-redis/templates/service.yml.j2) with `# nocheck: networks-literal`
because they intentionally attach to default-only.

Extend the lint to scan ALL roles/*/templates/**/*.yml.j2 (not only
compose.yml.j2) and to forbid the legacy include form; the marker
above is the documented escape hatch.

Coverage:
  - tests/unit/utils/networks/test_render.py (30 cases incl. encrypted=false literal pin)
  - tests/unit/plugins/lookup/test_compose_networks.py (5 cases incl. lookup_loader arg-schema)
  - tests/unit/plugins/lookup/test_container_networks.py (5 cases incl. lookup_loader arg-schema)
  - tests/integration/infrastructure/compose/test_networks_render.py (10 snapshot scenarios)
  - tests/integration/infrastructure/compose/test_network_includes.py (3 lint rules)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 18:42:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0ffbb73d72 refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).

* 25 unit tests for the lookup cover compose vs swarm prefix, custom
  flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
  fails on any raw `image:` line in roles/*/templates/*.j2 unless
  nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
  longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
  `{{ lookup('container_image', application_id, '<service_key>') }}`
  form. 26 roles use `custom=True` (mariadb, postgres, openldap,
  jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
  odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
  fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
  into consuming roles where application_id evaluates to the outer
  role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
  test_build_requires_image) recognise the standalone lookup as an
  implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
  the registry-prefixed image string because the custom var is now
  bare.

Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 00:17:11 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6e49fe74fb feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.

What's in this commit (210 paths):

Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
  top-level volumes:/configs:/secrets: section and per-service mount
  blocks from the canonical meta. container_volumes uses the SEMANTIC
  name as per-service mount source so it resolves against compose_volumes'
  top-level dict key (docker_name as source produced 'undefined volume'
  on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
  volume(application_id, name) returns a normalised single entry;
  volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
  _CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
  raw Jinja source: strings never enter the templar walk (root-cause of
  the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
  list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
  both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
  compose-only validate+reload behind a single when-block; swarm uses
  service-update instead (bind-mounts of a FILE pin source inode at
  task-create time, so atomic write-tmp+rename leaves the running task
  on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
  lookup('volume', application_id) | dict2items so NFS subdir
  pre-creation works under the new cache split.

Tests (12)
- New strict lints flagging inline mount sections in compose templates,
  single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
  call sites — all volume-mount surfaces MUST flow through container_volumes
  / compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
  accept both the legacy and the new lookup('volume', app, 'X').name forms
  during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
  nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
  alert_rules.yml bind mount.

Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
  with lookup('container_volumes', application_id, '<service>') |
  indent(N) per service + a single trailing lookup('compose_volumes',
  application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
  legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
  application_id, 'X').Y across 9 roles

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1d322f4e10 refactor(compose): replace lookup('applications') | compose_volumes(...) pipe with lookup('compose_volumes', ...)
Single SPOT lookup `plugins/lookup/compose_volumes.py` absorbs the
applications-lookup pipe step and auto-wires `DEPLOYMENT_MODE` plus
the `storage` group var from the templating context. Callers only
pass what genuinely varies per service (`application_id` and an
optional `extra_volumes` dict).

Behavior change (bug fix): the filter was always invoked with the
default `deployment_mode='compose'` because no caller passed it
explicitly, which silently skipped the swarm-NFS driver-opts
injection path. The lookup reads `DEPLOYMENT_MODE` from globals,
so swarm-NFS volumes now render the `driver: local` / `driver_opts:
type: nfs` block correctly. Callers can still override via
`deployment_mode=` / `storage=` kwargs when a template needs
non-default rendering.

Other coordinated changes:
- 81 template call sites migrated from pipe form to lookup form
  (single-line and multi-line shapes).
- `FilterModule` registration in plugins/filter/compose_volumes.py
  removed so the deprecated pipe form fails fast at render time.
  The underlying `compose_volumes()` function stays exported and is
  imported by the new lookup plus the existing 21-test filter unit
  test module.
- 10 new lookup unit tests pin the contract (term shape, kwargs
  pass-through, auto-wiring, override precedence, error cases).
- Integration test `test_compose_volumes_call_required.py` updated:
  asserts the new lookup form is present in every `compose.yml.j2`
  and adds a second test forbidding the deprecated pipe form
  anywhere under `roles/**/templates/`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:51 +02:00
kevinveenbirkenbachandClaude Opus 4.7 587a3a1796 feat(swarm): consolidate container_name + pull_policy into generic compose_only lookup
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.

Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
  `build:` with a harmless warning when `image:` is pre-pushed (via
  the existing handlers/swarm_registry_push.py flow in
  roles/sys-svc-compose). The actual swarm-rejected key is
  `pull_policy:` ("Additional property pull_policy is not allowed"),
  and 10 templates were silently broken because they declared inline
  `pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
  \!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
  the gating logic is identical (emit-key-or-nothing).

Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
  points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
  now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
  lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
  removed.

Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
  17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
  build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
  `container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
  deleted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a0f0e3ac03 feat(swarm): SPOT lookup gates container_name emission per deploy mode
container_name is illegal in swarm mode when replicas > 1 (docker rejects
with "services.deploy.replicas: can't set container_name and X as container
name must be unique"). All compose.yml.j2 templates now route through a
single lookup that suppresses the key in swarm mode and emits it in compose
mode.

SPOT: plugins/lookup/compose_container_name.py
  Reads DEPLOYMENT_MODE from the templating context, returns
  'container_name: "<name>"' under compose and an empty string under swarm.
  Call sites read:

      {{ lookup('compose_container_name', MY_CONTAINER) }}

  No explicit DEPLOYMENT_MODE pass-through needed.

Sweep: 82 templates rewritten across roles/web-app-*, roles/web-svc-*,
roles/svc-*, plus 4 partial includes (sso_proxy, redis, mariadb, postgres).
For container names assembled from multiple Jinja fragments the args use
tilde concat (X ~ '-' ~ Y) so the lookup receives a single resolved string
and the call stays a single well-formed Jinja expression.

Lint: tests/lint/ansible/jinja/test_no_raw_container_name.py blocks
  (a) raw 'container_name:' YAML keys in any roles/*/templates/**/*.j2,
  (b) split-Jinja regression where the outer braces close mid-call:
      {{ lookup('compose_container_name', A }}-{{ B) }},
  (c) embedded-Jinja regression inside a string argument:
      lookup(..., "{{ X }}_suffix") (removed in ansible-core 2.23).
  The (b) and (c) regexes accept either single- or double-quoted lookup
  name and scan the full file so multi-line shapes also surface. Suppress
  per-site with '# nocheck: container-name' on the same or preceding line.

Tests: tests/unit/plugins/lookup/test_compose_container_name.py pins the
lookup contract: compose vs swarm output, whitespace tolerance, default-to
compose when DEPLOYMENT_MODE is absent, AnsibleError on missing or excess
terms, non-string name coercion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 22cf6f38f1 refactor(env+make): rename static.env→default.env + sort + lint gates
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.

Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.

Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.

Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py        (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py   (# nocheck: markers exempt)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 70042a99f1 style(roles/playwright.env.j2): condense comments to single-line WHY-only
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ff7da26d30 refactor(playwright): migrate role specs to service-gating helper; split dashboard asset test per service
Removes direct process.env.<X>_SERVICE_ENABLED reads (and bespoke
per-role <ROLE>_OIDC_ENABLED / SERVICE_* / OIDC_ENABLED env flags) in
favor of skipUnlessServiceEnabled/isServiceEnabled from
roles/test-e2e-playwright/files/service-gating.js, per
docs/contributing/artefact/files/role/playwright.specs.js.md service-
gating contract.

Per-role changes:
- discourse, matrix, dashboard: replace process.env.<X>_SERVICE_ENABLED
  with the helper; drop now-redundant nocheck markers on the canonical
  env flags.
- taiga: drop TAIGA_OAUTH2_ENABLED / TAIGA_OIDC_ENABLED env lines;
  consume OAUTH2_SERVICE_ENABLED / OIDC_SERVICE_ENABLED via the helper.
- opentalk, opencloud, nextcloud: drop <ROLE>_OIDC_ENABLED (and
  NEXTCLOUD_LDAP_ENABLED) env lines; spec switches to
  isServiceEnabled('oidc'|'ldap').
- moodle: drop SERVICE_OIDC / SERVICE_LDAP / SERVICE_LAM in favor of
  the canonical OIDC_SERVICE_ENABLED / LDAP_SERVICE_ENABLED plus a new
  LAM_SERVICE_ENABLED rendered from 'web-app-lam' group membership.
- decidim: drop OIDC_ENABLED env line; SSO-button-visible scenario
  uses skipUnlessServiceEnabled('oidc').

Dashboard asset test split:
The previously bundled "dashboard loads core css, javascript,
simpleicons, and logo assets" test triple-gated (css/cdn/logout)
skipped the entire surface when any single shared service was off.
Split into six focused tests so disabling one service skips only its
specific assertion:
  - shared CSS injection (gate: css)
  - CDN-served role-local stylesheet (gate: cdn)
  - logout.js injection (gate: logout)
  - simpleicon-backed cards (gate: simpleicons; nocheck dropped)
  - role-core JS modules + logos (ungated, role-core)
  - iframe-sync postMessage handler (ungated, role-core)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:24:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4481976fb9 fix(playwright): two-pass OIDC Login link + PERSONA_BLOCKED for nextcloud/opencloud/minio
Two-pass Login link resolution in `personas/utils/keycloak.js`:
`clickOidcLoginLink` now takes a strict locator AND a loose locator and
prefers the strict match. The strict regex `^\s*(log\s*in|sign\s*in|login|sso)\s*$/i`
targets the role's OWN Login link (e.g. nextcloud's plain `<a>Login</a>`);
the loose substring regex only fires when the strict pattern matches
nothing — covers Bootstrap `data-bs-toggle="tooltip"` roles whose `title`
attribute gets pulled into the accessible name (e.g. dashboard's
"Login to Infinito.Nexus on infinito.example"). Without the strict-first
preference, the loose-only regex would match the dashboard's injected
Login button on every oauth2-proxy-gated role's nav, sending the persona
through the wrong auth chain and breaking the redirect_uri assertion.

PERSONA_BLOCKED for three roles whose auth chains can't be driven by the
generic persona helper, with bespoke tests already owning persona
coverage:

- web-app-nextcloud: `/login` reverse-proxies Keycloak's form on the
  nextcloud domain (no `/openid-connect/auth` URL marker). Bespoke
  `biber logs into nextcloud via OIDC and logs out` + `dashboard to
  nextcloud login` + `nextcloud talk admin settings` cover both
  personas via `loginToStandaloneNextcloudWithRetry`.
- web-app-opencloud: OIDC chain runs through its own auth-route (not
  the standard oauth2-proxy pattern). Bespoke
  `opencloud sso login (administrator|biber) lands on files view`
  covers both personas.
- web-app-minio: STS via OIDC, biber has no minio account. Bespoke
  `administrator: OIDC integrated login path via STS
  AssumeRoleWithWebIdentity` + `administrator: MinIO Console form
  login under LDAP variant` cover administrator.

Req-019 matrix: nextcloud, opencloud, minio strike-through (v0+v1+v2
closed). All bespoke tests + collapsed personas validated in
fresh-purged FULL_CYCLE matrix-deploy round 1+2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 02:17:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 faad246f65 fix(playwright/env): render CANONICAL_DOMAIN to unmask silent persona collapse
The shared biber/admin persona helpers in
roles/test-e2e-playwright/files/personas/*.js collapse the persona
scenario when either APP_BASE_URL or CANONICAL_DOMAIN is empty. The
twelve roles below shipped their env template without
CANONICAL_DOMAIN, so the helper fell into the auth-less collapse
branch and silently `test.skip()`-ed the persona scenarios despite
the roles having an actual Keycloak-backed auth surface. Render
CANONICAL_DOMAIN on every role whose spec drives a real persona so
the scenario runs end-to-end and a real regression cannot hide
behind a silent skip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 00:06:48 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c55ad69c41 refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).

* Document the rule in `docs/contributing/documentation.md` under
  "Comments". Code MUST describe the *why* in present-state language;
  contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
  `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
  `eslint.config.js`. Pure comment / docstring / log-message edits;
  no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
  enforce the rule going forward across `.js`, `.py`, `.yml`,
  `.yaml`, `.j2`, and `.sh` files. The legacy
  `test_no_stale_requirement_refs.py` is exempt as the only file
  whose purpose is to detect these very tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 db2a2d9638 refactor(playwright): move cross-service tests to provider specs
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.

Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
  declare `services.<svc>.{enabled, shared}` truthy in the merged
  applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
  parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
  assertion (dashboard tile reachability, prometheus `up=1`, matomo
  SitesManager registration). The matomo spec gains an explicit
  biber-denial test (request body validated against admin DOM markers
  per the body-marker rule); the prometheus admin-reach test drops
  the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
  biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
  `clickRoleTileFromDashboard` step and the `assertPrometheus*` /
  `assertMatomo*` cross-service probes are gone; `utils/{dashboard,
  prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
  literal `process.env.<KEY>` reads so the env-keys-used parity guard
  recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
  `DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
  `PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
  `MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
  helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
  globally exempts `dashboard` and `prometheus` from the
  services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
  unit tests for the lookup, including an explicit-contract test
  that pins the over-inclusive behaviour on raw Jinja strings (the
  lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
  rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
  `docs/contributing/artefact/files/role/playwright.specs.js.md`
  reflects the new persona shape and per-service catalogue, and is
  cleaned per `docs/contributing/documentation.md` (no SPOT self-
  labels, no req-references, em dashes replaced with complete
  sentences, multi-clause lines split onto separate lines, broken
  anchor `#two-personas-fixed-flow-` corrected).

Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cd2830507c test(lint): persona scenarios cannot silently SKIP on missing base URLs
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.

Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.

Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 16:30:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 44d51744a7 feat(playwright): migrate second batch of roles to shared persona library
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into another ~28 roles
(akaunting, baserow, bluesky, bookwyrm, bridgy-fed, decidim, discourse,
fider, friendica, fusiondirectory, gitea, hugo, jenkins, joomla, kix,
matrix, mattermost, mig, moodle, odoo, opencloud, peertube, pixelfed,
postmarks, sphinx, taiga, yourls, web-svc-libretranslate, web-svc-xmpp).
Each role gets the spec.js persona scenarios plus matching
meta/services.yml gate entries and playwright.env.j2 env wiring;
admin-flow callers pass a role-specific adminInteraction that drives
an actual management surface so the body is not a stub.

Reason: continuation of d3e5ebe44 — req 019 Rule 3 demands every
role exercise the canonical guest/biber/administrator flows; this
batch closes the long-tail role list against the central library so
per-role drift cannot reintroduce the stub-shaped bodies the new lint
forbids.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 16:09:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 dd565d8a9f feat(opencloud): dynamic ldap flag + track LDAP playwright coverage (req 018)
Migrate `web-app-opencloud/meta/services.yml` to the dynamic-flag form
for `ldap.enabled` ("{{ 'svc-db-openldap' in group_names }}") so the
flag follows the same convention as the other shared-service entries
and the matrix-deploy variants stay the single point of truth for
which branch the spec exercises. Variants 0 and 1 pin
`ldap.enabled: true` explicitly so the LDAP path stays on for both
of them; that is what the matrix is meant to cover.

Drop two stale env keys from `templates/playwright.env.j2`:
`OPENCLOUD_LDAP_ENABLED` and `OPENCLOUD_OPENTALK_URL` were declared
but never read by the spec — caught by the
`test_playwright_env_keys_used.py` lint. They will land again only
when an actual scenario consumes them.

Add requirement 018 (Playwright LDAP authentication coverage):
the policy is that every role with `ldap.enabled` and a Playwright-
drivable login surface MUST ship a separate LDAP scenario, gated via
`skipUnlessServiceEnabled('ldap')`. Today most specs only gate on
`oidc`, so a LDAP-only deploy (variant 1 here) skips every
authenticated test instead of validating LDAP. The requirement lists
the in-scope roles and the verification criteria.

Link the OpenCloud-specific follow-up from the role TODO so the gap
is visible from the role artefact tree, not only from the requirement
index.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 10:46:58 +02:00
kevinveenbirkenbachandClaude Opus 4.7 01e0937240 fix(iam): make OpenCloud and OpenTalk Playwright OIDC scenarios pass and add OpenTalk recorder
Iterates the role-local Playwright specs from a structural skeleton into
fully green end-to-end scenarios that exercise the SSO handoff for both
the administrator and the biber LDAP persona, against the actual
deployed stack, and adds optional meeting recording for OpenTalk.

OpenCloud (web-app-opencloud)
- Add Keycloak's URL to connect-src and frame-src in the role CSP via
  lookup('tls', 'web-app-keycloak', 'url.base') so the SPA can hit the
  OIDC discovery and authorize endpoints from the browser
- Switch to read-only LDAP semantics: PROXY_AUTOPROVISION_ACCOUNTS=false,
  GRAPH_USERNAME_MATCH removed, GRAPH_ASSIGN_DEFAULT_USER_ROLE=true
- Set OC_LDAP_DISABLE_USER_MECHANISM=none so the user-lookup filter
  stops referencing the OpenCloud-only `openclouduserenabled` attribute
  that the central LDAP schema does not define
- Pin the image to opencloudeu/opencloud:4.0.6
- Spec primes localStorage forceAllowOldBrowser, bypasses the "browser
  not supported" splash, submits via Enter, asserts the Files-view banner

OpenTalk (web-app-opentalk)
- Provision a dedicated confidential Keycloak client `opentalk-controller`
  via kcadm so the controller can introspect access tokens; the shared
  realm client stays public for the browser PKCE flow. The service
  account is granted the realm-management view-users / query-users roles
  for the Keycloak admin Web API user-search backend
- Inject an nginx vhost extension via webserver_extra_configuration that
  routes /v1/, /signaling, and /livekit on open.talk.<DOMAIN> to the
  controller and LiveKit upstreams; the default route stays on the
  frontend
- Fix the frontend env vars to upstream names (OIDC_ISSUER, BASE_URL=""),
  pin the frontend image to v2.8.1
- Spec races between auto-redirect and explicit "Sign in" CTA, submits
  via Enter, asserts the post-login Home-link in the dashboard
- Add OpenTalk recorder (registry.opencode.de/opentalk/recorder v0.16.2)
  plus RabbitMQ broker (rabbitmq:4.2.4); wire the controller to publish
  recording jobs on the `opentalk_recorder` queue and the recorder to
  consume them; provision a dedicated confidential `opentalk-recorder`
  Keycloak client for the recorder service account
- Reference Keycloak and the cross-app domain through lookup('tls', ...)
  and lookup('domain', ...) in meta/server.yml instead of literal
  `auth.<DOMAIN>` / `open.cloud.<DOMAIN>` strings

Cross-cutting
- Rename canonical hostnames to the <vendor>.<category> pattern:
  - web-app-opentalk:    talk.<DOMAIN>  -> open.talk.<DOMAIN>
  - web-app-bigbluebutton: meet.<DOMAIN> -> bbb.talk.<DOMAIN>
- Restructure both READMEs to the per-role README contract; drop emojis
  from headings; strip em dashes / history phrases / self-SPOT
  declarations from IAM.md / LDAP.md / TODO.md
- Replace technical galaxy_info descriptions with user-facing copy so
  the dashboard cards read clearly

Validated end-to-end via the Role Loop (deploy-fresh-kept-apps then
deploy-reuse-kept-apps with SERVICES_DISABLED=matomo,email): all
seven OpenTalk containers + the OpenCloud container come up healthy,
scripts/tests/e2e/rerun-spec.sh web-app-opencloud and
scripts/tests/e2e/rerun-spec.sh web-app-opentalk both finish with
`2 passed`, and `make test` is green (1604 + 162 tests).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 17:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 84058f0cba feat(iam): add web-app-opencloud and web-app-opentalk with shared OIDC + LDAP
Implements requirement 015. Both new roles join the central Keycloak
realm and consume the central OpenLDAP through the same patterns as
web-app-nextcloud, so users can move between Files (OpenCloud) and
Video (OpenTalk) without separate accounts.

OpenCloud (web-app-opencloud)
- Single-binary deploy of opencloudeu/opencloud:4.0.6 at open.cloud.<DOMAIN>
- External Keycloak as IdP (OC_EXCLUDE_RUN_SERVICES=idp,
  PROXY_AUTOPROVISION_ACCOUNTS=true, PROXY_ROLE_ASSIGNMENT_DRIVER=oidc)
- Direct OpenLDAP user/group source via OC_LDAP_*
- Co-deployable with web-app-nextcloud (next.cloud.<DOMAIN> stays put)
- WEB_OPTION_OPENTALK_URL surfaces the OpenTalk meeting launcher into
  the Files UI when web-app-opentalk is in scope

OpenTalk (web-app-opentalk)
- Controller + LiveKit + web-frontend + dedicated Redis + dedicated MinIO
  (controller refuses to start without an S3 backend) at talk.<DOMAIN>
- Uses shared svc-db-postgres for relational data and shared
  web-svc-coturn for STUN/TURN (no dedicated TURN provisioning)
- Native Keycloak integration via controller.toml [oidc.*] +
  [user_search] backend = "keycloak_webapi"; LDAP is consumed
  transitively through Keycloak's federation
- Frontend env exposes SHARED_FOLDER_OPENCLOUD_URL when web-app-opencloud
  is in scope, wiring the in-meeting OpenCloud file picker
- Bucket bootstrap is idempotent: an Ansible task waits for MinIO and
  creates the configured bucket via mc

Cross-cutting
- Extends PORT_BANDS public.relay to 65535 so LiveKit's media UDP range
  (60000-60999) can coexist with web-svc-coturn (20000-39999),
  web-app-bigbluebutton (40000-49999), and web-app-nextcloud's onboard
  TURN (50000-59999)
- All credentials (OIDC client secrets, LDAP bind, TURN auth_secret,
  MinIO root, LiveKit API, OpenCloud JWT/transfer) are sourced via
  lookup('config', ..., 'credentials.*'); none are committed in plain text
- Each role ships IAM/LDAP docs mirroring web-app-nextcloud and
  role-local Playwright specs that cover the administrator AND biber
  personas through OIDC

Validated end-to-end via the Role Loop (deploy-fresh-purged-apps then
deploy-reuse-kept-apps with SERVICES_DISABLED=matomo,email): all five
OpenTalk containers + the OpenCloud container come up cleanly, the
controller successfully completes OIDC discovery against Keycloak, and
both canonical hostnames return HTTP 200 through the central reverse
proxy. Browser-validated assertions (admin role mapping, full SSO logout
propagation, OpenCloud<->OpenTalk cross-launch from a real LDAP user)
remain as TODOs in each role's TODO.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 09:10:58 +02:00