Commit Graph
33 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 b83b361ea9 fix(web-app-bigbluebutton): run BBB v3.0.39 with the shared-notes server
The version-source update in #1146 moved the bigbluebutton/docker ref from
v3.0.23 to v3.0.39. That tag changes three things the role did not
handle, and it exposed a fourth defect in compose_file_args.

1. generate-compose renders etherpad and bbb-pads only behind
   USE_INSECURE_ETHERPAD and otherwise emits bbb-shared-notes-server with
   a `build:` block and no `image:` (upstream adds the image only after
   the tag, in 9a6a9a9a4c). compose_mods drops `build` only where an image
   is set, so the guard "Fail if any service still builds locally" stopped
   every BigBlueButton deploy (main run 39749899667, Nextcloud#11:
   `rc: 0`, `stdout: '    build:'`, bootstrap_compose_project.yml:35).
   meta/services.yml now pins alangecker/bbb-shared-notes-server:v3.0.39,
   compose_mods takes `pinned` (BBB_PINNED_SERVICES, every service the
   role config pins an image for) and drops their `build`, and the
   etherpad leftovers go (ep_redis_publisher patch, etherpad postgres
   entrypoint and initdb hook, etherpad database tasks, ETHERPAD_* env,
   etherpad_api_key secret).

2. The notes server connects to postgres once at start and exits with 2
   when it is not up yet; upstream gives it no restart policy. The
   override sets the role's restart policy, so it retries until postgres
   accepts connections.

3. Greenlight moves from v3.8.0 to v3.9.0.1, which sets
   `config.hosts = URL_HOST`, and the template passes
   `URL_HOST: https://${DOMAIN}`. Rails compares hostnames, so every
   request was blocked (`Blocked hosts: bbb.talk.infinito.test`, 403).
   Greenlight documents URL_HOST as the bare hostname; the override sets
   it to the role domain.

4. compose_file_args decided whether a role ships a compose override by
   templating "{{ application_id | abs_role_path_by_application_id }}"
   through the templar. ansible-core 2.21 does not render an untrusted
   string, so the check saw the literal and always returned False: the CA
   override generation and the pull handler never read BBB's
   compose.override.yml. That was harmless while compose.yml alone was
   valid; with the notes-server image living in the override, `compose
   config` failed ("has neither an image nor a build context"). The check
   now resolves the role path for the application id it is given. BBB is
   the only role that ships an override, so no other role's file list
   changes.

Existing installations drop the etherpad and bbb-pads containers on the
next `compose up --remove-orphans`; the old `etherpad` database stays in
PostgreSQL untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 16:56:48 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 9e11cdd37a fix(bigbluebutton): pin search_path so the cluster dump can be replayed
The DR drill in job 95883448699 could not restore the BBB instance from its own backup: baudolo-restore aborted on CREATE UNLOGGED TABLE public."user", whose GENERATED columns call public.immutable_lower_unaccent(). BBB defines that function with an unqualified unaccent() in its body, and pg_dumpall has replayed with search_path='' since PostgreSQL 10 (CVE-2018-1058). Postgres inlines the body while building the generated column, unaccent stops resolving, and ON_ERROR_STOP kills the whole replay.

The fallout was two red tests from one defect: the drill left the stack down (subjects are left down), so the svc-db-openldap CLI test five minutes later found its container gone and failed its bind.

A SET clause stops Postgres from inlining the function, so the body runs with its own resolved path and the empty search_path no longer reaches it. pg_dump carries the clause into the next dump, which is what makes the backup restorable. Reproduced and verified against postgres:17-alpine: proconfig empty before, 'search_path=public, pg_catalog' after, and the same GENERATED DDL then succeeds under search_path='' with 'Ärger' still folding to 'arger'.

The SQL file is the single spot for the logic. Its DO block is a no-op in the postgres database, so the same file first yields the database list and then patches each one - one task, one retry, no SQL in the playbook. The function is upstream BBB schema, absent from this repo, so patching it after deploy is the only lever we have.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 16:41:10 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 42449ebf0a fix(e2e): let the checks that needed a wait actually wait
Sixteen role specs decided something immediately after an action that had not settled, because locator.isVisible() discards the timeout it is handed. Three shapes: an assertion that could never fire, a login outcome read off the DOM before the app re-rendered, and a branch that selected a locator by whether an iframe had mounted yet.

The ones that could never fire are the worst of them. taiga test-seaweedfs polled for a rejection toast right after saveAction.click(), so a real upload rejection was swallowed and misreported as a SeaweedFS storage failure; friendica ldapauth decided whether to run its wrong-password test at the instant a fresh context finished goto, so a slow render skipped a security check silently. wordpress test-discourse-roundtrip picked titleBox by whether the Gutenberg iframe had mounted and, when it had not, bound the wrong one and then burned the full budget failing with a misleading message.

Every conversion keeps the control flow: the .catch(() => false) fallback stays, so a waitFor that throws still yields the boolean the caller expects. Twenty of the twenty-two waits use base 2000, one 3000, one 5000. resolveTimeout multiplies by five on an onion target, so the common case is ten seconds there. Across thirteen onion traces the largest wait of any kind that ever succeeded was 790 ms, so ten seconds is a twelvefold margin; the bases started higher and were pulled down because a swallowed deadline is a duration in disguise - where the optional element is normally absent, the full timeout is paid on every run, including green ones.

Not exercised: no spec was run. Three of these make a test stricter, so a job that turns red on them is surfacing a real failure that was previously hidden, not regressing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 14:01:43 +02:00
kevinveenbirkenbachandClaude Opus 5 c7ddf6eb24 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, each resolved by combining both sides rather than picking one.

compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.

daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.

The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 33e8c82414 fix(web-app-bigbluebutton): create the etherpad database during initdb
ueberDB's pg driver opens the named database but never creates it, and the role created it from ansible after the stack was already running. etherpad therefore crash-looped on every cold deploy -- seven FATAL: database "etherpad" does not exist between 14:07:47 and 14:08:48 in run 30699990912. Hook it into /docker-entrypoint-initdb.d instead, where upstream already creates bbb_graphql, hasura_app and greenlight, eight seconds before etherpad first connects.

Mounted 0555 on purpose: the postgres entrypoint sources a non-executable initdb hook into its own shell rather than running it, which would leak set -eu into the entrypoint. A separate hook rather than an entry in POSTGRES_MULTIPLE_DATABASES, so a database upstream adds later is not silently dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 18:32:20 +02:00
kevinveenbirkenbachandClaude Opus 5 d7e3aae20d test(e2e): skip OIDC specs when app and issuer live in different realms
BigBlueButton is never served over Tor (WebRTC needs UDP/ICE), but in an
onion deployment its issuer only exists as an onion address. The test
browser reaches clearnet or the onion net, never both, since the private
tor network has no exit — so the login round-trip timed out. A real Tor
Browser handles this fine, which is why the gate belongs in the spec and
not in the role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:51:17 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9c25f00cc3 fix(web-app-bigbluebutton): move etherpad DB off churning redis to postgres
Etherpad's ueberdb backend was pinned to the bbb redis service. During a
deploy that redis churns (RestartCount=2), taking etherpad's DB with it, so
etherpad crashes (RestartCount=1) and the health-container job fails. bbb's
own postgres stays stable (RestartCount=0) through the same deploy.

A boot wrapper rewrites settings.json dbType redis to postgres before the
upstream entrypoint runs; env carries the connection values; the etherpad
database is provisioned in the postgres password-sync task; compose adds a
postgres dependency. The ep_redis_publisher pub/sub client stays on redis
(its failures are non-fatal).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a21e1e23d6 fix(web-app-bigbluebutton): patch ep_redis_publisher so a redis reconnect can't crash etherpad
The p88f3f6b etherpad image ships ep_redis_publisher@0.0.3 whose retry() calls logger.error() with no logger in scope, so any redis reconnect throws ReferenceError and kills etherpad, which fails the shared hlth-container gate (observed on bbb-0 swarm and, collaterally, the nextcloud bundle that pulls the same image). Vendor the plugin index.js verbatim from the image plus a console-backed logger shim, and bind it read-only over the plugin file via meta/volumes.yml + stack_host_copy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 453ddb9170 fix(web-app-bigbluebutton): do not load BigBlueButton on an onion node
BigBlueButton is WebRTC (SRTP/UDP with ICE/STUN) which Tor cannot carry. Guard the role's core include on the resolved Keycloak domain not being an onion (alongside the existing run_once guard), disable the co-installed cloud_bbb Nextcloud app on onion at runtime, and skip the BBB Playwright specs on onion targets.

Relax the run_once schema test (test_service_core_first_task_run_once) to accept a when-list that contains the run_once guard, so a role can add extra include conditions while staying protected against duplicate execution.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 10:50:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c2c532c8e6 refactor(playwright): centralize dashboard-tile mechanics in web-app-dashboard SPOT; move specs into files/playwright/ subdir
Architecture
============
- New SPOT helper `roles/web-app-dashboard/files/playwright/dashboard-card-flow.js` (`runDashboardCardScenario(page, context, target)`): per `roles_with_service('dashboard')` target, asserts tile visible → click loads in `#main iframe` (outer URL gets `?iframe=` + iframe src matches canonical_domain) → header "Tab" button pops the embed into a fresh browser tab.
- `web-app-dashboard/files/playwright/playwright.spec.js` parametrised loop calls the helper for every `DASHBOARD_TARGET_ROLES_JSON` entry — the single owner of dashboard-tile mechanics across the whole codebase.

Spec relocation
===============
- 71 `roles/<role>/files/playwright.spec.js` → `roles/<role>/files/playwright/playwright.spec.js` (`git mv`, history preserved). odoo + pixelfed show as D+A in `git status` because of substantial content rewrites; coverage retained.
- New SPOT constant `ROLE_FILE_PLAYWRIGHT_SPEC` in `utils/roles/mapping.py`, registered as optional on `ROLE_TYPE_APPLICATION` (disallowed elsewhere, mirrors `ROLE_FILE_META_VARIANTS` pattern).
- `roles/test-e2e-playwright/tasks/run_one.yml`: staging task replaced with `with_fileglob` over `files/playwright/*.js` — roles can ship companion helpers (like `dashboard-card-flow.js`) next to the spec without further wiring.

Spec migrations (no more dashboard-iframe path in app specs)
============================================================
13 specs rewritten so the persona/SSO journey starts at the app URL directly, never through the dashboard tile or iframe:
* fider, mailu, mattermost, nextcloud, odoo, taiga: iframe-wrap removed (no more `frameLocator("#main iframe")`)
* bigbluebutton, discourse, kix, matrix, openwebui, peertube: dashboard goto preamble removed
* pixelfed: ~270 LOC of iframe-aware helpers/login/logout collapsed to direct-URL flow

Title cleanup
=============
- 64 specs: `"<persona>: dashboard → app → universal logout"` / `"administrator: dashboard → prometheus → app → universal logout"` → `"<persona>: app → universal logout"`.
- 13 specs: `"<persona>: dashboard to <app> ..."` → `"<persona>: <app> ..."`.

Dead-code cleanup
=================
- `dashboardBaseUrl` consts + beforeEach assertions removed from 11 specs (no functional uses; personas helper handles env validation).
- `DASHBOARD_BASE_URL=` env-template lines removed from fider, mattermost, odoo, taiga, pixelfed, kix.
- `DASHBOARD_SERVICE_ENABLED=` env-template lines removed (pixelfed retained for the still-gated spec).
- Stale "loads inside the iframe" / "FrameLocator parameter" comments cleaned in fider, odoo, prometheus, nextcloud.

Tooling
=======
- 5 lint Python files + 1 integration test now use `from utils.roles.mapping import ROLE_FILE_PLAYWRIGHT_SPEC`.
- `eslint.config.js` glob pattern + `roles/test-e2e-playwright/meta/main.yml` description + `roles/test-e2e-playwright/README.md` updated to new path.
- 25 markdown/docstring references in `docs/`, `roles/**/README.md`, `roles/**/TODO.md`, persona-helper docstrings updated to new path.
- `docs/contributing/artefact/files/role/playwright.specs.js.md`: file-placement section now documents the `files/playwright/` convention + `ROLE_FILE_PLAYWRIGHT_SPEC` SPOT constant; dashboard SPOT delegation explicit at 3 places.

Net diff: 124 files, +1142/-1564 (−422 LOC).
Integration suite: 183/183 green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:24:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5bdb921304 fix(req-019): pre-deploy Class 2 spec/env hardening (6 roles)
Static fixes for the 6 Class 2 roles flagged by CI run 25680106742,
applied without a fresh deploy:

- fediwall: PERSONA_BIBER_BLOCKED + PERSONA_ADMINISTRATOR_BLOCKED
  rendered (static Vue SPA, no auth surface; biber covered by the
  bespoke cross-fediverse scenario). Confidence 30% → 75%.
- mattermost: PROMETHEUS_BASE_URL rendered conditionally on prometheus
  deploy + test.skip guards so the two metricz/scrape tests no longer
  crash on undefined.
- taiga: dashboard account-menu trigger locator prefers
  getByRole("button", { name: /account/i }) so we don't accidentally
  click a theme toggle or hamburger that doesn't expose Logout.
- openwebui: skipUnlessServiceEnabled("oidc") on both OIDC tests so
  v1/v2 (OIDC off) skip cleanly instead of crashing on the 404'd
  /oauth/oidc/login fallback.
- bigbluebutton: skipUnlessServiceEnabled("oidc") + Greenlight
  ?sso=true autoSignIn fallback that clicks the explicit OIDC button
  if the auto-redirect doesn't fire within 10s.
- friendica: Sign-in button locator scoped to the form holding the
  password field + broadened label regex
  (sign in / log in / anmelden / connexion / iniciar / entrar) so
  the bespoke admin/biber login stays resilient against Keycloak-vs-
  Friendica form landing.

Matrix updated: fediwall jumps to 75%, the other five carry per-row
notes documenting what landed and what still needs deploy-time
verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 02:17:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4320d6b640 chore(lint): tighten prefer-template and convert remaining +-concats
Promote `prefer-template` to error in the spec/persona JS rule block
and convert the nine remaining `a + 'literal' + b` URL-builders to
template literals. Behaviour-equivalent; brings stylistic consistency
with the rest of the codebase, which already uses template literals
throughout, and prevents new `+`-concatenations from sneaking past
the lint floor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 18:18:21 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0e651e4a22 chore(lint): tighten ESLint rules and clean up flagged offenders
Promote four cheap rules in `eslint.config.js`:
* `no-unused-vars`: warn → error (with `argsIgnorePattern: "^_"`)
* `no-var`: error
* `eqeqeq`: error (always)
* `playwright/no-wait-for-navigation`: off → error
* `linterOptions.reportUnusedDisableDirectives`: off → warn

Add a repo-root CommonJS block so `eslint.config.js` itself sees Node
globals (`require`, `module`).

Fix every flagged offender:
* 138 stale persona-helper imports across 70 spec files
  (`isServiceEnabled`, `skipUnlessServiceEnabled`, `runAdminFlow`,
  ... — leftovers from the SPOT consolidation).
* 4 unused catch bindings → modern `catch {}` syntax.
* 2 unused `dashboardBaseUrl` declarations (minio, moodle).
* 1 unused `label` arg → renamed to `_label` to opt out per the
  argsIgnorePattern.
* 1 intentionally-kept legacy helper in web-app-wordpress's spec
  receives a per-line `eslint-disable-next-line` with reason; the
  preceding docstring already explained why it stays.
* 1 `page.waitForNavigation` in web-app-fediwall's Mastodon OIDC flow
  receives a per-line disable with reason — the navigation target
  depends on the runtime OIDC issuer; `waitForURL` would need a
  runtime-built pattern.
* 10 stale `// eslint-disable-next-line no-console` directives now
  surfaced by `reportUnusedDisableDirectives`; removed.

ESLint after this pass: 0 errors, 154 warnings (148 pre-existing
`playwright/expect-expect` plus 6 `playwright/no-force-option` that
were already inherited from `playwright/flat/recommended`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c55ad69c41 refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).

* Document the rule in `docs/contributing/documentation.md` under
  "Comments". Code MUST describe the *why* in present-state language;
  contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
  `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
  `eslint.config.js`. Pure comment / docstring / log-message edits;
  no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
  enforce the rule going forward across `.js`, `.py`, `.yml`,
  `.yaml`, `.j2`, and `.sh` files. The legacy
  `test_no_stale_requirement_refs.py` is exempt as the only file
  whose purpose is to detect these very tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6a92143d21 fix(playwright): add missing persona scenarios for Rule 3 compliance
Seven web-app-* specs failed the Rule 3 contract from
docs/contributing/artefact/files/role/playwright.specs.js.md:

  - bigbluebutton, decidim, discourse, matrix, openwebui, peertube
    each shipped `administrator:` and `biber:` test titles but no
    `guest:` test.
  - keycloak shipped four detailed admin / biber tests under non-
    canonical names (`master-realm super administrator…`, `normal-
    realm administrator logs in through account interface…`, etc.)
    and no `<persona>: <flow>`-prefixed scenarios at all.

None of the seven declared a `PERSONA_<persona>_BLOCKED=true`
opt-out in their `templates/playwright.env.j2`, and none qualify
for the auth-less collapse exception (every role has an OIDC auth
surface; keycloak IS the auth surface).

Changes
-------
- Each of the six guest-only-missing specs gains a single appended
  scenario:
    test("guest: public-landing → auth chain → never authenticated", …)
      → runGuestFlow(page);
- Keycloak gains all three persona scenarios (guest, biber,
  administrator) using the shared `runGuestFlow` / `runBiberFlow` /
  `runAdminFlow` runners. Existing detailed tests stay untouched
  alongside the persona contract.
- The `./personas` import in each spec is amended with whichever
  runners are now needed.

Verification
------------
- 420 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- 0 Rule-3 violators remain across all `web-app-*/files/playwright.spec.js`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 370f381d1c refactor(playwright): share Keycloak login-form helper
Fourteen role specs (bigbluebutton, discourse, fider, gitea, mailu,
matrix, mattermost, odoo, openwebui, peertube, postmarks, prometheus,
yourls, libretranslate) each carried an inline `performOidcLogin`
that drove the Keycloak login form. The bodies fell into four
clusters split between the role-based selector strategy
(`getByRole`) and the legacy input-name strategy
(`input[name='username']`); the existing shared
`performKeycloakLogin` already covered both via `.or()` chains but
also asserted post-login navigation, so it could not be reused
verbatim.

Changes
-------
- `personas/utils/keycloak.js` gains `performKeycloakLoginForm
  (target, username, password)` — form-fill + submit only, accepts
  `Page` OR `FrameLocator` so iframe-embedded Keycloak pages work
  without branching at the call site. The selectors are the union
  of all four prior cluster shapes (role-based getter chained with
  `.or(input-name fallback)`, sign-in matched against
  `/sign in|login|log in/i` plus `input#kc-login` family). The
  existing `performKeycloakLogin` and
  `performKeycloakLoginExpectingDenial` now delegate to the new
  helper, so their public API is unchanged.
- The 14 specs strip their inline `performOidcLogin` and amend
  `require("./personas")` with `performKeycloakLoginForm`. Every
  call site and comment reference is renamed in lockstep.

Drift handling
--------------
- Cluster 1 (5 specs) expected the accessible name `"Password"`
  exactly; the canonical regex `/^password$/i` is case-insensitive
  and matches every prior name plus lowercase variants — strictly
  more permissive.
- Cluster 1's sign-in button matched `/sign in/i`; the canonical
  matches `/sign in|login|log in/i` plus the legacy `input#kc-login`
  family — strictly broader.
- Cluster 1 awaited `usernameField.press("Tab")` without a `.catch`;
  the canonical wraps the press in `.catch(() => {})` so an
  unexpected Tab failure no longer aborts the flow.
- Cluster 2 / 3 used `expect(usernameField).toBeVisible({ timeout })`
  for the visibility wait; the canonical uses
  `usernameField.waitFor({ state: "visible", timeout })`, the
  Locator-method counterpart of the same wait — functionally
  equivalent.

Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
  baseline).
- 0 inline `performOidcLogin` definitions remain in any role spec,
  and 0 stale references to the legacy name in any context (call
  sites and comments are renamed in the same pass).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7018b3c991 refactor(playwright): share CSP header/meta/violation helpers
Nine role specs (bigbluebutton, dashboard, discourse, keycloak,
matomo, matrix, moodle, openwebui, wordpress) each carried inline
copies of the same six CSP helpers and the `EXPECTED_CSP_DIRECTIVES`
constant. The bodies were semantically identical; only message
wording, brace style, and one captured-field-set differed.

Changes
-------
- Extends `roles/test-e2e-playwright/files/personas/utils/csp.js`
  (already home to `assertCspInjections`) with
  `installCspViolationObserver`, `readCspViolations`,
  `parseCspHeader`, `assertCspResponseHeader`,
  `assertCspMetaParity`, `expectNoCspViolations`, and the
  `EXPECTED_CSP_DIRECTIVES` constant. The helpers are re-exported
  through `personas/utils/index.js` so specs that
  `require("./personas")` get them via the existing aggregator.
- Strips the inline definitions from the nine role specs and amends
  each spec's `require("./personas")` destructuring with whichever
  helpers the body still references.

Drift handling
--------------
- moodle's reduced violation-record shape (only `violatedDirective`
  + `blockedURI`) is replaced by the canonical 5-field record.
  Moodle's downstream consumer asserts `.toEqual([])` on the array
  only, so the extra fields are diagnostic-only — no behaviour
  change.
- dashboard's 2-arg `expectNoCspViolations(page, label)` call site
  is updated to the canonical 3-arg form
  `expectNoCspViolations(page, null, label)`. The shared helper
  guards `if (diagnostics && Array.isArray(diagnostics.cspRelated))`
  before checking the diagnostics arm, so passing `null` keeps the
  exact pre-migration behaviour for that call site.
- bigbluebutton and matrix had subtly different failure-message text
  in `assertCspResponseHeader` and `expectNoCspViolations`. The
  canonical wording is more descriptive; passing tests are
  unaffected, failing tests get clearer diagnostics.

Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` is
extended in two places: the invariants section names the full CSP
test surface and pins the "import from `./personas`, no inline copy"
rule, and the per-service catalogue's CSP / injectors row lists the
helpers by name.

Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
  baseline).
- `markdownlint-cli2`: 0 errors on the touched doc.
- 0 inline definitions of any extracted helper / constant remain in
  any role spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 122c82910b refactor(playwright): share decodeDotenvQuotedValue + normalizeBaseUrl
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.

Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
  the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
  re-exported through `personas/utils/index.js` so specs that
  `require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
  `./personas` instead. Specs that only reference one of the two
  carry only that one in the destructuring (no unused-import noise).

Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.

Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
  baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
  definitions remain in any role spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 239c0b71e6 feat(playwright): wire bigbluebutton + fediwall to shared persona library
Wires runGuestFlow / runBiberFlow / runAdminFlow into bigbluebutton and
fediwall, plus matching meta/services.yml gate entries and
playwright.env.j2 env wiring. fediwall's adminInteraction drives a real
management surface so the admin-flow body is not a stub; bigbluebutton
imports isServiceEnabled for follow-on work.

Reason: continuation of d3e5ebe44 / 44d51744a / 735ae5e29 — finishing
the long-tail role list against the central persona library so the
new lint cannot find a stub-shaped body anywhere.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 16:14:06 +02:00
kevinveenbirkenbach 490e2cd541 test(web-app-bigbluebutton): add universal /logout + logged-out assertion
Rewrites both admin and biber OIDC scenarios to end in a clearly logged-out
state: bbbLogout hits the universal /logout endpoint (intercepted by the
app's nginx vhost and proxied to web-svc-logout to terminate Greenlight and
Keycloak in one hop), then assertLoggedOut polls /rooms for the
unauthenticated sign-in affordance. Scenarios renamed from "...lands on
greenlight" to "...OIDC login and logout" to reflect the final-state
coverage required by the role-spec MUSTs.
2026-04-22 22:06:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e8122b165 test(web-app-bigbluebutton): add playwright e2e spec
Covers CSP header enforcement, canonical-domain surfacing, and the
dashboard → Greenlight OIDC login flow. Passes against a reuse-kept
deploy.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 20:18:57 +02:00
Environment TestandClaude Sonnet 4.6 c1f54d8b36 revert(bigbluebutton): restore changed tracking in greenlight_ensure_admin
db3df8ec dropped ||= and role changed=true to fix CodeQL
useless-assignment alerts, but that broke the CHANGED/NOOP
output when only provider, language, or role changed.

Reverts the logic introduced by bad AI consulting.

Refs #260 #261

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 11:43:30 +02:00
Environment TestandClaude Sonnet 4.6 db3df8ec53 fix(bigbluebutton): resolve CodeQL useless-assignment alerts #260 #261
Remove useless assignments to changed before the unconditional
changed = true at the password block. Also fix ||= short-circuit bug
that skipped ensure_attr! when changed was already true.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 12:11:51 +02:00
Environment TestandClaude Sonnet 4.6 750c110f45 fix(docs): apply documentation rules, fix URLs, and remove orphaned references
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see  #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 00:38:42 +02:00
kevinveenbirkenbach e1d572c37d fix(bigbluebutton/greenlight): create admin via rails runner with required fields
https://chatgpt.com/share/697a3457-92a8-800f-8df7-810a2e177c90
https://chatgpt.com/share/697a3470-764c-800f-aa26-5a4bc8298905
2026-01-28 17:07:41 +01:00
kevinveenbirkenbach 563d5fd528 Huge role refactoring/cleanup. Other commits will propably follow. Because some bugs will exist. Still important for longrun and also for auto docs/help/slideshow generation 2025-07-08 23:43:13 +02:00