`process.env.X || <value>` is a second source for something the deployment
already renders, and the two drift without a sound. playwright.config.js read
`process.env.PLAYWRIGHT_ACTION_TIMEOUT || 30_000` while nothing ever set that
variable, so every run used the unscaled 30 s and no failure said so.
Normalising an absent variable stays allowed, because it substitutes nothing:
`|| ""`, `|| '[]'`, `|| false`, `|| 0` and falling through to another
`process.env` entry only let the caller test absence itself.
The six sites the rule found:
n8n and flowise built a Postgres port from `|| '5432'` while env.j2 renders it
from `lookup('database', application_id, 'port')`. Node throws
ERR_SOCKET_BAD_PORT on the resulting NaN, so an unfed port now names itself
instead of connecting somewhere plausible.
The bluesky broker did the same with BROKER_PORT, which compose.yml.j2 renders
unconditionally. Its INSECURE_TLS default was `"false"`, which is what `""`
already evaluates to.
matomo defaulted MATOMO_TRACKING_SCOPE to "sub"; the only comparison is against
"root", so `""` takes the identical branch.
moodle and simpleicons defaulted a name their playwright.env.j2 always renders.
simpleicons already asserts the value is truthy; moodle now fails on the scope
it cannot find rather than on a name nobody configured.
Four values the code itself owns keep a default under `// nocheck: env-default`:
PDS_INTERNAL_URL (which falls through to requireEnv, not to a literal),
HANDOFF_COOKIE_NAME, HANDOFF_COOKIE_MAX_AGE and PASSWORD_SALT_HASH_ROUNDS. No
role renders any of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Playwright declares the timeout option on locator.isVisible() and isHidden() as deprecated and ignored: both return immediately and never wait. Passing resolveTimeout(10_000) to one therefore changes nothing at runtime and only tells the reader a wait happened that never did. That reading is what let 7538b8a0b conclude two genuine call sites were fine, and what hid the false negative b97cf8888 had to fix.
Sixty call sites across fifty-eight role-local specs lose the argument and nothing else. Every added line in this commit is a bare isVisible(); a grep for added lines that are not one returns zero, so a behaviour change cannot have entered here. Files whose diff also alters waiting semantics are deliberately not part of this commit.
A repository lint added in b97cf8888 keeps the construct out. It matches isVisible and isHidden only, because isChecked, isDisabled, isEditable and isEnabled take a real timeout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The spec navigates through gotoOnion and carries no timeout literals of its
own, so the import had no consumer and failed eslint's no-unused-vars, which
took the whole CI lint job down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.
Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).
Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-deployment Matomo tracking scope, defaulting to per-subdomain:
- services.matomo.tracking.scope 'sub' (default) registers one Matomo site per
consumer subdomain with an isolated cookie; 'root' keeps the prior behaviour
of one shared site per registrable base domain with cross-subdomain linking.
- sys-front-inj-matomo derives matomo_site_domain (full host vs registrable
base) and the shared matomo_site_url from it; getSitesId / addSite and the
tracking snippet (setCookieDomain / setDomains / enableCrossDomainLinking) all
follow the scope.
- The matomo Playwright env emits MATOMO_TRACKING_SCOPE; the SitesManager test
matches the consumer's full host in 'sub' and its base domain in 'root'.
- variants.yml variant 0 sets scope=root so the matrix exercises both flavours
(every other deploy runs the sub default).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Break the single 457-line playwright.spec.js into topic files discovered by the
harness testMatch "**/*.@(spec|test).js", mirroring the pihole / gitea layout:
- _shared.js env vars, the setup hook, loginAsAdmin, attachDiagnostics,
hostOf / baseDomainOf and the MATOMO_TARGET_ROLES_JSON list
- playwright.spec.js CSP + canonical-domain smoke (filename kept; harness asserts it)
- auth.spec.js admin login/logout and biber denied at the admin surface
- tracking.spec.js SitesManager site registration and per-consumer _paq injection
- persona-flows.spec.js guest / biber / admin universal-logout flows
Test logic is unchanged: same titles, selectors, regexes, timeouts and
assertions (verified by title parity and an adversarial diff against HEAD). The
only behavioural delta is that the extracted loginAsAdmin waits for the username
field to be visible before filling, which the SitesManager login previously
skipped (hardening only). The duplicated admin-login form-fill and the old
beforeEach are now the shared loginAsAdmin and setupMatomoPage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The matomo injection registers one shared SitesManager site per registrable base
domain (addSite urls=proto://base_domain), so a subdomained consumer such as
logout.infinito.example is tracked under the infinito.example site, not its own.
The SitesManager test asserted the full canonical host as a substring of a site
URL, which never matched the base-domain site and failed in swarm.
Reduce the consumer's canonical_domain to its base domain (mirroring the regex in
sys-front-inj-matomo/vars/main.yml) and match it against each site's hostname
exactly. The exact match also drops the substring false positive, where a site at
evil-infinito.example would have satisfied infinito.example. Per-consumer tracker
presence stays covered by the separate per-role injection test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matomo 5.x rejects the SitesManager.getAllSites API call with 401 when only the browser session cookie is sent; it requires token_auth. Surface the admin API token into the Playwright env (MATOMO_API_TOKEN) and pass it in the request POST body so it stays out of the URL/logs. Mode-agnostic test/env change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removes prose block comments, JSDoc headers, and multi-line nocheck
rationale paragraphs added in the previous two commits — keeps only
the bare nocheck markers (which carry actual exception semantics
the lint test enforces) and the unchanged function/test bodies. No
behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the shared `assertInjectedAssetLoadsWithoutCspBlock(page, opts)`
helper in personas/utils/csp.js. It installs the CSP violation
observer, navigates the consumer URL with `waitUntil: networkidle`,
and asserts both:
1. the browser observed a successful 2xx/3xx response for the
requested resourceType(s) from one of the injector hosts, AND
2. no `securitypolicyviolation` event named one of those hosts as
its `blockedURI`.
Closes the loophole where the markup-only checks would pass even
when the consumer's CSP `style-src` / `script-src` did not whitelist
the injector host — the `<link>` / `<script>` was in the HTML but
the browser silently dropped the load.
Wires the helper into the three consumer-injection provider specs:
- web-svc-css: replaces the HTML-regex link check with the helper
(`stylesheet` resource type, css.* + cdn.* hosts).
- web-svc-logout: extends the existing `logout.js`-in-HTML check with
the helper (`script` resource type, cdn.* host). Adds CDN_BASE_URL
to the env template since logout.js is served from the CDN.
- web-app-matomo: extends the per-consumer tracker test with the
helper (`script` resource type, matomo canonical host) before the
existing `_paq` / `matomo.php` markup tokens, so the matomo.js
async fetch after DOMContentLoaded is verified end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- plugins/lookup/roles_with_service.py: resolve url.base via the tls
sublookup and return canonical_url alongside id/canonical_domain.
- web-app-matomo/files/playwright/playwright.spec.js: parameterised
matomo-tracker injection assertion per consumer over
MATOMO_TARGET_ROLES_JSON, using target.canonical_url.
- web-svc-logout: new files/playwright/playwright.spec.js + templates/
playwright.env.j2; checks universal-logout injection on every consumer
declared in LOGOUT_TARGET_ROLES_JSON. meta/services.yml gains nocheck
markers explaining why this provider has no user-facing surface to
assert on.
- tests/unit/plugins/lookup/test_roles_with_service.py: cover the new
canonical_url field via a stubbed tls sublookup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
