readCspString re-fetched the current URL through the Playwright API
request context and, when that failed, waited actionTimeout (30s) for a
<meta http-equiv="Content-Security-Policy"> that no template in this
repo emits. On a healthy runner that path works and costs one extra
request; when name resolution degrades it collapses, because the API
request context resolves DNS separately from the browser: the re-fetch
burns its 10s cap and the meta wait then drains whatever is left of the
caller's test budget.
That is what killed web-svc-logout's guest persona in run 32597248629,
job Pretix#0, whose swarm manager had a single resolver on 127.0.0.1
with dnsmasq no longer answering. Every guest test on that host inflated
to ~40s and the logout one, loading two external assets on top, died on
its 60s timeout having asserted nothing. The same role in Bluesky#0 and
YOURLS#0 of the same run, same role set, finished in 0.85s and 0.82s -
so this is a resilience fix, not a repair of a broken assertion.
gotoOnion now installs a page.on("response") recorder, so the header is
taken from the document the browser already received. That also covers
the surface reached through an SSO round trip, where no navigation
Response is in scope at the assertion - only guest.js holds one, biber,
admin and the suitecrm saml spec discard theirs. The re-fetch stays as
the fallback for a spec that navigates without gotoOnion; the meta
fallback is deleted.
The suite pins the header precedence (enforced over report-only) and
every guard in assertCspInjections: a disabled injector, a gate that
throws, an enabled injector without a base URL, a missing gate, a second
enabled injector, and the case-insensitive host match. Inverting the
precedence or dropping toLowerCase each trips exactly one test.
Verified against a headless Chromium and a local server counting
requests: the header survives a 302, no extra request is issued, a
missing injector host still fails the assertion, and a header-less page
falls back to exactly one re-fetch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.
Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).
Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Realtime apps (websockets/polling) keep the connection busy so page.goto never reaches networkidle; the asset loads the assertion checks are captured during navigation regardless, so swallow the navigation timeout instead of failing the spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five eslint errors blocking make lint-javascript:
* personas/guest.js, web-app-opencloud/playwright.spec.js,
web-app-opentalk/playwright.spec.js — re-thrown wrapped errors now
carry { cause: err } so the symptom keeps a link to the root cause.
* personas/utils/csp.js — drop let blockedHost = "" initial value;
the catch arm returns early, so the empty string was never read.
* web-app-wordpress/_shared.js — drop let match = null initial value;
both branches assign before the readers fire.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removes prose block comments, JSDoc headers, and multi-line nocheck
rationale paragraphs added in the previous two commits — keeps only
the bare nocheck markers (which carry actual exception semantics
the lint test enforces) and the unchanged function/test bodies. No
behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the shared `assertInjectedAssetLoadsWithoutCspBlock(page, opts)`
helper in personas/utils/csp.js. It installs the CSP violation
observer, navigates the consumer URL with `waitUntil: networkidle`,
and asserts both:
1. the browser observed a successful 2xx/3xx response for the
requested resourceType(s) from one of the injector hosts, AND
2. no `securitypolicyviolation` event named one of those hosts as
its `blockedURI`.
Closes the loophole where the markup-only checks would pass even
when the consumer's CSP `style-src` / `script-src` did not whitelist
the injector host — the `<link>` / `<script>` was in the HTML but
the browser silently dropped the load.
Wires the helper into the three consumer-injection provider specs:
- web-svc-css: replaces the HTML-regex link check with the helper
(`stylesheet` resource type, css.* + cdn.* hosts).
- web-svc-logout: extends the existing `logout.js`-in-HTML check with
the helper (`script` resource type, cdn.* host). Adds CDN_BASE_URL
to the env template since logout.js is served from the CDN.
- web-app-matomo: extends the per-consumer tracker test with the
helper (`script` resource type, matomo canonical host) before the
existing `_paq` / `matomo.php` markup tokens, so the matomo.js
async fetch after DOMContentLoaded is verified end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The guest scenario's `page.goto` previously swallowed every navigation
error via `.catch(() => null)`. When the role's public surface was
unreachable, the subsequent CSP probe in `readCspString` ran another
HTTP request without an explicit timeout and fell back to Playwright's
30s default. Stacked on top of the stalled goto this consumed the 60s
test-level limit and surfaced as `Test timeout of 60000ms exceeded`,
masking the real cause (backend unhealthy).
Re-throw with a precise message when goto rejects so the CI artifact
points at the unreachable URL inside the 30s navigation timeout, and
cap `page.request.get` inside `readCspString` at 10s so a single
stalled CSP fetch can no longer add 30s to the test budget.
Nine role specs (bigbluebutton, dashboard, discourse, keycloak,
matomo, matrix, moodle, openwebui, wordpress) each carried inline
copies of the same six CSP helpers and the `EXPECTED_CSP_DIRECTIVES`
constant. The bodies were semantically identical; only message
wording, brace style, and one captured-field-set differed.
Changes
-------
- Extends `roles/test-e2e-playwright/files/personas/utils/csp.js`
(already home to `assertCspInjections`) with
`installCspViolationObserver`, `readCspViolations`,
`parseCspHeader`, `assertCspResponseHeader`,
`assertCspMetaParity`, `expectNoCspViolations`, and the
`EXPECTED_CSP_DIRECTIVES` constant. The helpers are re-exported
through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Strips the inline definitions from the nine role specs and amends
each spec's `require("./personas")` destructuring with whichever
helpers the body still references.
Drift handling
--------------
- moodle's reduced violation-record shape (only `violatedDirective`
+ `blockedURI`) is replaced by the canonical 5-field record.
Moodle's downstream consumer asserts `.toEqual([])` on the array
only, so the extra fields are diagnostic-only — no behaviour
change.
- dashboard's 2-arg `expectNoCspViolations(page, label)` call site
is updated to the canonical 3-arg form
`expectNoCspViolations(page, null, label)`. The shared helper
guards `if (diagnostics && Array.isArray(diagnostics.cspRelated))`
before checking the diagnostics arm, so passing `null` keeps the
exact pre-migration behaviour for that call site.
- bigbluebutton and matrix had subtly different failure-message text
in `assertCspResponseHeader` and `expectNoCspViolations`. The
canonical wording is more descriptive; passing tests are
unaffected, failing tests get clearer diagnostics.
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` is
extended in two places: the invariants section names the full CSP
test surface and pins the "import from `./personas`, no inline copy"
rule, and the per-service catalogue's CSP / injectors row lists the
helpers by name.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- `markdownlint-cli2`: 0 errors on the touched doc.
- 0 inline definitions of any extracted helper / constant remain in
any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5