mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 03:38:23 +00:00
kevinveenbirkenbachandClaude Opus 4.7
c2c532c8e6
Architecture
============
- New SPOT helper `roles/web-app-dashboard/files/playwright/dashboard-card-flow.js` (`runDashboardCardScenario(page, context, target)`): per `roles_with_service('dashboard')` target, asserts tile visible → click loads in `#main iframe` (outer URL gets `?iframe=` + iframe src matches canonical_domain) → header "Tab" button pops the embed into a fresh browser tab.
- `web-app-dashboard/files/playwright/playwright.spec.js` parametrised loop calls the helper for every `DASHBOARD_TARGET_ROLES_JSON` entry — the single owner of dashboard-tile mechanics across the whole codebase.
Spec relocation
===============
- 71 `roles/<role>/files/playwright.spec.js` → `roles/<role>/files/playwright/playwright.spec.js` (`git mv`, history preserved). odoo + pixelfed show as D+A in `git status` because of substantial content rewrites; coverage retained.
- New SPOT constant `ROLE_FILE_PLAYWRIGHT_SPEC` in `utils/roles/mapping.py`, registered as optional on `ROLE_TYPE_APPLICATION` (disallowed elsewhere, mirrors `ROLE_FILE_META_VARIANTS` pattern).
- `roles/test-e2e-playwright/tasks/run_one.yml`: staging task replaced with `with_fileglob` over `files/playwright/*.js` — roles can ship companion helpers (like `dashboard-card-flow.js`) next to the spec without further wiring.
Spec migrations (no more dashboard-iframe path in app specs)
============================================================
13 specs rewritten so the persona/SSO journey starts at the app URL directly, never through the dashboard tile or iframe:
* fider, mailu, mattermost, nextcloud, odoo, taiga: iframe-wrap removed (no more `frameLocator("#main iframe")`)
* bigbluebutton, discourse, kix, matrix, openwebui, peertube: dashboard goto preamble removed
* pixelfed: ~270 LOC of iframe-aware helpers/login/logout collapsed to direct-URL flow
Title cleanup
=============
- 64 specs: `"<persona>: dashboard → app → universal logout"` / `"administrator: dashboard → prometheus → app → universal logout"` → `"<persona>: app → universal logout"`.
- 13 specs: `"<persona>: dashboard to <app> ..."` → `"<persona>: <app> ..."`.
Dead-code cleanup
=================
- `dashboardBaseUrl` consts + beforeEach assertions removed from 11 specs (no functional uses; personas helper handles env validation).
- `DASHBOARD_BASE_URL=` env-template lines removed from fider, mattermost, odoo, taiga, pixelfed, kix.
- `DASHBOARD_SERVICE_ENABLED=` env-template lines removed (pixelfed retained for the still-gated spec).
- Stale "loads inside the iframe" / "FrameLocator parameter" comments cleaned in fider, odoo, prometheus, nextcloud.
Tooling
=======
- 5 lint Python files + 1 integration test now use `from utils.roles.mapping import ROLE_FILE_PLAYWRIGHT_SPEC`.
- `eslint.config.js` glob pattern + `roles/test-e2e-playwright/meta/main.yml` description + `roles/test-e2e-playwright/README.md` updated to new path.
- 25 markdown/docstring references in `docs/`, `roles/**/README.md`, `roles/**/TODO.md`, persona-helper docstrings updated to new path.
- `docs/contributing/artefact/files/role/playwright.specs.js.md`: file-placement section now documents the `files/playwright/` convention + `ROLE_FILE_PLAYWRIGHT_SPEC` SPOT constant; dashboard SPOT delegation explicit at 3 places.
Net diff: 124 files, +1142/-1564 (−422 LOC).
Integration suite: 183/183 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
127 lines
7.5 KiB
Markdown
127 lines
7.5 KiB
Markdown
# Playwright Tests 🎭
|
|
|
|
This page covers Playwright end-to-end test requirements, runner integration, and file contracts.
|
|
For implementation guidance when writing or updating role-local files, see
|
|
[Agent `playwright.spec.js`](../../../agents/files/role/playwright.spec.js.md) and
|
|
[Agent `playwright.env.j2`](../../../agents/files/role/playwright.env.j2.md).
|
|
For what the `files/playwright/playwright.spec.js` of a role MUST contain, see
|
|
[`playwright.spec.js` (role file rules)](../../artefact/files/role/playwright.specs.js.md).
|
|
|
|
## Framework 🧰
|
|
|
|
- [Playwright](https://playwright.dev/) runs inside Docker via the shared `test-e2e-playwright` role.
|
|
- There is no standalone `make` target for Playwright. Tests run as part of local deploy flows
|
|
(e.g. `make deploy-fresh-purged-apps APPS=web-app-matomo FULL_CYCLE=true`). For spec-only
|
|
iteration after the first successful deploy, see [Playwright Spec Loop](../../../agents/action/iteration/playwright.md).
|
|
|
|
## When to Write ✍️
|
|
|
|
- You MUST provide Playwright tests for every `web-*` role.
|
|
- For when a spec MUST be added or updated (including the `javascript.js` / `style.css` trigger), see the Scenarios section of [`playwright.spec.js`](../../artefact/files/role/playwright.specs.js.md#scenarios-).
|
|
|
|
## Role-Local Files 📁
|
|
|
|
Every Playwright-enabled role MUST provide exactly these two files:
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `files/playwright/playwright.spec.js` | Test scenarios executed by the Playwright runner |
|
|
| `templates/playwright.env.j2` | Rendered `.env` passed to the Playwright container |
|
|
|
|
`playwright.config.js` MUST remain centralized in `roles/test-e2e-playwright/files/` and `package.json` in `roles/test-e2e-playwright/templates/package.json.j2` (rendered per-deploy into the staging dir). You MUST NOT duplicate or override either per role.
|
|
|
|
The `@playwright/test` version pinned into the rendered `package.json` comes from `playwright.version` in `roles/test-e2e-playwright/meta/services.yml` via the `image_version` filter. That YAML value is the single source of truth for the Playwright version.
|
|
|
|
## CI Image Source 🪞
|
|
|
|
The Playwright container image is declared in `roles/test-e2e-playwright/meta/services.yml` under the `playwright` service entry (`image:` + `version:`). CI mirrors that image into GHCR through the standard mirror pipeline before deploy tests run, so Playwright jobs do not depend on direct pulls from MCR during the test phase.
|
|
|
|
The `version:` MUST stay pinned to an exact Playwright version and distro pair such as `v1.59.1-noble`. You MUST NOT switch this to a mutable tag such as `latest`, because mutable upstream tags are more vulnerable to CDN propagation delays and make flaky CI failures much harder to reason about.
|
|
|
|
## Runner 🚀
|
|
|
|
The `test-e2e-playwright` role discovers Playwright-enabled roles through the presence of
|
|
`templates/playwright.env.j2`. For each discovered role it:
|
|
|
|
1. stages the test project
|
|
2. renders `.env` from `playwright.env.j2`
|
|
3. copies `files/playwright/playwright.spec.js` into the project as `tests/playwright.spec.js`
|
|
4. renders the central `templates/package.json.j2` and copies the central `files/playwright.config.js` into the staging dir
|
|
5. waits until the app is reachable
|
|
6. runs Playwright in Docker
|
|
|
|
## Artefact Retention 🎞️
|
|
|
|
The runner keeps Playwright traces, screenshots, and videos for every test only when the resolved environment variable `PLAYWRIGHT_KEEP_ALL` evaluates to `true` (case-insensitive).
|
|
Any other value, including unset or empty, keeps artefacts only when a test fails.
|
|
|
|
The propagation chain has four hops:
|
|
|
|
1. The host environment variable `PLAYWRIGHT_KEEP_ALL` (set on the host shell, by `act --env PLAYWRIGHT_KEEP_ALL=true`, or by the GitHub Actions repository variable forwarded into each `test-deploy-*.yml` workflow as `PLAYWRIGHT_KEEP_ALL: ${{ vars.PLAYWRIGHT_KEEP_ALL }}`).
|
|
2. `roles/test-e2e-playwright/defaults/main.yml` reads the host value into `TEST_E2E_PLAYWRIGHT_KEEP_ALL` via `lookup('env', 'PLAYWRIGHT_KEEP_ALL')`. Inventory MAY override the role variable per environment in `group_vars` or `host_vars`.
|
|
3. `roles/test-e2e-playwright/tasks/run_one.yml` forwards the resolved value into the Playwright container as `-e PLAYWRIGHT_KEEP_ALL={{ TEST_E2E_PLAYWRIGHT_KEEP_ALL }}`.
|
|
4. `roles/test-e2e-playwright/files/playwright.config.js` switches `trace`, `screenshot`, and `video` to `on` when the container value is `true`. It otherwise leaves the failure-only defaults (`retain-on-failure` for trace and video, `only-on-failure` for screenshot).
|
|
|
|
Set the variable for the contexts you need:
|
|
|
|
- **Local deploy**: prefix every `make deploy-*` invocation with `PLAYWRIGHT_KEEP_ALL=true`.
|
|
- **Local `act` runs**: append `--env PLAYWRIGHT_KEEP_ALL=true` to the `act` command.
|
|
- **CI**: set the `PLAYWRIGHT_KEEP_ALL` repository variable. For the GitHub click path see [CI Configuration](../../tools/github/actions/configuration.md#playwright_keep_all-).
|
|
- **Inventory pin**: set `TEST_E2E_PLAYWRIGHT_KEEP_ALL: "true"` in `group_vars` or `host_vars` of the target environment.
|
|
|
|
| Resolved container value | Trace | Screenshot | Video |
|
|
|---|---|---|---|
|
|
| `true` (case-insensitive) | `on` | `on` | `on` |
|
|
| any other value, unset, or empty | `retain-on-failure` | `only-on-failure` | `retain-on-failure` |
|
|
|
|
## File Requirements 📄
|
|
|
|
See the dedicated SPOT pages for authoring rules:
|
|
|
|
- [`playwright.spec.js`](../../artefact/files/role/playwright.specs.js.md): what the role-local spec MUST contain (entry point, scenarios, selectors, final state)
|
|
- [`playwright.env.j2`](../../../agents/files/role/playwright.env.j2.md): rendered test input surface, variable naming, and entry-point selection
|
|
|
|
## Recording Tests 🎬
|
|
|
|
### Playwright Codegen (built-in) 🔧
|
|
|
|
Playwright ships a built-in code recorder:
|
|
|
|
```bash
|
|
npx playwright codegen https://<app-url>
|
|
```
|
|
|
|
This opens the Playwright Inspector alongside a browser. Every interaction is recorded and emitted as a runnable spec. Use the output as a starting draft and clean it up according to the authoring rules in [`playwright.spec.js`](../../artefact/files/role/playwright.specs.js.md).
|
|
|
|
### playwright-recorder (Docker-based) 🐳
|
|
|
|
For infrastructure-heavy setups where Playwright should not live in the main repository, use [playwright-recorder](https://github.com/kevinveenbirkenbach/playwright-recorder). It wraps Playwright codegen in Docker with X11/XWayland forwarding and an ephemeral workspace so no state leaks between recordings.
|
|
|
|
Requirements: Docker, X11 or XWayland, `xhost`
|
|
|
|
```bash
|
|
# Install
|
|
make install
|
|
|
|
# Record a flow starting at a specific URL
|
|
make codegen
|
|
# or directly:
|
|
./scripts/codegen.sh https://<app-url>/login
|
|
|
|
# Replay a recorded spec headlessly
|
|
make replay
|
|
# or directly:
|
|
./scripts/replay.sh recordings/login.spec.ts
|
|
```
|
|
|
|
Generated files are persisted in the local workspace (git-ignored). Copy the relevant parts into `files/playwright/playwright.spec.js` and adapt them.
|
|
|
|
## Development Procedure 📋
|
|
|
|
This page owns framework and runner mechanics only. The procedure for writing, deploying, and reviewing a role-local spec lives in the dedicated SPOTs:
|
|
|
|
- [Agent `playwright.spec.js`](../../../agents/files/role/playwright.spec.js.md): authoring procedure and live review.
|
|
- [Role Loop](../../../agents/action/iteration/role.md): baseline deploy, redeploy rules, Certificate Authority trust, Inspect-before-redeploy.
|
|
- [Playwright Spec Loop](../../../agents/action/iteration/playwright.md): spec-only inner loop against the already-running stack.
|
|
- [`playwright.spec.js` (role file rules)](../../artefact/files/role/playwright.specs.js.md): what the spec MUST contain, including logged-out final state and console cleanliness.
|