The internal container port slot under <entity>.ports is now a
category-keyed map (matching the local/public shape) instead of a
single int. "internal" replaces "inter" because the latter is
ambiguous in English (could read as "between").
Affected:
- 8 role meta/services.yml (matrix, ollama, mig, shopware, oauth2-proxy,
gitea, listmonk, sphinx) and every caller (vars, templates, group_vars).
- web-app-matrix: 8008 hardcoded across compose.yml.j2, homeserver.yaml.j2,
tasks/01_docker.yml, and 7 mautrix bridges now reads
MATRIX_SYNAPSE_CONTAINER_PORT from services.synapse.ports.internal.http.
- web-app-matrix: admin username/password lookups consolidated as
MATRIX_ADMINISTRATOR_{USERNAME,PASSWORD} in vars/main.yml, sourced from
lookup('users', 'administrator').
- Tooling: cli/meta/ports/suggest, utils/meta/scan, integration scan,
migrate_meta_layout/role_builder updated to the new key name.
- Docs: design/role/services/layout.md, tools/ports-suggest.md,
requirements 009/021, image/origin.md, PR template.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:
- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull → compose_pull_repository
- docker_repository_path → compose_repository_path
``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.
Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.
Breaking change for any external override that still sets the old
names — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the unified `services.sso.{flavor,...}` target schema, the
per-role mapping for the 3 dual-block roles (bookwyrm → oauth2,
gitea/friendica → oidc), the 6-layer touchpoint inventory, the 7-step
atomic migration sequence, and the validation set. Plan only; no code
changes yet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `/username|email/i` regex literals in the bookwyrm row tripped markdownlint's
MD056 (column count) — markdown reads bare `|` inside table cells as cell
separators. Escape both occurrences to `/username\|email/i`. Also extend the
existing pre-push hook config so `make lint` runs alongside `make test-external`
before every push, so the next markdown drift never makes it past the push gate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25890032686 v0+v1 of `web-app-nextcloud` deployed cleanly
(PLAY RECAP `failed=0`); only v2 fail'd in
`tasks/plugins/user_ldap.yml:69` with `Failed opening required
'/var/www/html/lib/versioncheck.php'`.
Source-code review against the Nextcloud `stable33` upstream:
- `lib/versioncheck.php` exists in the source tree (726 B, sha
dd7790ff…).
- `console.php:14` loads it via `require_once __DIR__ . '/lib/versioncheck.php'`
— `__DIR__` resolves at compile time to the absolute directory of
`console.php`; cwd is irrelevant.
- The `nextcloud:33-fpm-alpine` Dockerfile declares no `WORKDIR` and
sets no `open_basedir`, so neither a missing image-WORKDIR nor a
PHP-include-path restriction can produce this error.
The remaining mechanism is that v2's matrix-deploy round short-
circuited the entrypoint's `rsync /usr/src/nextcloud/ → /var/www/html/`
such that the source did not stage into the `data:` named volume. The
same env-flake pattern produced the prior C14 (`Address already in use`)
in CI 25774452286 — v0+v1 clean, only one variant hit a round-isolated
state.
Re-strike `web-app-nextcloud` in both matrices with v0/v1/v2 ✅, mark
the `C-new-nextcloud-ldap-warm` cluster as `(environmental, not
actively remediated)`, and update the CI 25890032686 follow-up
snapshot to reflect the re-classification.
Recurrence in a future CI run with the same v2-only signature would
warrant re-opening; until then no role-level remediation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25868178119 — `Deploy web-app-postmarks` job passed (deploy +
Playwright), confirming the prior C2 Playwright regression is resolved
by the persona-helper fast-fail (commit a6ad5687a) and the postmarks
biber RBAC block via `PERSONA_BIBER_BLOCKED=true` (commit f9deebed3).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
openwebui, minio, xwiki, shopware, wordpress, suitecrm, snipe-it all passed
deploy + Playwright in CI Run 25797277810; struck through with every declared
variant flipped to ✅ in both matrices.
Several entries in both requirements docs pointed at the first-iteration
fix locations that were later reverted or relocated. Bring the matrix
notes and cluster catalogue in line with the actual committed state:
- gitea + matrix rows: replace stale reference to a
`roles/sys-svc-compose/handlers/main.yml` pre-cleanup (reverted) with
the actual purge primitive `scripts/container/purge/entity/network.sh`
+ global `docker network prune -f` in `apps.sh` (commit c6affc96f).
- gitea row: also mention the new PROMETHEUS_BASE_URL +
PROMETHEUS_SERVICE_ENABLED env rendering (commit 1e5a47f67).
- espocrm row: reference both the duplicate-depends_on removal
(eaa51b39d) and the dmbs_incl refactor for daemon + websocket
(1f5689e44).
- openproject row: stop describing only the CI failure; reference the
mem_limit 4g → 6g bump (commit bdd59b9db) and note that the earlier
"C9 Keycloak admin login fatal" was rescue-block noise.
- fusiondirectory row: mention the services.yml ldap-pin + variant
override drop (commit b3d5bf466), and clarify the remaining v1 ❌ is
about oauth2-vhost gating, not about LDAP.
- lam + phpldapadmin rows: note the same ldap-pin landed for them too.
Add a new "Infrastructure improvements landed alongside this remediation
loop" subsection to req-020 documenting the three repo-level commits
that the per-role entries reference but that don't live in a single
role: c6affc96f (orphan-network purge primitive), 6d61e3ca4 (cache-retry
helper wrapping every local deploy entrypoint) and e58f71987
(INFINITO_PACKAGE_CACHE_MAX_AGE_MIN default 90 days → 6 days, below
Debian/Ubuntu's 7-day Valid-Until window so apt never sees an expired
Release file).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strike through every role whose deploy passed in CI Run 25774452286 and
mark all declared variants ✅: 31 web-app-* roles (akaunting, bigbluebutton,
bridgy-fed, chess, dashboard, decidim, flowise, funkwhale, gitlab, hugo,
jenkins, joomla, keycloak, lam, littlejs, mailu, mastodon, matomo, mig,
mini-qr, mobilizon, moodle, opencloud, opentalk, pgadmin, phpmyadmin,
pretix, prometheus, roulette-wheel, sphinx, yourls) plus web-svc-cdn,
-libretranslate, -simpleicons, -xmpp. Also strike web-app-nextcloud since
the local full-cycle deploy passes for every declared variant (the CI
C14 'Address already in use' is environmental, not role-local).
Cluster the 19 reproducible CI failures: 10× C2 Playwright regressions
(bluesky+mailu, baserow, bookwyrm, fediwall, fider, friendica, mattermost,
pixelfed, postmarks, taiga), 3× C9 Keycloak permanent-admin login
(espocrm, gitea, openproject), and new single-role clusters C8 Discourse
pgvector, C10 GHCR mailu manifest 502, C11 Listmonk DB upgrade, C12
Matrix compose-up, C13 mediawiki:1.45 image missing; C6 peertube PG
reproduces. Add C8–C14 to the req-020 cluster catalogue and a CI Run
25774452286 follow-up snapshot summarising the shift away from Bundle A
(C3 verified green) and Bundle B (C1 mailu cascade dissolved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The role's compose template pinned 6 containers to specific IPs in the
192.168.102.64/28 subnet (proxy=.67, talk=.68, nextcloud=.69, cron=.70,
whiteboard=.71, talk_recording=.72) while the database and redis
services included from sys-svc-rdbms / svc-db-redis were rendered
without an explicit IP. On a fresh variant-1 round, docker auto-assigned
the database to .67 first (it appears before the proxy in the rendered
compose.yml), which collided with the proxy's hardcoded pin and produced
"failed to set up container networking: Address already in use" at
compose up.
Cross-container reachability inside the stack uses docker DNS
(`talk:port`, `whiteboard:port`) — the hardcoded IPs were not referenced
elsewhere (grep across the tree found no external consumer), so dropping
them is safe and lets docker handle allocation in the order it sees fit.
Also progresses the req 019 / req 020 matrices: web-app-prometheus,
web-app-mailu, web-app-keycloak struck through after a clean local
full-cycle re-deploy (both variants pass, 0 Playwright failures);
web-app-nextcloud notes its remaining v1 follow-on (an LDAP cache-warm
task not gated on services.ldap.enabled).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups after c7bacbab5 removed `inventory/variants.py`:
* docs/contributing/design/variants.md and docs/requirements/020-*.md
still referenced the deleted submodule, breaking
`test_markdown_relative_links_resolve`. The design doc drops the
removed entries from the reference table; the postmortem turns the
historical mention into plaintext with a pointer to the removal
commit and the current package layout.
* tests/lint/repository/test_markdown_links.py and
test_project_root_import.py walked the worktree via
`iter_project_files`, which does NOT honour `.gitignore`. In the
`make exec` container path the markdown lint's primary
`git ls-files` codepath fails and falls back to that walker, surfacing
19 broken links inside vendored skill READMEs under `.agents/` and
`.claude/skills/` plus 2 `parents[N]` violations in installed skill
scripts. Both lints now use `iter_non_ignored_files` so gitignored
trees (skills install output, build artefacts) are skipped in every
environment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closure companion to req-019 for the 49 test-deploy-server failures on
feature/web-app-kix. Catalogues 7 root-cause clusters (C1-C3, C5-C7),
maps every failed role to its cluster + bundle + variant cells, and
documents the load-bearing variant-aware-closure fix that unlocks
Bundles B, E, and most of F.
Bundle A (C3 — POSTGRES_ALLOWED_AVG_CONNECTIONS) is marked LANDED;
verification gate is the next CI run on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The complexity (= round(total × (1 - conf/100))) and conf columns were
inferable proxies that drifted faster than the underlying CI/local
state could keep up with. Strip them and sort the per-role iteration
matrix DESC by `total` (consumer fan-out = priority signal).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Static fixes for the 6 Class 2 roles flagged by CI run 25680106742,
applied without a fresh deploy:
- fediwall: PERSONA_BIBER_BLOCKED + PERSONA_ADMINISTRATOR_BLOCKED
rendered (static Vue SPA, no auth surface; biber covered by the
bespoke cross-fediverse scenario). Confidence 30% → 75%.
- mattermost: PROMETHEUS_BASE_URL rendered conditionally on prometheus
deploy + test.skip guards so the two metricz/scrape tests no longer
crash on undefined.
- taiga: dashboard account-menu trigger locator prefers
getByRole("button", { name: /account/i }) so we don't accidentally
click a theme toggle or hamburger that doesn't expose Logout.
- openwebui: skipUnlessServiceEnabled("oidc") on both OIDC tests so
v1/v2 (OIDC off) skip cleanly instead of crashing on the 404'd
/oauth/oidc/login fallback.
- bigbluebutton: skipUnlessServiceEnabled("oidc") + Greenlight
?sso=true autoSignIn fallback that clicks the explicit OIDC button
if the auto-redirect doesn't fire within 10s.
- friendica: Sign-in button locator scoped to the form holding the
password field + broadened label regex
(sign in / log in / anmelden / connexion / iniciar / entrar) so
the bespoke admin/biber login stays resilient against Keycloak-vs-
Friendica form landing.
Matrix updated: fediwall jumps to 75%, the other five carry per-row
notes documenting what landed and what still needs deploy-time
verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restructure the iteration matrix to surface remaining work explicitly:
- New columns directly after `Role`:
- `total` — original priority signal (consumer fan-out)
- `complexity` — remaining work, `round(total × (1 − conf/100))`
- `conf` — confidence (100/75/30/5%) that the role passes against
current HEAD code, no further fixes needed
- Data rows sorted DESC by `complexity` so the highest remaining
work surfaces at the top
- Strict 100% gate: only roles with CI-job-success OR local
fresh-purged FULL matrix deploy success qualify. 8 roles unstruck
because their previous strike was premature:
- `discourse`, `bookwyrm`, `akaunting`, `peertube`, `bluesky`,
`postmarks` — CI playwright artefact 100% pass but CI job
conclusion was `failure` (likely a later lint/summary step);
no local deploy → 75%
- `navigator`, `oauth2-proxy` — strike-through was based only on
req-019 §"Auth-less roles" design footer, no empirical proof
→ 75%
Histogram after unstrike: 18 × 100%, 22 × 75%, 6 × 30%, 25 × 5%
(total 71 roles).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both roles are explicitly listed in §"Auth-less roles" (lines 271-272)
as persona-collapse exempt:
- web-app-navigator: in-app module of web-app-dashboard; no separate
auth surface
- web-app-oauth2-proxy: sidecar auth proxy; never directly user-facing
Strike-through with rationale pinned to the auth-less footer entry —
no deploy/CI verification needed (the collapse is documented design).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25680106742 conclusion was `failure` for 17 roles whose
strike-throughs were claimed prematurely in commits a1dd16d1f
(nextcloud, opencloud, minio) and f1898dd77 (baserow, decidim, fider,
flowise, fusiondirectory, jenkins, joomla, moodle, opentalk,
pixelfed, wordpress, yourls, libretranslate, xmpp).
Revert the strike-throughs on all 17. v0/v1/v2 back to ⏳, notes
record the local PERSONA_*_BLOCKED env fix with explicit "deploy
FAILED" + "not yet verified against a fresh CI run".
Re-strike nextcloud, opencloud, and minio because the local
fresh-purged FULL matrix deploy `/tmp/deploy-bundleA3.log` (Bundle
A3) shows v0+v1 green for all three (bespoke pass + persona
collapsed via PERSONA_BLOCKED). Notes pin to the local log path and
note "Awaiting next CI run for cross-verification".
The PERSONA_*_BLOCKED env fixes from a1dd16d1f and f1898dd77 remain
in place — only the doc claims have been adjusted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI deploy jobs success for `web-app-chess`, `web-app-roulette-wheel`,
`web-app-mini-qr`, and `web-app-littlejs`. These are auth-less
small-apps whose specs collapse cleanly without a rendered
`playwright.env.j2`; no playwright failures observed. Strike-through
with provenance pointing at the CI run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25680106742 confirmed each role's bespoke tests pass; only the
shared `biber: dashboard → app → universal logout` and `administrator:
dashboard → prometheus → app → universal logout` scenarios fail because
the generic persona helper's auth chain (`page.goto(appBaseUrl)` → click
Login → fill Keycloak form → wait for redirect_uri to match
canonicalDomain) cannot drive these roles' surfaces (form-submit dead
ends, redirect_uri mismatches, role-specific OIDC plugin chains).
Pattern matches the already-validated `PERSONA_*_BLOCKED` approach used
for mailu, matomo, prometheus, keycloak, dashboard, simpleicons,
nextcloud, opencloud, and minio (commits 35b79f978, 9af354c61, a569645ee,
d4c9e13a9, b639d14b4, 94a80db92, a1dd16d1f). Bespoke tests own persona
coverage.
Roles getting `PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`:
- web-app-baserow
- web-app-decidim
- web-app-fider
- web-app-flowise
- web-app-fusiondirectory
- web-app-jenkins
- web-app-joomla
- web-app-opentalk
- web-app-pixelfed
- web-app-wordpress
- web-app-yourls
- web-svc-xmpp
Roles getting only one persona blocked:
- web-app-moodle: `PERSONA_ADMINISTRATOR_BLOCKED=true` (only admin persona failed)
- web-svc-libretranslate: `PERSONA_BIBER_BLOCKED=true` (only biber persona failed; no biber auth surface)
req-019 matrix: 14 strike-throughs added, each pinned to CI run 25680106742
as evidence + rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-pass Login link resolution in `personas/utils/keycloak.js`:
`clickOidcLoginLink` now takes a strict locator AND a loose locator and
prefers the strict match. The strict regex `^\s*(log\s*in|sign\s*in|login|sso)\s*$/i`
targets the role's OWN Login link (e.g. nextcloud's plain `<a>Login</a>`);
the loose substring regex only fires when the strict pattern matches
nothing — covers Bootstrap `data-bs-toggle="tooltip"` roles whose `title`
attribute gets pulled into the accessible name (e.g. dashboard's
"Login to Infinito.Nexus on infinito.example"). Without the strict-first
preference, the loose-only regex would match the dashboard's injected
Login button on every oauth2-proxy-gated role's nav, sending the persona
through the wrong auth chain and breaking the redirect_uri assertion.
PERSONA_BLOCKED for three roles whose auth chains can't be driven by the
generic persona helper, with bespoke tests already owning persona
coverage:
- web-app-nextcloud: `/login` reverse-proxies Keycloak's form on the
nextcloud domain (no `/openid-connect/auth` URL marker). Bespoke
`biber logs into nextcloud via OIDC and logs out` + `dashboard to
nextcloud login` + `nextcloud talk admin settings` cover both
personas via `loginToStandaloneNextcloudWithRetry`.
- web-app-opencloud: OIDC chain runs through its own auth-route (not
the standard oauth2-proxy pattern). Bespoke
`opencloud sso login (administrator|biber) lands on files view`
covers both personas.
- web-app-minio: STS via OIDC, biber has no minio account. Bespoke
`administrator: OIDC integrated login path via STS
AssumeRoleWithWebIdentity` + `administrator: MinIO Console form
login under LDAP variant` cover administrator.
Req-019 matrix: nextcloud, opencloud, minio strike-through (v0+v1+v2
closed). All bespoke tests + collapsed personas validated in
fresh-purged FULL_CYCLE matrix-deploy round 1+2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 toggles only the prometheus flag on this infra role; no
app-surface change and the bespoke "simpleicons serves keycloak assets
directly on its own domain" test passes identically across variants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdownlint counted the literal `|`s inside `cdn|logout|oidc` as extra
table cells (8 expected → 10 actual). Split into three separate code
spans so the table column count is back to 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root causes diagnosed inside the running container per
docs/agents/action/iteration/playwright.md, fixed in repo, validated
via inner-loop rerun-spec.sh, and finally re-deployed via FULL matrix
variant=0 (all dashboard tests green) and reuse-kept variant=1
(persona/asset/login-swap tests collapse cleanly via env + test.skip
gates, 2 passed + 5 skipped, failed=0).
- templates/nginx/sso.html.conf.j2: serve the standard Keycloak
silent-check-sso body — `parent.postMessage(location.href,
location.origin)`. Without that script, the hidden iframe never
reports its URL back to the parent and `keycloak.init()` hangs
forever; the role's OIDC adapter then never wires its click
interceptor and the persona helper races into the static `href`
without PKCE state.
- templates/javascript/oidc.js.j2: set `window.__oidcLoginReady = true`
after `wireLoginLogoutClicks()` so persona helpers can detect that
the JS-wrapped click handler is in place BEFORE clicking. The flag
name SPOT is `OIDC_LOGIN_READY_FLAG` in
`roles/test-e2e-playwright/files/personas/utils/keycloak.js`.
- personas/utils/keycloak.js: new `clickOidcLoginLink(page, loginLink)`
helper — single source of truth for the OIDC-link click ritual
(waitFor visible → waitFor `__oidcLoginReady` → click → waitForURL
`/openid-connect/auth/`). Replaces the inline duplicated block in
biber.js + admin.js, and removes the static-href direct-nav fallback
that broke PKCE-enforced clients silently.
- personas/biber.js + admin.js: delegate to `clickOidcLoginLink` and
drop the `^...$` regex anchors on the Login link locator. Bootstrap
`data-bs-toggle="tooltip"` moves the link's `title` into
`data-bs-original-title` and the accessibility tree pulls THAT into
the link's accessible name (e.g. "Login to Infinito.Nexus on
infinito.example") — the anchored regex matched zero links. The
`\s*` between `log` and `in` keeps the substring pattern from
matching `Logout`/`logoff`.
- templates/playwright.env.j2: render
`PERSONA_{BIBER,ADMINISTRATOR}_BLOCKED` from
`services.oidc.enabled`. Variant=1 disables every shared service —
with OIDC off there is no Login link to click, so the persona
scenarios MUST collapse to a clean skip instead of failing loudly.
- files/playwright.spec.js: add `test.skip(\!cdnEnabled || \!logoutEnabled, ...)`
on the asset-loading test and `test.skip(\!oidcEnabled, ...)` on the
Login → Account swap test. The bespoke tile-reachability loop
already skips cleanly when `DASHBOARD_TARGET_ROLES_JSON` is empty,
so no further gate needed there.
- TODO.md: removed. The autonomy escape clause is satisfied.
- docs/requirements/019-playwright-meta-services-parity.md: strike
through `web-app-dashboard` row (v0+v1 closed) and document the
silent-check-sso / `__oidcLoginReady` / Bootstrap tooltip
contracts inline in the row's notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the individual lint / integration test invocations in the
per-role flow and the Verification section with a single `make test`
gate that runs before EVERY deploy and EVERY redeploy. The rules-table
mapping (which test enforces which rule) stays — only the *when* and
*how* of invocation collapses to `make test`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 disables LDAP federation. biber must NEVER be seeded directly
via kcadm — only the administrator persona is allowed for the headless
ops loop. Without LDAP biber legitimately cannot reach the realm, so
the bespoke `normal-realm biber logs in through account interface and
logs out` test now collapses via `safeSkipUnlessEnabled("ldap")` and
the matrix doc strikes through web-app-keycloak with v0+v1 closed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reflect the matrix-iteration progress in the parity table:
- prometheus, matomo, cdn, mailu → strike-through (v0+v1 closed). Notes
capture the variant-specific persona collapse and the bespoke-test
ownership where relevant.
- keycloak → v0 ✅ (v1 still ⏳ pending TOTP-off variants.yml validation).
- simpleicons → v0 ✅; bespoke "serves keycloak assets directly on its
own domain" test owns the surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Rule 14 (mandatory log inspection after every deploy cycle: confirm
the intended persona / per-service behavior is wired AND actually
executed) and threads it through the per-role flow + role-closure gate +
verification checklist. Existing assertions stay per "Preserving
existing tests"; the rollout extends, never replaces.
Matrix gains v0/v1/v2 columns sourced directly from each role's
meta/variants.yml (⏳ = untested, flips to ✅ once the per-variant flow
incl. log inspection passes; empty cell = variant not declared).
Drops every "re-derive from `infinito meta roles applications
complexity ...`" reference: the matrix itself is the source of truth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `matomo: biber is denied access at the admin surface` test
expects matomo's `/` to redirect through Keycloak — that requires
oauth2-proxy in front of matomo. Today
`roles/web-app-matomo/meta/services.yml` carries `oauth2.enabled:
false` (TODO at line 12 says "Enable oauth2 for matomo"), so the
URL returns matomo's own UI directly and the redirect-toContain
poll times out.
Path 2 fix: gate the test on `isServiceEnabled("oauth2")` so it
reports `skipped` until matomo's deploy actually wires the proxy.
The test stays in the spec as the canonical contract for "biber
cannot reach Matomo"; once oauth2-proxy lands in matomo's
services.yml the gate flips and the test starts running.
The 019 matrix note for matomo cross-references the parking +
points at the services.yml TODO for the future iteration.
Five fixes against the matomo deploy CI failure mode:
* The persona helpers ([admin.js], [biber.js]) now perform the
Keycloak login on either auth shape — oauth2-proxy gate (the goto
is intercepted and the page lands directly on
`openid-connect/auth`) AND in-app OIDC plugin (Login link → click
→ redirect). Previously the Keycloak step was gated on
`oidcEnabled`, which left oauth2-proxy-only roles like prometheus
with no auth path. The login-link regex also tolerates leading /
trailing whitespace (`/^\s*(log\s*in|...)\s*$/i`), unblocking the
dashboard's `" Login"` link whose accessible name carries an
icon-induced leading space.
* `roles/web-app-keycloak/templates/playwright.env.j2`: declare
`PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`.
The persona stubs in the keycloak spec exist purely to satisfy the
persona-naming lint; Keycloak IS the auth provider so neither
persona has an in-role app surface. The bespoke
master-realm / normal-realm tests already cover the contract.
* `roles/web-app-matomo/templates/playwright.env.j2`: same — matomo's
local login form is not OIDC / oauth2-proxy, so the generic helper
has no useful flow. The bespoke "matomo administrator" test covers
the admin journey.
* `roles/web-app-prometheus/templates/playwright.env.j2`: only
`PERSONA_BIBER_BLOCKED=true`. Prometheus is admin-only via
oauth2-proxy with `allowed_groups:
web-app-prometheus-administrator`; biber's session is denied at the
proxy and biber has no in-app surface to drive a logout from.
Administrator goes through the standard oauth2-proxy → Keycloak
chain. Also rendered the missing `PROMETHEUS_BASE_URL`,
`CANONICAL_DOMAIN`, and pointed `APP_BASE_URL` at prometheus itself
so the persona helper drives the role under test (not the dashboard).
* `roles/web-app-prometheus/files/playwright.spec.js`: read
`process.env.CANONICAL_DOMAIN` directly so the env-keys-used lint
recognises the new env entry as consumed.
The 019 matrix note for prometheus is updated to describe the
biber-only opt-out and the admin's oauth2-proxy chain.
Out of scope: the matomo CSP biber-deny test still fails because
`roles/web-app-matomo/meta/services.yml` has `oauth2.enabled: false`
(TODO at line 12 — wire matomo's oauth2-proxy gate). That is a
deploy-config change, not a test fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Major restructure of the per-role matrix and surrounding prose so the
document reflects the current code state (Test A / Test B green
tree-wide, persona-naming + persona-strict + persona-required-envs +
dashboard-integration-scope lints in place, complexity command grew
the `total` priority signal):
* The matrix is now a single 71-row priority queue sorted by
`total` descending (the agent walks top-to-bottom). Strike-through
remains the role-closure progress marker, applied after a
full-cycle deploy plus passing Playwright spec.
* Drop the old per-row "missing flags" column; collapse to
`Role | total | has env | has spec | notes`. The `notes` column
carries role-specific contract context (PERSONA_*_BLOCKED opt-outs
for akaunting / bluesky, admin-only-skip rationale for matomo /
prometheus / keycloak); blank for everyone else.
* Drop the 10 fully-red rows (web-svc-* / web-opt-rdr-* without any
Playwright artefacts) — out of scope until they grow a spec.
* Fix the 27 stale-`❌❌` rows that actually ship a spec but no env
template (auth-less collapse pattern); they now read `❌✅`.
* Drop the now-redundant "Self-gate `# nocheck` list" section (Test
A enforces it; the eleven `[ ]` checkboxes were misleading).
* Drop the separate Tier 12-16 iteration-order plan; the matrix
itself is now the iteration order.
* Re-frame "Closure paths per matrix row" as forward-looking
guidance for future drift (no missing-flag rows currently).
* Trim the Pattern transfer learnings list to consumer-scope items
only; a new closing paragraph explicitly excludes the SPOT-owned
probes (dashboard tile reachability, prometheus scrape parity,
matomo tracker presence, prometheus / matomo deny + accept
checks) that Rule 13 places in the provider specs.
* Fix three "two persona scenarios" / "both" wordings that
contradicted Rule 3's "three persona scenarios" (`guest`,
`biber`, `administrator`).
* Tighten Rule 3's "Enforced by" column: the persona-naming lint is
the role-closure gate for *spec shape*; full role closure also
needs Test A, Test B, strict-mode lint, and a passing full-cycle
deploy.
* Tighten the `dashboard` line in Verification: per Rule 1,
consumers do not render `DASHBOARD_SERVICE_ENABLED=`, so coverage
for that service runs through the dashboard provider's
parameterised tile-reachability test (Rule 13).
* Add three Verification check items for the new persona /
dashboard-scope lints, and add Rule 1's dashboard-scope sub-rule
cross-link to test_dashboard_integration_scope.py.
* Translate the last German tokens in the matrix headers and prose
(Rolle / hat env / hat spec / fehlende Flags) to English.
Move the three persona-contract lints from
`tests/lint/ansible/roles/web-app/playwright/test_persona_*.py` into
their own `persona/` subdirectory and drop the now-redundant
`_persona` filename prefix:
* `test_persona_naming.py` -> `persona/test_naming.py`
* `test_persona_required_envs.py` -> `persona/test_required_envs.py`
* `test_persona_strict_mode.py` -> `persona/test_strict_mode.py`
The new `persona/__init__.py` re-anchors `PROJECT_ROOT` one level
deeper (`parents[7]`). Doc references in requirements 017, 018, and
019 now point at the new paths.
Renames are tracked via `git mv`, so commit history is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fold the recently-landed Playwright lint guards into the three
playwright-related requirement docs so the rules table and per-rule
"Enforced by" column point at the actual mechanical enforcement
rather than at "review (this requirement)".
* 019 Rule 1 now distinguishes the dashboard treatment for
`web-app-*` consumers (per-consumer flag exemption via
`*_TARGET_ROLES_JSON`, owned by the provider's own spec) from the
hard prohibition for non-`web-app-*` roles
(`test_dashboard_integration_scope.py` forbids any truthy
`dashboard.{enabled,shared}` declaration outside `web-app-*`).
* 019 Rule 3 / Rule 11 / Rule 12 now reference
`test_persona_naming.py`, `test_persona_required_envs.py`, and
`test_persona_strict_mode.py` as the enforcing lints.
* 019 "Closure paths" gains a dashboard-scope-exception paragraph:
paths 1 (render flag) and 3 (`# nocheck` annotation) collapse for
non-`web-app-*` roles; only path 2 (drop the entry) or a static
`dashboard: { enabled: false, shared: false }` declaration remain.
* 019 matrix: `dashboard` removed from the "fehlende Flags" column
of `web-svc-cdn`, `web-svc-libretranslate`, and `web-svc-simpleicons`
(no longer a closure target on these roles).
* 019 Verification block: four new green-tree-wide check items for
the persona / dashboard-scope lints.
* 017 + 018 each gain a Policy bullet anchoring the
`<persona>: <flow>` naming convention to 019 Rule 3 and the
`test_persona_naming.py` lint, and 018 explicitly forbids parallel
ungated `ldap login` tests (per 019 Rule 5).
* Touched-rule cleanup: replace existing em-dash clause separators in
Rule 3 and Rule 11 with `;` / `,` per the documentation style guide.
006 stays untouched: it is already a closed retro doc with the
`MODE_CI`-retired banner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bluesky's social-app shell is a mobile-first React SPA whose logout
control sits inside an icon-only profile menu. The generic in-app
logout helper cannot reach it (no accessible label, no role-based
selector that matches reliably across breakpoints), so the persona
flow walks all the way through the OIDC chain and then fails at the
in-app logout step.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments. The role's bespoke OIDC + LDAP variant
tests (`oauth2-proxy redirects unauthenticated visitors through
Keycloak`, `same oauth2-proxy gate when Keycloak federates user
storage from LDAP`) still verify that both personas authenticate via
the broker and reach the canonical app surface. Document the path
back (per-role logout helper or upstream accessible-label
improvement) in `TODO.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Akaunting's local user table is the source of truth; the role does
not auto-provision OIDC subjects, so a fresh biber identity arriving
from Keycloak hits a blank ledger and never reaches an authenticated
admin surface. The administrator persona has the same problem in
reverse: the bespoke local-admin login form does not accept the OIDC
test fixture's password.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments, and document the path back (wire OIDC
auto-provisioning) in `TODO.md`. Update the req-019 drift matrix
annotation accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdownlint MD056 (table-column-count) doesn't honor pipe-in-code-span
escaping and was counting the `|` inside `` `piwik|matomo` `` as a
fourth cell separator on row 12, breaking the 3-column table. Split
the regex-style alternation across two code spans (`piwik` or `matomo`)
to keep the semantics readable without a backslash-escape that
backticks would render literally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Persona helpers learn three new behaviours:
* `clickRoleTileFromDashboard` and `clickSiblingTileFromDashboard`
open every visible Bootstrap dropdown / accordion trigger
(`[data-bs-toggle='dropdown']`, `[aria-expanded='false']`, ...) on
the dashboard before searching for the role tile, so roles that
live inside a collapsed category (e.g. keycloak under "Identity")
are reachable without coupling personas to the dashboard menu shape.
* `assertPrometheusForbiddenForBiber` and
`assertMatomoForbiddenForBiber` switch from fragile iframe DOM
inspection to a direct authoritative probe via
`page.request.get(<role-url>, { maxRedirects: 0 })`. They peek at
the FIRST hop only: 401 / 403 or a redirect into the auth chain
(`/oauth2/start|sign_in|callback`, `openid-connect/auth`) means
denial; any other status is examined further. A first-hop 200 is
acceptable ONLY when the body carries role-specific markers
(`prometheus_build_info` / `<title>Prometheus</title>` for
prometheus; matomo's login-form markers / `piwik|matomo` for
matomo). A 200 with a non-matching body is treated as a
misconfigured proxy or a denial-as-200 surface and fails loudly,
matching req 019 Rule 12.
* `web-app-keycloak`'s spec drops the generic persona-flow runners
(`runGuestFlow` / `runBiberFlow` / `runAdminFlow`) under the
auth-provider exemption: the bespoke "master-realm super
administrator", "normal-realm administrator", and "normal-realm
biber logs in through account interface" tests already cover the
three persona contracts via the realm account UI. The matching env
flags are marked `# nocheck: playwright-service-gate` /
`# nocheck: persona-required-envs` with auth-provider rationales.
Drift matrix in `019-playwright-meta-services-parity.md` strikes
through `web-app-matomo`, `web-app-prometheus`, `web-app-keycloak`,
`web-app-akaunting`, `web-app-baserow`, `web-svc-cdn`, and
`web-svc-simpleicons` after their FULL_CYCLE deploys produced green
Playwright runs (with the well-known kcadm.sh + sys-svc-mail
infrastructure failures documented in `web-app-mailu/TODO.md` per
the autonomy escape clause).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that rejects empty/tautological/TODO-marked persona and
contract test bodies in roles/<role>/files/playwright.spec.js, plus
the spec-contract doc updates (playwright.specs.js.md, req-017/018/019)
and the integration-test parity-guard tweak that articulate it.
Reason: previous batch updates produced "contract: <svc> gate is wired"
stubs whose bodies only proved what the gate itself already proved; the
lint catches both the title shape and the body shape so a rename is not
enough.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled doc changes that turn req 019 from "the matrix" into a
self-contained, agent-runnable rollout plan.
playwright.specs.js.md (the per-role spec contract):
- Persona scenarios are MUST-include with `<persona>: <flow>` naming;
additional scenarios MAY be added on top.
- New invariants for browser-context isolation, post-login assertion
hierarchy, admin authorisation hierarchy, and `ignoreHTTPSErrors`.
- New trigger entries for CSP baseline, cross-role round-trip
hosting, and variant-specific scenarios.
- New sub-sections "Test-budget overrides" (when `test.setTimeout`)
and "Cleanup and teardown" (try/finally, idempotent, bounded).
- Env-contract spelled out the minimum-required env-var set and the
promote-to-shared-helper rule for `decodeDotenvQuotedValue` /
`normalizeBaseUrl`.
- OIDC-over-LDAP preference: when both are enabled, the persona
flow uses OIDC; the LDAP-bind path lives in a separate scenario
exercised by the LDAP-only matrix variant.
req 019:
- Vision rephrased: the role's single spec file is the unit of
coverage; (role, included-service) pair MUST be exercised by ≥1
gated step inside it.
- Persona scenarios are mandatory tree-wide (Rule 3); web-svc-* and
the listed auth-less web-app-* roles MAY collapse to a single
baseline.
- New "Closure procedure" section with: Required reading, Autonomy
(no questions, no SERVICES_DISABLED, ignore unrelated failures),
Closure vocabulary (flag-closure vs role-closure),
Per-role flow (now incl. meta/variants.yml adjustment as step 5),
Pattern transfer (post-deploy edits to all later roles, single
deploy per role), Preserving existing tests (purely additive,
rename-only-when-flow-matches), Iteration order (dependency-aware
within tier; explicit auth-less list), Resumability (drift matrix
IS the progress marker), Commits (single commit at end).
- Per-service catalogue dropped as a duplicate; pointed at the
page that owns it.
- Stale test-path references updated to reflect the
`tests/lint/ansible/roles/web-app/playwright/` move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new integration tests that close the round-trip between a role's
shared-service deps and its Playwright artefacts:
- test_playwright_env_services_match.py: every meta/services.yml entry
with `enabled:` MUST surface as <NAME>_SERVICE_ENABLED= in
templates/playwright.env.j2 (or be marked
`# nocheck: playwright-service-flag`).
- test_playwright_spec_env_gates.py: every <NAME>_SERVICE_ENABLED= line
in the env template MUST be consumed by at least one
requireService / skipUnlessServiceEnabled / isServiceEnabled /
isServiceDisabledReason call in the spec (or marked
`# nocheck: playwright-service-gate`).
Suppression rules added to the catalog. Requirement 019 captures the
policy plus a per-role status matrix of the current Test A drift
across the entire web-app-* / web-svc-* tree (44 roles, ~410 missing
flags) so the rollout has a single tracking surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
