Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.
The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.
Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.
Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.
Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.
45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Console cleanliness moved to its own scenario
---------------------------------------------
``test-header-navbar-logos.js`` failed in CI when port-ui's Keycloak
silent-check-sso iframe was blocked by frame-ancestor CSP — the
``expectNoUnexpectedDiagnostics`` call at the test's end caught a
console.error logged by chrome for the blocked iframe (chrome-error://
chromewebdata + ``Framing 'https://auth.…'``) and failed every retry.
That diagnostic was a generic "no surprises" tail, unrelated to the
logo assertions the test is named after.
- Drop the diagnostic-tail call from the logo test; it now does
exactly what its name says.
- New ``test-no-console-noise.js`` runs the diagnostic check as its
own scenario, with two noise filters enabled
(``ignoreMatomoConsoleNoise`` + ``ignoreOidcSilentCheckNoise``).
- New ``isOidcSilentCheckNoise`` helper in ``_shared.js`` matches the
``chrome-error://chromewebdata`` URL plus the ``Framing '…'`` /
``silent-check-sso`` text patterns. These are test-env config
artefacts (Keycloak ``frame-ancestors`` not whitelisted), not
dashboard regressions.
- ``expectNoUnexpectedDiagnostics`` learns a corresponding
``ignoreOidcSilentCheckNoise`` option.
_shared.js no longer hides single-consumer code
-----------------------------------------------
Audited every export of ``_shared.js`` against every ``test-*.js``
consumer; ten symbols had exactly one caller and were therefore not
"shared". Moved each to the test file that owns it:
- ``runGuestFlow`` / ``runBiberFlow`` / ``runAdminFlow`` →
matching ``test-*-persona.js`` (now ``require("./personas")`` direct)
- ``assertCspMetaParity`` / ``assertCspResponseHeader`` /
``expectNoCspViolations`` / ``installCspViolationObserver`` plus
``canonicalDomain`` → ``test-csp-canonical-domain.js`` (CSP observer
install is now a local ``test.beforeEach`` in that file)
- ``dashboardTargetRoles`` parser → ``test-tile-reachability.js``
- ``cdnBaseUrl`` + each prefix (sharedCss / roleCss / sharedJs) →
``test-shared-css.js`` / ``test-cdn-role-stylesheet.js`` /
``test-logout-js-injection.js``
- ``platformLogoUrl`` + ``platformFaviconUrl`` →
``test-header-navbar-logos.js``
- ``appBaseUrl`` / ``oidcIssuerUrl`` / ``loginUsername`` /
``loginPassword`` / ``expectedOidcAuthUrl`` → ``test-oidc-login.js``
Dropped completely as dead test code: ``companyLogoUrl`` (no
spec ever read it) and its ``COMPANY_LOGO_URL`` env render in
``playwright.env.j2``. ``DASHBOARD_COMPANY_LOGO_URL`` in
``vars/main.yml`` stays — ``config.yaml.j2`` still consumes it for
the port-ui company.logo.source field.
``_shared.js`` is down to genuinely-shared state: ``dashboardJsBaseUrl``
+ ``matomoBaseUrl`` (each ≥ 2 callers), the diagnostic plumbing
(``attachDiagnostics`` / ``expectNoUnexpectedDiagnostics`` /
``waitForDashboardReady`` / ``waitForResourceResponse``), the noise
predicates, ``beforeEach``, and the ``skipUnlessServiceEnabled``
re-export. 116 LOC of state/exports collapse to 63.
Hitchhikers
-----------
- ``vars/main.yml`` loses an outdated docstring above the branding
SPOTs (the SPOTs themselves stay; ``config.yaml.j2`` consumes them).
- ``sys-svc-compose/defaults/main.yml``: linter re-joined the
multi-line ``compose_repository_pull`` default onto one line; pure
formatting, no semantic change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Playwright spec split
---------------------
The 747-line ``playwright.spec.js`` monolith is split into a thin
orchestrator + ``_shared.js`` (env, ``beforeEach``, common helpers) +
one ``test-*.js`` per scenario, matching the convention used by
web-app-fediwall/friendica/nextcloud. Each scenario file holds its own
specialised helpers (e.g. the auth/dropdown helpers live in
``test-oidc-login.js``, image-loaded assertions live in
``test-header-navbar-logos.js``); only helpers used by ≥ 2 tests stay
in ``_shared.js``. ``dashboard-card-flow.js`` becomes
``test-tile-reachability.js`` (git-detected rename) — the
``runDashboardCardScenario`` helper had a single consumer, so the
indirection is gone.
SPOT for branding URLs
----------------------
The ``'https:' ~ source if source.startswith('//') else source``
transformation lived inline in ``config.yaml.j2`` and was duplicated in
``playwright.env.j2`` when the logo assertion was added. Three new
SPOTs in ``vars/main.yml``:
- ``DASHBOARD_PLATFORM_LOGO_URL``
- ``DASHBOARD_PLATFORM_FAVICON_URL``
- ``DASHBOARD_COMPANY_LOGO_URL``
Both templates now reference these vars; the inline transformations
and the now-redundant ``{% set ... %}`` lines in ``config.yaml.j2``
are removed. The slurp + parse of the rendered config in
``tasks/01_core.yml`` stays because it covers the host-override case
(``DASHBOARD_CONFIG_INV_PATH`` template may use different branding),
but is now documented as the override-discovery path rather than the
default source of truth.
port-ui 2.0.0 + test alignment
------------------------------
- ``meta/services.yml``: port-ui pinned ``1.2.0 → 2.0.0`` (probe-first
asset resolver, see kevinveenbirkenbach/port-ui#2.0.0).
- ``test-header-navbar-logos.js``: drop the ``asset`` skip-gate (port-ui
>= 2.0.0 embeds the URL directly instead of producing the broken
``/static/<absolute-url>``), replace the shape regex with an
exact-match against the SPOT ``PLATFORM_LOGO_URL`` env var, and add
a favicon assertion that checks the rendered ``<link rel="icon">``
href against the SPOT ``PLATFORM_FAVICON_URL``.
- ``test-cdn-role-stylesheet.js``: keep the ``asset`` skip-gate — that
test exercises ``web-svc-cdn`` which depends on ``web-svc-asset`` as
its file backend, unrelated to the port-ui bug.
- ``playwright.env.j2``: rendered ``ASSET_SERVICE_ENABLED``,
``PLATFORM_LOGO_URL``, ``PLATFORM_FAVICON_URL``, ``COMPANY_LOGO_URL``.
- ``meta/services.yml``: drop the now-stale
``# nocheck: playwright-service-flag`` annotation on the ``asset``
service block.
.claude/settings.json
---------------------
Resort ``permissions.allow`` after the runtime appended
``WebFetch(domain:cloud.veen.world)`` at the end of the array during
the asset-resolver work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes direct process.env.<X>_SERVICE_ENABLED reads (and bespoke
per-role <ROLE>_OIDC_ENABLED / SERVICE_* / OIDC_ENABLED env flags) in
favor of skipUnlessServiceEnabled/isServiceEnabled from
roles/test-e2e-playwright/files/service-gating.js, per
docs/contributing/artefact/files/role/playwright.specs.js.md service-
gating contract.
Per-role changes:
- discourse, matrix, dashboard: replace process.env.<X>_SERVICE_ENABLED
with the helper; drop now-redundant nocheck markers on the canonical
env flags.
- taiga: drop TAIGA_OAUTH2_ENABLED / TAIGA_OIDC_ENABLED env lines;
consume OAUTH2_SERVICE_ENABLED / OIDC_SERVICE_ENABLED via the helper.
- opentalk, opencloud, nextcloud: drop <ROLE>_OIDC_ENABLED (and
NEXTCLOUD_LDAP_ENABLED) env lines; spec switches to
isServiceEnabled('oidc'|'ldap').
- moodle: drop SERVICE_OIDC / SERVICE_LDAP / SERVICE_LAM in favor of
the canonical OIDC_SERVICE_ENABLED / LDAP_SERVICE_ENABLED plus a new
LAM_SERVICE_ENABLED rendered from 'web-app-lam' group membership.
- decidim: drop OIDC_ENABLED env line; SSO-button-visible scenario
uses skipUnlessServiceEnabled('oidc').
Dashboard asset test split:
The previously bundled "dashboard loads core css, javascript,
simpleicons, and logo assets" test triple-gated (css/cdn/logout)
skipped the entire surface when any single shared service was off.
Split into six focused tests so disabling one service skips only its
specific assertion:
- shared CSS injection (gate: css)
- CDN-served role-local stylesheet (gate: cdn)
- logout.js injection (gate: logout)
- simpleicon-backed cards (gate: simpleicons; nocheck dropped)
- role-core JS modules + logos (ungated, role-core)
- iframe-sync postMessage handler (ungated, role-core)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root causes diagnosed inside the running container per
docs/agents/action/iteration/playwright.md, fixed in repo, validated
via inner-loop rerun-spec.sh, and finally re-deployed via FULL matrix
variant=0 (all dashboard tests green) and reuse-kept variant=1
(persona/asset/login-swap tests collapse cleanly via env + test.skip
gates, 2 passed + 5 skipped, failed=0).
- templates/nginx/sso.html.conf.j2: serve the standard Keycloak
silent-check-sso body — `parent.postMessage(location.href,
location.origin)`. Without that script, the hidden iframe never
reports its URL back to the parent and `keycloak.init()` hangs
forever; the role's OIDC adapter then never wires its click
interceptor and the persona helper races into the static `href`
without PKCE state.
- templates/javascript/oidc.js.j2: set `window.__oidcLoginReady = true`
after `wireLoginLogoutClicks()` so persona helpers can detect that
the JS-wrapped click handler is in place BEFORE clicking. The flag
name SPOT is `OIDC_LOGIN_READY_FLAG` in
`roles/test-e2e-playwright/files/personas/utils/keycloak.js`.
- personas/utils/keycloak.js: new `clickOidcLoginLink(page, loginLink)`
helper — single source of truth for the OIDC-link click ritual
(waitFor visible → waitFor `__oidcLoginReady` → click → waitForURL
`/openid-connect/auth/`). Replaces the inline duplicated block in
biber.js + admin.js, and removes the static-href direct-nav fallback
that broke PKCE-enforced clients silently.
- personas/biber.js + admin.js: delegate to `clickOidcLoginLink` and
drop the `^...$` regex anchors on the Login link locator. Bootstrap
`data-bs-toggle="tooltip"` moves the link's `title` into
`data-bs-original-title` and the accessibility tree pulls THAT into
the link's accessible name (e.g. "Login to Infinito.Nexus on
infinito.example") — the anchored regex matched zero links. The
`\s*` between `log` and `in` keeps the substring pattern from
matching `Logout`/`logoff`.
- templates/playwright.env.j2: render
`PERSONA_{BIBER,ADMINISTRATOR}_BLOCKED` from
`services.oidc.enabled`. Variant=1 disables every shared service —
with OIDC off there is no Login link to click, so the persona
scenarios MUST collapse to a clean skip instead of failing loudly.
- files/playwright.spec.js: add `test.skip(\!cdnEnabled || \!logoutEnabled, ...)`
on the asset-loading test and `test.skip(\!oidcEnabled, ...)` on the
Login → Account swap test. The bespoke tile-reachability loop
already skips cleanly when `DASHBOARD_TARGET_ROLES_JSON` is empty,
so no further gate needed there.
- TODO.md: removed. The autonomy escape clause is satisfied.
- docs/requirements/019-playwright-meta-services-parity.md: strike
through `web-app-dashboard` row (v0+v1 closed) and document the
silent-check-sso / `__oidcLoginReady` / Bootstrap tooltip
contracts inline in the row's notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the verbose
`(lookup('applications', application_id).server.domains.canonical | first)`
chain with the dedicated `lookup('domain', application_id)` plugin
across every playwright.env.j2 that renders `CANONICAL_DOMAIN`. The
helper already exists at plugins/lookup/domain.py and resolves the
canonical primary domain via the cached domain-merge pipeline, so the
inline canonical-list traversal is unnecessary noise that obscures
the actual contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Render BIBER_USERNAME/BIBER_PASSWORD on the role-local env templates
that drive the shared biber persona helper and, where the
administrator branch shares the same template, ADMIN_USERNAME/
ADMIN_PASSWORD too. Without these the shared persona helper read an
empty credential pair, skipped the Keycloak login step, parked on the
auth page, and the auth-marker assertion failed loudly with no clue
that the cause was a missing env declaration.
For web-app-pixelfed the previous indirection through
PIXELFED_PLAYWRIGHT_FIRST_USER_KEY / _SECOND_USER_KEY rendered the
same biber/administrator credentials under the role-local
LOGIN_USERNAME_{FIRST,SECOND} aliases. Drop the indirection and
consume BIBER_*/ADMIN_* directly in the role's spec so the env
contract matches the rest of the persona-driven roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expose compose.services.css.enabled as CSS_ENABLED in the dashboard
playwright env and have the "dashboard loads core css, javascript,
simpleicons, and logo assets" test call `test.skip(\!cssEnabled, …)`
when the shared CSS service is disabled for the deployment (e.g.
FULL_CYCLE runs with SERVICES_DISABLED="css …"), so the scenario no
longer flags a failure on an intentionally absent asset pipeline.
Closes Verification checkboxes in requirement 003. Each spec exercises
the `lookup('applications')` + `lookup('users')` pathway end-to-end:
login via lookup('users'), OIDC round-trip from dashboard (where the
app supports it), canonical-domain DOM assertion, and a full CSP
assertion covering every directive emitted by csp_filters.build_csp_header
(15 directives), meta-parity, enforced policy (not Report-Only), and
zero securitypolicyviolation events.
The dashboard spec is extended in place with the CSP + canonical-domain
assertions while keeping its existing login/logout coverage.
rerun-spec.sh is pointed at the Jinja template + the staged copy of
package.json (the repo has templates/package.json.j2, not files/package.json),
so the inner spec-only iteration loop works without re-deploying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lookups for 'service', 'service_registry', and 'applications_current_play'
always re-derive the merged applications mapping from inventory+role defaults
instead of trusting the passed kwargs, which could be shadowed or left
unrendered by nested template evaluations.
Dashboard/keycloak tests now consume the administrator password via
lookup('users','administrator').password so the rendered Playwright .env
matches the realm user provisioned by Ansible. Remove the hardcoded
DASHBOARD_KEYCLOAK_TEST_LOGIN_PASSWORD / KEYCLOAK_LOCAL_REALM_ADMIN_PASSWORD
overrides that previously masked the mismatch.
run_one.yml: drop users/applications from the template vars block. Those
entries created a shadowing context that broke nested lookup('users', ...)
inside lookup('template', ...).
utils/runtime_lookup_data: decrypt Ansible 2.19 EncryptedString values at
the lookup boundary and try templar rendering before falling back to
aggregated defaults when override mappings arrive as placeholders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace legacy filter with the canonical lookup plugin across
nextcloud config, group_vars, dashboard template, and vars
template. Switches Talk TURN from onboard to standalone coturn
by default; gates talk container on spreed plugin flag.
Adds lint test to catch remaining get_app_conf usages.
Adds MATOMO_ENABLED to the Playwright env template, derived from the
Ansible application config, so the matomo tracking assets test can be
skipped when SERVICES_DISABLED=matomo is set (e.g. in the minimal CI
deploy step).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
