Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.
The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.
Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.
Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.
Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.
45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the per-template Jinja loop (which mis-unpacked dict2items and rendered every flag as KEY_ADDON_ENABLED=false, silently skipping every addon spec) with a single lookup('addon_env_flags', application_id). The plugin is the single source of truth for the <ADDON_ID>_ADDON_ENABLED flags consumed by tests/addon-gating.js: env-key derivation matches addon-gating.js exactly, and a flag is true only when the addon is both enabled AND required (required:false addons skip). Switch all six role templates to the lookup, extend the env-file schema lint to recognise whole-line {{ ... }} expressions, and add the plugin unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause: upstream friendica-addons/ldapauth's `ldap_createaccount`
returns the raw `User::create()` result (`['user' => UserRow, 'password' => '']`),
but the `authenticate` hook only sets `authenticated = 1` when
`$user['uid']` is non-empty — never true for the autocreate path. First-
login LDAP users (e.g. biber) were inserted into `friendica.user` but
the session was never started, so /network kept serving the login form.
Encoded as an idempotent in-container patch task in `tasks/02_ldap.yml`
+ `files/patch_ldapauth_autocreate.sh` (reports PATCHED on first run,
ALREADY on subsequent runs). Runs whenever LDAP is wired, gated by
FRIENDICA_LDAP_ENABLED.
Other changes in the same stack:
- meta/services.yml: drop keycloakpassword addon (oauth2-proxy in front
owns SSO); ldap.enabled now follows svc-db-openldap group presence so
the variant matrix can disable it without a static override.
- tasks/01_core.yml + tasks/06_user_bootstrap.yml: pre-create the
administrator only in LDAP-disabled variants — when LDAP is wired,
ldapauth materialises the row on first admin login.
- files/playwright.spec.js: variant-aware spec replacing the parked
test.skip cases. Three tests now run: guest flow (always), admin
native login (picks double-login vs direct based on
OAUTH2_SERVICE_ENABLED), biber + admin side-by-side in isolated
contexts. PERSONA_* still blocked — friendica has no header-trusted
autocreate path.
- templates/playwright.env.j2: expose BIBER_* + service-gate flags.
- inventories/development/default.yml: biber email aligned with the
oauth2-proxy `email_domains = infinito.example` filter so biber's
Keycloak login isn't 403'd at the proxy.
- TODO.md: SAML migration outlined as the long-term replacement for
the double-login (currently second hop silently bypasses Keycloak 2FA).
Drive-by: trim double-space before `# nocheck` in
roles/sys-aur/meta/services.yml.
Verified on VARIANT=0 (oauth2 + ldap + oidc all enabled): three
playwright tests green, failed=0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together stop one badly-behaved role from stalling
the whole deploy:
1. `roles/test-e2e-playwright/tasks/run_one.yml` — pass `timeout:
{{ TEST_E2E_PLAYWRIGHT_RUN_TIMEOUT }}` to the `command` task. New
default `1200` (20 min) lives in `defaults/main.yml`. Without an
outer Ansible-side cap, a single hung test could burn 5 min ×
3 retries × N specs before either `--max-failures` or the
deploy supervisor noticed; friendica's run today sat at 27 min
before the operator killed it manually.
2. `roles/test-e2e-playwright/defaults/main.yml` — append
`--max-failures=5` to the default Playwright command. Once five
tests have failed, the sidecar stops instead of chewing through
the rest of the retry budget for tests that are obviously broken.
3. `roles/web-app-friendica/templates/playwright.env.j2` — declare
`PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`.
Friendica authenticates against its own local DB / LDAP via the
`ldapauth` addon, not OIDC. The generic `runBiberFlow` /
`runAdminFlow` helpers drive an OAuth2-proxy + Keycloak round-trip
that Friendica cannot reconcile, so the generic persona tests
wedge on the Keycloak sign-in form. The bespoke admin/biber tests
above own the actual login flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The three playwright.env.j2 templates rendered
`APP_BASE_URL=lookup('tls', 'web-app-dashboard', 'url.base')` — a
stray copy-paste from the dashboard-tile flow that put the persona
helper on the dashboard login instead of the role's own surface.
After the dashboard's Keycloak round-trip the URL stayed on
dashboard.infinito.example, the canonical-domain poll
(`taiga.kanban.*`, `mattermost.*`, `social.*`) timed out at 60s and
the universal-logout test failed with
`Expected redirect back to <app>.infinito.example after Keycloak login`.
Point each `APP_BASE_URL` at the role's own canonical URL (the
project-wide convention used by baserow, bluesky, bookwyrm, akaunting,
hugo, jenkins, mig, moodle, …). The persona helper now lands on the
role's surface, the OIDC chain redirects to that role's Keycloak
client, the assertion passes, and the in-app logout step has a real
authenticated surface to operate on.
The shared biber/admin persona helpers in
roles/test-e2e-playwright/files/personas/*.js collapse the persona
scenario when either APP_BASE_URL or CANONICAL_DOMAIN is empty. The
twelve roles below shipped their env template without
CANONICAL_DOMAIN, so the helper fell into the auth-less collapse
branch and silently `test.skip()`-ed the persona scenarios despite
the roles having an actual Keycloak-backed auth surface. Render
CANONICAL_DOMAIN on every role whose spec drives a real persona so
the scenario runs end-to-end and a real regression cannot hide
behind a silent skip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into another ~28 roles
(akaunting, baserow, bluesky, bookwyrm, bridgy-fed, decidim, discourse,
fider, friendica, fusiondirectory, gitea, hugo, jenkins, joomla, kix,
matrix, mattermost, mig, moodle, odoo, opencloud, peertube, pixelfed,
postmarks, sphinx, taiga, yourls, web-svc-libretranslate, web-svc-xmpp).
Each role gets the spec.js persona scenarios plus matching
meta/services.yml gate entries and playwright.env.j2 env wiring;
admin-flow callers pass a role-specific adminInteraction that drives
an actual management surface so the body is not a stub.
Reason: continuation of d3e5ebe44 — req 019 Rule 3 demands every
role exercise the canonical guest/biber/administrator flows; this
batch closes the long-tail role list against the central library so
per-role drift cannot reintroduce the stub-shaped bodies the new lint
forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop env declarations that were rendered but never consumed by the
matching playwright.spec.js, surfaced by
tests/lint/ansible/roles/test_playwright_env_keys_used.py:
- web-app-friendica: OIDC_ISSUER_URL
- web-app-mailu: MAILU_DOMAIN
- web-app-opentalk: OPENTALK_OPENCLOUD_URL
- web-svc-xmpp: OIDC_ISSUER_URL, ADMIN_USERNAME, ADMIN_PASSWORD
The keys will land again only when an actual scenario consumes them.
For the deeper coverage gap that some of these env files imply (xmpp
in particular has no auth scenarios despite declaring admin
creds), see requirements 017 (biber RBAC coverage) and 018 (LDAP
authentication coverage).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Friendica: admin dashboard login + biber-to-admin direct message flow,
both via the ldapauth addon path (no OIDC redirect). Logout exits
through the universal /logout endpoint.
Joomla: admin control-panel login at /administrator using the local
account provisioned from users.administrator.*, with the universal
/logout confirmed by re-rendering of the admin login form. Joomla is
a CMS, so no cross-user messaging scenario is covered.
Both roles gain templates/playwright.env.j2 (the discovery marker for
test-e2e-playwright) and files/playwright.spec.js.
