Nobody could sign in to Odoo through Keycloak. Odoo sent an http redirect
URI, and Keycloak refused it as "Invalid parameter: redirect_uri".
The addon auth_oauth_https builds that URI from web.base.url. Two things
kept it from doing so after the image moved to Odoo 20.
Its manifest declared version 19.0.1.8.0, and Odoo marks a module whose
version names another series as not installable. It only warns about that,
so the install task stayed green and the addon was missing. The manifest
now declares 1.8.0, which Odoo prefixes with its own series.
Once installed, the addon read the parameter with get_param, which Odoo 20
replaced by typed getters. The call raised AttributeError, a broad except
logged it and fell back to the URL of the request, which is http behind
the proxy. The addon reads the parameter with get_str now and no longer
catches the error.
A deploy now fails when the addon is not installed afterwards. The check
reads ir_module_module, because the output of "odoo -i" does not tell an
installed module from a skipped one.
The spec logged out by navigating to /web/session/logout. Odoo 20 accepts
POST only on that route and answered 405, so both SSO tests failed after
a successful login. The spec logs out through the user menu now.
A host that runs the suite with several workers still failed it. Odoo
computes the session token of an account from its OAuth access token, so
every new sign-in ends the other sessions of that account, and the spec
files share the administrator. The role caps its suite at one worker
with PLAYWRIGHT_MAX_WORKERS, which the central config now honours, and
the spec asserts the cap.
Closes#822
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.
The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.
Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.
Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.
Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.
45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the per-template Jinja loop (which mis-unpacked dict2items and rendered every flag as KEY_ADDON_ENABLED=false, silently skipping every addon spec) with a single lookup('addon_env_flags', application_id). The plugin is the single source of truth for the <ADDON_ID>_ADDON_ENABLED flags consumed by tests/addon-gating.js: env-key derivation matches addon-gating.js exactly, and a flag is true only when the addon is both enabled AND required (required:false addons skip). Switch all six role templates to the lookup, extend the env-file schema lint to recognise whole-line {{ ... }} expressions, and add the plugin unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Matcher in web-app-prometheus's parametrized scrape test now also
reads `labels.app`. Native scrape jobs (set `app="web-app-<id>"` via
each role's `templates/prometheus.yml.j2` fragment) now match alongside
blackbox-healthz targets. A single iteration asserts up=1 for both
surfaces per consumer, so per-app native tests become redundant.
- Scenario I (`/metricz`) iterates `prometheusTargetRoles` and asserts
each consumer's `app="<role_id>"` label is present in the
lua-resty-prometheus output. Subsumes the per-app metricz-label spot
checks.
- Remove four redundant per-app prometheus tests from web-app-gitea and
web-app-mattermost playwright specs (metricz-label + native-scrape
up=1), drop the orphaned `PROMETHEUS_BASE_URL=` declarations from
their env.j2 files, and clean up `OIDC_ISSUER_URL=` in gitea's env.j2
(only used by the removed scenarios).
- Drop the dead `const prometheusBaseUrl` declarations plus their
eslint-disable shut-up comments from web-app-fider, web-app-odoo, and
web-app-taiga playwright specs, along with the matching
`PROMETHEUS_BASE_URL=` blocks in those roles' env.j2.
- Correct the persona-required-envs lint: only `MATOMO_BASE_URL` is
actually consumed (by `personas/utils/csp.js`). `DASHBOARD_BASE_URL`
and `PROMETHEUS_BASE_URL` were aspirationally required by a lint
whose docstring described persona-collapse behaviour that no persona
implements. Update `_REQUIRED_URLS` and the docstring, and drop the
unused `\!dashboardBaseUrl` token from `_ALLOWED_SKIP_GUARDS` in
test_strict_mode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The generic `runBiberFlow` / `runAdminFlow` helpers in
roles/test-e2e-playwright/files/personas/{biber,admin}.js single-click
the role's "Sign In" link and expect the next URL to contain
`openid-connect/auth`. Odoo's login page is a two-step OIDC flow
(landing → username/password form with OIDC button below → click OIDC
button → Keycloak), which the helper does not navigate. The
`biber/administrator: app → universal logout` tests fail with
`no in-app logout control reachable on the current authenticated
surface` because the test lands back on the public Odoo landing page.
The role already ships bespoke biber+admin coverage:
- `odoo: admin sso login, verify ui, logout`
- `odoo: biber sso login, verify ui, logout`
Both drive the full OIDC flow through Odoo's own login surface and
both have been passing in CI. The generic persona tests are
redundant for Odoo and add nothing on top.
Set `PERSONA_BIBER_BLOCKED=true` / `PERSONA_ADMINISTRATOR_BLOCKED=true`
in `templates/playwright.env.j2` so the generic helpers cleanly
`test.skip(...)` per the documented escape clause.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pattern B in CI 25939428268: `biber: dashboard → app → universal logout`
and `administrator: dashboard → prometheus → app → universal logout`
timed out with "Expected redirect back to <role-domain> after Keycloak
login. Received https://dashboard.infinito.example/".
The personas (`runBiberFlow` / `runAdminFlow`) navigate via
`page.goto(APP_BASE_URL)` and then assert the post-Keycloak URL
contains the role's `canonicalDomain`. Their documented contract
(biber.js / admin.js header):
> Each role's biber scenario therefore visits its OWN canonical URL
> directly (no dashboard tile click) and exercises only that role.
After the recent `e14606b6c` flipped APP_BASE_URL to the dashboard URL
for several roles, that contract broke: the persona hit the dashboard
under oauth2-proxy, Keycloak round-tripped back to the dashboard, and
the canonical-domain assertion failed.
Fix per role:
- gitea: APP_BASE_URL back to `lookup('tls', application_id, 'url.base')`.
No spec-internal `page.goto("/")` calls, so no further changes needed.
- odoo: APP_BASE_URL back to the role's own URL AND render an explicit
DASHBOARD_BASE_URL + DASHBOARD_SERVICE_ENABLED + PROMETHEUS_BASE_URL.
The spec-internal "dashboard to odoo" tests now build their dashboard
URL from DASHBOARD_BASE_URL instead of relying on baseURL, and gate
with `skipUnlessServiceEnabled("dashboard")` so they skip cleanly in
matrix-deploy variants that disable the dashboard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The shared biber/admin persona helpers in
roles/test-e2e-playwright/files/personas/*.js collapse the persona
scenario when either APP_BASE_URL or CANONICAL_DOMAIN is empty. The
twelve roles below shipped their env template without
CANONICAL_DOMAIN, so the helper fell into the auth-less collapse
branch and silently `test.skip()`-ed the persona scenarios despite
the roles having an actual Keycloak-backed auth surface. Render
CANONICAL_DOMAIN on every role whose spec drives a real persona so
the scenario runs end-to-end and a real regression cannot hide
behind a silent skip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into another ~28 roles
(akaunting, baserow, bluesky, bookwyrm, bridgy-fed, decidim, discourse,
fider, friendica, fusiondirectory, gitea, hugo, jenkins, joomla, kix,
matrix, mattermost, mig, moodle, odoo, opencloud, peertube, pixelfed,
postmarks, sphinx, taiga, yourls, web-svc-libretranslate, web-svc-xmpp).
Each role gets the spec.js persona scenarios plus matching
meta/services.yml gate entries and playwright.env.j2 env wiring;
admin-flow callers pass a role-specific adminInteraction that drives
an actual management surface so the body is not a stub.
Reason: continuation of d3e5ebe44 — req 019 Rule 3 demands every
role exercise the canonical guest/biber/administrator flows; this
batch closes the long-tail role list against the central library so
per-role drift cannot reintroduce the stub-shaped bodies the new lint
forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces legacy `users.KEY.field` and `users['KEY'].field` references with
the runtime `lookup('users', 'KEY').field` pattern across vars, templates,
and tasks so the role aligns with the new users lookup plugin.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses review feedback: biber and other LDAP users are now
auto-created on first SSO login via Keycloak OAuth signup rather
than hardcoded in Ansible.
OAuth template user mechanism:
- Sets website.auth_signup_uninvited=b2c to allow OAuth signup
- Creates _oauth_template user with Internal User group
- Sets base.template_portal_user_id (key param Odoo actually uses)
- Fallback grants Internal User to existing OAuth users
Adds biber Playwright test to verify OAuth auto-creation works.
