The shared Playwright config passes --unsafely-treat-insecure-origin-as-secure
for every .onion origin in the environment, but Playwright's default
chrome-headless-shell ignores that switch. On the tor axis the odoo origin
therefore stayed an insecure context: navigator.clipboard was undefined and
Odoo 20's onboarding tour crashed with "Cannot read properties of undefined
(reading 'writeText')" and "Cannot destructure property 'TourInteractive'",
opening error dialogs that the addon specs read as a missing module (odoo#2,
swarm tor, main run 38038541631).
When ODOO_BASE_URL is an onion URL the odoo specs now use channel "chromium",
the full Chromium build in the Playwright image, and block service workers;
clearnet runs keep every option unchanged. The worker is blocked because a
secure context lets Odoo register /web/service-worker.js, whose fetch handler
answers a failed navigation with a cached page or /odoo/offline: gotoOnion
would get a 200 instead of the transport error it retries on. The switch stays
in the role: a use.channel in the shared config would also reach the Firefox
launch of onion-test.js, which rejects it with "Unsupported firefox channel",
and would move every other role's onion run to a browser none of them was
tested on.
Verified with Playwright 1.63 in the runner's staging layout and the real
odoo specs: an onion ODOO_BASE_URL resolves channel "chromium" and
serviceWorkers "block", also for the contexts _shared.js creates itself, and
launches chromium-<rev>/chrome-linux64/chrome; a clearnet one, and HEAD's
specs on onion, keep the headless shell and "allow". All spec files stay on
one worker, which the suite's one-worker test requires. Against Odoo 20's
service worker with a dropped connection, full Chromium returned the cached
/odoo page with status 200, while with the worker blocked page.goto threw
net::ERR_CONNECTION_FAILED as the headless shell does. In the v1.64.0 image
full Chromium made no :443 attempt to an onion host and the onion origin
reported isSecureContext with crypto.subtle, which the headless shell did not.
A local deploy of web-app-odoo variant 2 over Tor ran the odoo suite on
/ms-playwright/chromium-1248/chrome-linux64/chrome and passed all seven addon
specs, the one-worker test and the guest flow without a retry; in the variant
0 round the suites of the other roles kept chrome-headless-shell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nobody could sign in to Odoo through Keycloak. Odoo sent an http redirect
URI, and Keycloak refused it as "Invalid parameter: redirect_uri".
The addon auth_oauth_https builds that URI from web.base.url. Two things
kept it from doing so after the image moved to Odoo 20.
Its manifest declared version 19.0.1.8.0, and Odoo marks a module whose
version names another series as not installable. It only warns about that,
so the install task stayed green and the addon was missing. The manifest
now declares 1.8.0, which Odoo prefixes with its own series.
Once installed, the addon read the parameter with get_param, which Odoo 20
replaced by typed getters. The call raised AttributeError, a broad except
logged it and fell back to the URL of the request, which is http behind
the proxy. The addon reads the parameter with get_str now and no longer
catches the error.
A deploy now fails when the addon is not installed afterwards. The check
reads ir_module_module, because the output of "odoo -i" does not tell an
installed module from a skipped one.
The spec logged out by navigating to /web/session/logout. Odoo 20 accepts
POST only on that route and answered 405, so both SSO tests failed after
a successful login. The spec logs out through the user menu now.
A host that runs the suite with several workers still failed it. Odoo
computes the session token of an account from its OAuth access token, so
every new sign-in ends the other sessions of that account, and the spec
files share the administrator. The role caps its suite at one worker
with PLAYWRIGHT_MAX_WORKERS, which the central config now honours, and
the spec asserts the cap.
Closes#822
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A role-files-layout exemption frees a tree from files/<lang>/ because
its import paths are fixed from outside. The mirrored-test lint derived
its scope from that same directory, so every layout exemption silently
became a test exemption nobody ever argued for.
The language now comes from the file suffix. Test code is excluded by
scope rather than exempted: a test-* role and a files/playwright/ or
files/test/ tree are the tests, so demanding a test of them is circular.
The eleven files this newly reaches are framework-instantiated and carry
a stated reason each.
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same class as 642055a20, missed by the filter that selected it: files were grouped by whether they CONTAIN a waitFor state call rather than by whether the diff ADDS one, so four whose change is a pure deletion were sorted into the waiting-semantics bundle instead. Their diffs alter nothing but the discarded argument, and every added line here is a bare isVisible().
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each of the seven addon specs opened a fresh context and ran the full SSO
round-trip. Over Tor that is roughly twelve minutes apiece, and odoo 0 of run
31340965955 died on 'Test timeout of 600000ms exceeded' inside
gotoOnion -> loginToOdoo -> contacts.spec.js:13.
authenticatedContext caches the storage state after the first login and restores
it for later callers. A restored state that no longer renders the web client is
discarded and the full login runs, so an expired session degrades to today's
behaviour rather than failing the spec.
Coverage is not reduced: the SSO flow keeps its own dedicated tests in
playwright.spec.js:98 and :160 (admin and biber, log in / verify ui / log out),
which this change does not touch. The seven addon logins were repetition of an
already-covered path, and no addon spec asserts anything about the login or
needs an anonymous session. What does change is isolation -- the specs now share
one restored session instead of each starting clean.
The TEST_E2E_PLAYWRIGHT_ONION_TEST_TIMEOUT bump belongs in this commit: four
specs lost their own test.setTimeout(resolveTimeout(180_000)), which resolved to
900s on an onion. Without the bump those four would regress from 900s to 600s.
Cause B is untouched: the global ONION_RUN_TIMEOUT of 5400s left eight tests
unstarted. The cache should relieve it, but that is arithmetic on measured
single-login times, not an observed run, and raising a timeout that spans every
onion role needs its own measurement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A transient Debian/Ubuntu mirror fetch aborts a docker build layer with exit 100 (it killed the web-app-nextcloud swarm openldap build while the same image built fine elsewhere). New lint under tests/lint/docker/dockerfile/apt/ flags any RUN with apt-get update/install that omits Acquire::Retries; retrofit all 15 apt Dockerfiles with -o Acquire::Retries=3. Both apt lints move into the apt/ subpackage. dnf retries by default and apk has no equivalent flag, so both stay out of scope.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The upgrade ran via `container exec` into a live-serving odoo: serving backends hold RowShareLock while the migration DDL wants AccessExclusiveLock, so `-u all` and the workers deadlock. The install task now pauses crons, touches a marker and restarts; the entrypoint runs `-u all --stop-after-init` as the sole DB actor under a pg advisory lock, marker-gated, before serving. The marker path lives in one source (vars/main.yml), fed to the container env and the touch task.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.
Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).
Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
odoo -u all writes every ir_cron row while loading base data; live cron workers (N replicas x max_cron_threads) hold FOR NO KEY UPDATE row locks for the whole job, so the NOWAIT probe aborted with 'Record cannot be modified right now'. Snapshot active crons into infinito_ir_cron_pause and deactivate them (the UPDATE drains in-flight jobs), run all module operations inside a block whose always leg resumes from the snapshot (crash-safe: a killed run leaves the table and the next pause reuses it), and drop the redundant -u core update that ran outside the window.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sessions are redis, the filestore /var/lib/odoo is on the NFS-shared odoo_data volume, and cron serializes on Postgres advisory locks, so the only multi-replica hazard was the entrypoint's check-then-init TOCTOU: two replicas booting against an empty DB could both run odoo --init. Replace _schema_present with _guarded_init, which holds a database-scoped pg_advisory_lock across the schema check and the init in one connection (auto-released if the process dies). Uncontended (compose/single-node) it is a no-op, so behavior is unchanged there. Confidence ~95%.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The odoo config pins db_name with list_db=False, so on an empty schema
odoo preloads that database at boot, crashes (KeyError 'ir.http',
relation ir_module_module does not exist) and never binds its port; the
swarm converge wait then deadlocks because the healthcheck can never
pass and 01_init_database only runs after convergence.
Wrap the image entrypoint: probe the schema via psycopg2 against the
mounted odoo.conf, install the core modules once when absent, then hand
off to the stock entrypoint so the server boots healthy on the first
attempt. Proven live: a container against a fresh postgres installs
base and serves /web/health 200. Compose is idempotent (schema present
-> skip). The entrypoint file/paths come from a single ODOO_ENTRYPOINT_
SCRIPT_FILE SPOT (host abs/rel + docker dest), built via the same
build-arg pattern as web-app-suitecrm.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The OIDC/LDAP provisioning writes ir.config_parameter rows (web.base.url,
its freeze flag, signup scope) via direct SQL, which the four running
workers never observe: get_param() is ormcached per worker, so a worker
that auto-detected its base url keeps building OAuth redirect_uris that
Keycloak rejects ("Invalid parameter: redirect_uri") while other workers
serve correct links — the partial addon-spec failures in CI. Restart the
container once after provisioning so every worker boots with the frozen
values, and give the web client the same 120s first-render budget that
openModule already documents for cold asset compilation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The manager-local /mnt/custom-addons host bind was rejected on swarm workers. Bake files/addons into the odoo image so every node has it, and relax the config mount mode for the non-root odoo user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
odoo: sale_management.spec.js asserted `breadcrumb.or(appTitle)` without
`.first()`; when the Sales module installs correctly BOTH the menu brand
and the breadcrumb render, so `.or()` resolved to 2 elements and
`toBeVisible()` raised a strict-mode violation — failing the success case.
Add `.first()` (matching the same file's createQuotation locator).
nextcloud: the GitLab OAuth provisioning hard-`raise`d when no Organization
existed yet (default org is seeded asynchronously). Resolve/create it
defensively and add `retries/until` so a transient not-yet-seeded window no
longer fails the whole deploy.
jenkins: the deploy-lockdown service stop had no retry; under load systemd
intermittently refused with "Transaction is destructive". Add `retries/until`
so the transient contention is ridden out.
moodle: the shared `compose up` handler had no retry; a transient missing
external network (central DB stack still creating it) failed the deploy. Add
`retries/until`, matching the handler's existing compose-pull pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
odoo: sale_management.spec.js asserted `breadcrumb.or(appTitle)` without
`.first()`; when the Sales module installs correctly BOTH the menu brand
and the breadcrumb render, so `.or()` resolved to 2 elements and
`toBeVisible()` raised a strict-mode violation — failing the success case.
Add `.first()` (matching the same file's createQuotation locator).
nextcloud: the GitLab OAuth provisioning hard-`raise`d when no Organization
existed yet (default org is seeded asynchronously). Resolve/create it
defensively and add `retries/until` so a transient not-yet-seeded window no
longer fails the whole deploy.
jenkins: the deploy-lockdown service stop had no retry; under load systemd
intermittently refused with "Transaction is destructive". Add `retries/until`
so the transient contention is ridden out.
moodle: the shared `compose up` handler had no retry; a transient missing
external network (central DB stack still creating it) failed the deploy. Add
`retries/until`, matching the handler's existing compose-pull pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Playwright native-login helper clicked the first match of a broad 'form button[type=submit]' selector, which on the website login page non-deterministically resolved to the navbar search button (oe_search_button, type=submit) instead of the Log in button — an unstable element the click waited on until the 180s timeout, making the admin-login addon specs (account/contacts/crm/project/sale_management) flaky. Scope the selector to the login form (form:has(input[name=login])) and target the Log in button by role (getByRole button, name /log in/i).
Diagnosed from the CI error-context page snapshot (navbar search button before the login form) and the run's flaky pass/fail pattern with all failures at the login step; not yet re-run against a live Odoo deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In the SSO-disabled matrix variant, loginToOdoo waited for the auth_oauth SSO button (which never renders), so every addon spec timed out in clickOdooSsoButton — the run's actual failure (the earlier stock.spec union fix was a no-op there, since stock is addon-gated off in that variant). Branch loginToOdoo on the sso service flag: when SSO is disabled, log in natively via the Odoo /web/login form (input[name=login]/[name=password] with the provisioned administrator credentials, scoped .oe_login_form submit), so the native admin login is genuinely exercised and the addon specs run instead of skipping. When SSO is enabled the existing OIDC/Keycloak flow is unchanged.
Selectors verified against the deployed Odoo 19 login form captured in the CI trace (name=login/password, .oe_login_form button[type=submit]). Admin creds are the same lookup('users','administrator') values provisioned into res_users by tasks/05_admin.yml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
stock.spec.js applied .first() to each .or() operand (breadcrumb and appTitle) before combining them, so breadcrumb.or(appTitle) was a union of two locators that resolved to 2 visible elements (the stock.menu_stock_root brand link AND the 'Inventory Overview' breadcrumb) — expect().toBeVisible() requires exactly one match and threw a Playwright strict-mode violation. The two matches actually PROVE the stock module installed; the assertion message's 'module failed to install' premise was wrong. Mirror the passing account.spec.js pattern: drop the per-branch .first() and apply a single .first() after .or(). The assertion intent (Inventory identity must render) is preserved — no green-wash.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The account addon spec opened /odoo/accounting (the journal dashboard, which in Odoo 19 has no .o_account_dashboard and not the asserted invoice vocabulary) and timed out even though the account module is installed. Navigate to /odoo/invoicing — the canonical account.move customer-invoices action — and assert the invoices list/kanban plus account-specific vocabulary (customer invoice / register payment / invoice date / tax excluded). Verified green live against Odoo 19 via 'make compose-playwright role=web-app-odoo'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The account addon spec asserted bare .o_control_panel/.o_breadcrumb, which render on every Odoo page, so it passed green even when the account module was enabled-but-not-installed. Require Accounting domain vocabulary (Customer Invoices / Vendor Bills / Chart of Accounts / Journal Entries) and text-gate the generic containers, mirroring the crm/stock specs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the generic surface check with the Inventory app identity, no invalid-action
error, the stock.picking operations dashboard, and stock-specific operation types
(Receipts/Deliveries/Internal Transfers).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the generic surface check with the Sales app identity (breadcrumb/title),
the sale.order quotations list/kanban action, and the New create button.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the generic surface check with no-error-page + the CRM control panel and a
CRM-specific surface (lead/opportunity kanban, New, Pipeline) plus CRM domain
vocabulary, proving the crm module loaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the generic web-client surface check with module-specific signals: no Odoo
error dialog, the Accounting/Invoicing app brand in the navbar, and an
account-specific top menu (Customers/Vendors/Chart of Accounts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
loginToOdoo treated the OAuth round-trip's URL as proof of login, but the implicit-token round-trip can land back on Odoo without an established session (false positive), and an existing Keycloak realm session bounces straight to an authenticated Odoo without ever showing the login form. Rework: race the issuer URL against the web-client locator, fill the Keycloak form only when it actually appears, then confirm the authenticated desk by waiting for the web client to render (waitFor — the non-waiting isVisible check raced the SPA bootstrap and flaked). Retry twice, then fail loudly. Verified in-container: odoo 10 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Odoo bootstraps a per-module web-client asset bundle on first access; under serial Playwright load the first render can exceed a minute. openModule now waits up to 120s for the app shell (90s for navigation) instead of 60s, so the module-UI coupling assertion is not a false negative on a busy instance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move `rm -rf /var/lib/apt/lists/*` before `apt-get update` in every
Debian-based Dockerfile so a stale base-image-baked InRelease
(Valid-Until expired) cannot abort the build layer with exit 100. The
trailing rm at end-of-RUN never ran because `apt-get update` aborted
first; the leading rm forces a fresh fetch from the configured source.
Affected: moodle, bridgy-fed, bookwyrm (2 RUN), postmarks, taiga
backend, svc-db-openldap, svc-db-postgres, jenkins, joomla, mediawiki,
odoo, suitecrm (2 RUN), wordpress.
Add tests/lint/docker/dockerfile/test_apt_update_purge_lists.py to
enforce the contract repo-wide: every `RUN` containing `apt-get
update` MUST `rm -rf /var/lib/apt/lists/*` first in the same RUN, and
MUST NOT pass `Acquire::Check-Valid-Until=false` (which would silence
apt's replay-protection rather than refresh the list).
Rename roles/web-app-taiga/templates/Dockerfile.{back,front}.j2 to
Dockerfile.{backend,frontend}.j2 and update the matching tasks/
main.yml template refs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bespoke `test(...)` blocks that drive a real Keycloak SSO round-trip
(odoo, fider, mattermost, peertube) had no `test.skip(\!oidcEnabled, ...)`
guard. In matrix variants that render `OIDC_SERVICE_ENABLED=false`, the
Playwright locator for the SSO button never resolves; each test waits
300s, retries 2x, and 2-3 such tests in series blow past the 1200s
Ansible task-wrapper timeout — Bundle α odoo variant 1 reproduced this.
Pattern parity with `roles/web-app-moodle/files/playwright/playwright.spec.js`:
import `service-gating`, derive `<flag>Enabled` once, prefix each
affected test with `test.skip(\!<flag>Enabled, "...")`.
- odoo (verified empirically against running variant-1 stack via
`make run`: 4 skipped, 1 passed, 2.2s — was 1200s timeout)
- fider, mattermost, peertube: same pattern, 2 bespoke tests each
- kix: gates on `oauth2Enabled` + `ldapEnabled` instead — the role's
variant matrix has no `oidc` flag, the flow is
KIX → OAuth2-proxy → Keycloak → KIX-LDAP, both can be independently
disabled per variant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Matcher in web-app-prometheus's parametrized scrape test now also
reads `labels.app`. Native scrape jobs (set `app="web-app-<id>"` via
each role's `templates/prometheus.yml.j2` fragment) now match alongside
blackbox-healthz targets. A single iteration asserts up=1 for both
surfaces per consumer, so per-app native tests become redundant.
- Scenario I (`/metricz`) iterates `prometheusTargetRoles` and asserts
each consumer's `app="<role_id>"` label is present in the
lua-resty-prometheus output. Subsumes the per-app metricz-label spot
checks.
- Remove four redundant per-app prometheus tests from web-app-gitea and
web-app-mattermost playwright specs (metricz-label + native-scrape
up=1), drop the orphaned `PROMETHEUS_BASE_URL=` declarations from
their env.j2 files, and clean up `OIDC_ISSUER_URL=` in gitea's env.j2
(only used by the removed scenarios).
- Drop the dead `const prometheusBaseUrl` declarations plus their
eslint-disable shut-up comments from web-app-fider, web-app-odoo, and
web-app-taiga playwright specs, along with the matching
`PROMETHEUS_BASE_URL=` blocks in those roles' env.j2.
- Correct the persona-required-envs lint: only `MATOMO_BASE_URL` is
actually consumed (by `personas/utils/csp.js`). `DASHBOARD_BASE_URL`
and `PROMETHEUS_BASE_URL` were aspirationally required by a lint
whose docstring described persona-collapse behaviour that no persona
implements. Update `_REQUIRED_URLS` and the docstring, and drop the
unused `\!dashboardBaseUrl` token from `_ALLOWED_SKIP_GUARDS` in
test_strict_mode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adding `DASHBOARD_BASE_URL` (Pattern A) and `PROMETHEUS_BASE_URL` to the
playwright.env.j2 of these roles was required by the
`persona-required-envs` lint when `DASHBOARD_SERVICE_ENABLED` is
rendered. The companion `test_env_keys_used` lint then refused them
because no .js file consumed the keys.
Declare them as const in each spec so `process.env.<KEY>` appears
literally in a tracked source. The personas that the persona-required-
envs lint cites as consumers are blocked for these roles
(`PERSONA_*_BLOCKED=true`), so a benign const declaration with an
`eslint-disable-next-line no-unused-vars` is the smallest viable bridge
between the two lints.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pattern B in CI 25939428268: `biber: dashboard → app → universal logout`
and `administrator: dashboard → prometheus → app → universal logout`
timed out with "Expected redirect back to <role-domain> after Keycloak
login. Received https://dashboard.infinito.example/".
The personas (`runBiberFlow` / `runAdminFlow`) navigate via
`page.goto(APP_BASE_URL)` and then assert the post-Keycloak URL
contains the role's `canonicalDomain`. Their documented contract
(biber.js / admin.js header):
> Each role's biber scenario therefore visits its OWN canonical URL
> directly (no dashboard tile click) and exercises only that role.
After the recent `e14606b6c` flipped APP_BASE_URL to the dashboard URL
for several roles, that contract broke: the persona hit the dashboard
under oauth2-proxy, Keycloak round-tripped back to the dashboard, and
the canonical-domain assertion failed.
Fix per role:
- gitea: APP_BASE_URL back to `lookup('tls', application_id, 'url.base')`.
No spec-internal `page.goto("/")` calls, so no further changes needed.
- odoo: APP_BASE_URL back to the role's own URL AND render an explicit
DASHBOARD_BASE_URL + DASHBOARD_SERVICE_ENABLED + PROMETHEUS_BASE_URL.
The spec-internal "dashboard to odoo" tests now build their dashboard
URL from DASHBOARD_BASE_URL instead of relying on baseURL, and gate
with `skipUnlessServiceEnabled("dashboard")` so they skip cleanly in
matrix-deploy variants that disable the dashboard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces controller-side `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` calls with a project-local `database_query`
Ansible module that wraps `container exec -i <container> <psql|mysql>`
and pipes SQL via stdin. The module routes via the DB container's own
network so the same task works identically in shared mode (central
`svc-db-*` container) and non-shared mode (per-role `<entity>-database`
container) — the `container` field of `lookup('database', …)` is
shared-aware and the module just reads it. SQL payloads come from a
file (`query_file:`, preferred) or as a raw string (`query:`, escape
hatch). `named_args:` substitutes `%(name)s` placeholders with engine-
appropriate escaping (postgres single-quote-doubling, mariadb backslash
escapes), so role tasks stop string-interpolating Jinja into SQL.
New backends extend the module's `_ENGINES` dispatch table; everything
else is engine-agnostic. The module ships at `plugins/modules/` (Ansible
collection-style layout) — the shebang lint test learned to recognise
that path as a valid module location alongside the existing per-role
`library/` layout.
Two lint tests enforce the convention:
* `tests/lint/ansible/modules/test_no_raw_db_query_module_usage.py`
forbids `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` in role tasks. Opt out per-line with
`# nocheck: database-query` (used by `svc-db-postgres/tasks/02_init.yml`
where the role manages its own DB and the `database` lookup would be
self-referential).
* `tests/lint/ansible/modules/test_no_raw_database_query_string.py`
forbids inline `query:` on `database_query:` tasks. Opt out with
`# nocheck: database-query-raw` (no opt-outs in the current tree —
every SQL payload reaches the module through a `.sql` file under
`files/sql/`).
Role migrations:
* `web-app-peertube` — 3 tasks migrated, 3 new `.sql` files,
`templates/upsert-oidc-plugin.sql.j2` deleted in favour of
`files/sql/upsert-oidc-plugin.sql` + `named_args:`.
* `web-app-openproject` — 3 tasks migrated, 3 new `.sql` files for
the LDAP-auth-source check/update/insert chain.
* `web-app-odoo` — 33 tasks migrated, 9 new `.sql` files for the
LDAP / admin / init-database flows; 17 pre-existing OIDC `.sql`
files renamed snake_case → kebab-case and relocated from
`files/oidc/` to `files/sql/oidc/` (via `git mv` so rename history
stays intact).
Unit tests cover every dispatch-table entry, the named_args escaper
(strings, ints, floats, bools, None, single-quote and backslash
injection vectors), and the password-redactor that hides
PGPASSWORD/MYSQL_PWD assignments from the module's returned `cmd`
field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote `prefer-template` to error in the spec/persona JS rule block
and convert the nine remaining `a + 'literal' + b` URL-builders to
template literals. Behaviour-equivalent; brings stylistic consistency
with the rest of the codebase, which already uses template literals
throughout, and prevents new `+`-concatenations from sneaking past
the lint floor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote four cheap rules in `eslint.config.js`:
* `no-unused-vars`: warn → error (with `argsIgnorePattern: "^_"`)
* `no-var`: error
* `eqeqeq`: error (always)
* `playwright/no-wait-for-navigation`: off → error
* `linterOptions.reportUnusedDisableDirectives`: off → warn
Add a repo-root CommonJS block so `eslint.config.js` itself sees Node
globals (`require`, `module`).
Fix every flagged offender:
* 138 stale persona-helper imports across 70 spec files
(`isServiceEnabled`, `skipUnlessServiceEnabled`, `runAdminFlow`,
... — leftovers from the SPOT consolidation).
* 4 unused catch bindings → modern `catch {}` syntax.
* 2 unused `dashboardBaseUrl` declarations (minio, moodle).
* 1 unused `label` arg → renamed to `_label` to opt out per the
argsIgnorePattern.
* 1 intentionally-kept legacy helper in web-app-wordpress's spec
receives a per-line `eslint-disable-next-line` with reason; the
preceding docstring already explained why it stays.
* 1 `page.waitForNavigation` in web-app-fediwall's Mastodon OIDC flow
receives a per-line disable with reason — the navigation target
depends on the runtime OIDC issuer; `waitForURL` would need a
runtime-built pattern.
* 10 stale `// eslint-disable-next-line no-console` directives now
surfaced by `reportUnusedDisableDirectives`; removed.
ESLint after this pass: 0 errors, 154 warnings (148 pre-existing
`playwright/expect-expect` plus 6 `playwright/no-force-option` that
were already inherited from `playwright/flat/recommended`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five role specs (dashboard, nextcloud, odoo, pixelfed, taiga)
duplicated four small utility helpers across one-cluster-each
inline copies:
- isVisible(locator) — `locator.first().isVisible()`
with the strict-mode / detached
error swallowed.
- waitForFrameUrl(iframe, ...) — poll the embedded iframe URL
until it contains a matcher.
- findFirstVisibleCandidate(list) — pick the first visible
candidate from a label/locator
list and return the resolved
single-element locator.
- escapeRegex(value) — escape regex metacharacters
so a raw string can be used as
a `new RegExp(...)` source.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dom.js` collects
all four helpers (canonical 1-cluster bodies, byte-equivalent to the
former inline copies).
- `personas/utils/index.js` re-exports the new module so callers reach
the helpers via `require("./personas")`.
- The five specs strip their inline definitions and amend the
destructuring with whichever helpers their body still references.
Drift handling
--------------
None. All four helpers were 1-cluster across the call sites; this is
mechanical substitution.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- 0 inline `function isVisible|waitForFrameUrl|findFirstVisibleCandidate|escapeRegex`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourteen role specs (bigbluebutton, discourse, fider, gitea, mailu,
matrix, mattermost, odoo, openwebui, peertube, postmarks, prometheus,
yourls, libretranslate) each carried an inline `performOidcLogin`
that drove the Keycloak login form. The bodies fell into four
clusters split between the role-based selector strategy
(`getByRole`) and the legacy input-name strategy
(`input[name='username']`); the existing shared
`performKeycloakLogin` already covered both via `.or()` chains but
also asserted post-login navigation, so it could not be reused
verbatim.
Changes
-------
- `personas/utils/keycloak.js` gains `performKeycloakLoginForm
(target, username, password)` — form-fill + submit only, accepts
`Page` OR `FrameLocator` so iframe-embedded Keycloak pages work
without branching at the call site. The selectors are the union
of all four prior cluster shapes (role-based getter chained with
`.or(input-name fallback)`, sign-in matched against
`/sign in|login|log in/i` plus `input#kc-login` family). The
existing `performKeycloakLogin` and
`performKeycloakLoginExpectingDenial` now delegate to the new
helper, so their public API is unchanged.
- The 14 specs strip their inline `performOidcLogin` and amend
`require("./personas")` with `performKeycloakLoginForm`. Every
call site and comment reference is renamed in lockstep.
Drift handling
--------------
- Cluster 1 (5 specs) expected the accessible name `"Password"`
exactly; the canonical regex `/^password$/i` is case-insensitive
and matches every prior name plus lowercase variants — strictly
more permissive.
- Cluster 1's sign-in button matched `/sign in/i`; the canonical
matches `/sign in|login|log in/i` plus the legacy `input#kc-login`
family — strictly broader.
- Cluster 1 awaited `usernameField.press("Tab")` without a `.catch`;
the canonical wraps the press in `.catch(() => {})` so an
unexpected Tab failure no longer aborts the flow.
- Cluster 2 / 3 used `expect(usernameField).toBeVisible({ timeout })`
for the visibility wait; the canonical uses
`usernameField.waitFor({ state: "visible", timeout })`, the
Locator-method counterpart of the same wait — functionally
equivalent.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- 0 inline `performOidcLogin` definitions remain in any role spec,
and 0 stale references to the legacy name in any context (call
sites and comments are renamed in the same pass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
re-exported through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
`./personas` instead. Specs that only reference one of the two
carry only that one in the destructuring (no unused-import noise).
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
