Auth-isolated matrix variants (exactly one of LDAP / OIDC-OAuth2
enabled, and no non-auth `enabled: true` alongside) must explicitly
pin every non-auth service-key in the role's variant union to literal
`enabled: false, shared: false`. Without this, the variant's
auth-only test scope balloons into a full integration deploy via the
services.yml-default propagation, wasting CI minutes and obscuring
the regression signal.
Adds `tests/integration/roles/meta/variants/test_auth_isolation.py`
(rule key `variants-auth-isolation`) plus the matching catalog entry.
The trigger is narrow on purpose: a variant that enables auth
alongside other services is a deliberate integration test, not an
auth-isolated one, and the rule does not fire there.
Mass-fixes the 24 legacy auth-isolated variants the rule turned up:
each gets the missing `enabled: false, shared: false` declarations
appended to its `services:` block. Where the false-override clashes
with a static-literal base in meta/services.yml, the override line
also picks up a `# nocheck: variants-static-override` marker. The
svc-bkp-container-2-local variant 2 additionally carries
`# nocheck: variants-services-match` on the new keys because they
are resolver-only matrix hooks with no consumer contract in the base
meta.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/install/venv.sh
-----------------------
INFINITO_VENV_BASE is a static-default convention; the active VENV
may end up outside that base (e.g. \$HOME/.venvs/<name> when
/opt/venvs is not writable, or a wholly custom override). Probing
writability on the legacy base triggered a spurious sudo branch
whenever VENV lived elsewhere -- fatal in no_new_privs sandboxes
where sudo cannot escalate. Switch the writability + mkdir + chown
chain to \$(dirname "\${VENV}") so the script prepares the directory
it is actually about to create the venv in.
tests/lint/ansible/roles/web-app/playwright/test_app_base_url_uses_role.py
-------------------------------------------------------------------------
New lint: every role's templates/playwright.env.j2 must declare
APP_BASE_URL as a reference to the role's own surface. APP_BASE_URL
feeds the shared guest-persona helper and Playwright's global
baseURL; a role that hardcodes a sibling role name (most commonly
'web-app-dashboard') sends the guest test to the wrong site and
breaks any matrix variant where the sibling is not deployed
(ERR_SSL_UNRECOGNIZED_NAME_ALERT).
The 16 web-app role templates that violated the contract switch
from sibling lookups (`lookup('tls', 'web-app-dashboard', ...)`)
to the canonical `lookup('tls', application_id, ...)` form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace inline `networks:` blocks in every service of web-app-kix and
web-app-nextcloud with `{% include 'roles/sys-svc-container/templates/networks.yml.j2' %}`,
matching the convention the other ~70 roles already follow. The include
derives shared LDAP/DB/Ollama network attachments from `service_registry`
and the role's `services.*.shared` flags, so the hardcoded `openldap:`
attachment in kix's backend (and the missing shared-network handling in
nextcloud's proxy/HPB/whiteboard/recording services) become consistent.
Also fix a latent indent bug in web-app-nextcloud's HPB service: when
`NEXTCLOUD_HPB_NETWORK_MODE \!= 'bridge'`, `network_mode:` was rendered
at 2-space indent (service-name level) instead of 4 (service-property
level), which would have broken `docker compose config` in host/none
mode. Hidden until now by the `bridge` default.
Add `tests/integration/infrastructure/compose/test_network_includes.py`
to lock in the convention: forbids literal `networks:` keys in
compose.yml.j2 and requires exactly one top-level
`sys-svc-compose/networks.yml.j2` include per file (exempting templates
that exclusively use `network_mode:`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split the previous two-file plugin pipeline into three single-responsibility
stages and group them under tasks/plugin/, with per-plugin hooks now sitting
under tasks/plugin/hooks/. Each stage owns exactly one OCC call (app:install,
app:enable, config:app:set) and one continuation decision driven by the
nextcloud_install_status filter.
Layout:
tasks/
├── 06_plugin.yml (entry; includes plugin/01_install.yml)
└── plugin/
├── 01_install.yml (occ app:install + classify)
├── 02_enable.yml (occ app:enable + classify)
├── 03_configure.yml (occ config:app:set + run hook)
├── hooks/ (per-plugin extension tasks)
│ └── user_ldap.yml
└── README.md (pipeline contract + extension guide)
The new 02_enable.yml mirrors the version-compatibility tolerance that
01_install.yml already had: when occ rejects the enable with the
"is not compatible with this version of the server" marker, the
failed_when clause downgrades the task to a non-fatal classification,
emits a debug warning, and skips the entire 03_configure.yml include via
a single set_fact-driven flag instead of repeating the filter call on
every downstream task. This unblocks deploys against a Nextcloud release
whose appstore lags briefly while the plugin's compatibility metadata
catches up (e.g. metadata, drawio, files_bpm in the NC 33 window).
NEXTCLOUD_CNODE_PLUGIN_TASKS_PATH in vars/main.yml is rebased to the new
tasks/plugin/hooks/ directory so plugin hooks resolve at the new location
without touching call sites.
README.md follows the role + docs README contracts: one sentence per line,
no em dashes, RFC 2119 keywords, no historical narrative, cross-links to
the filter plugin and the role vars file.
NC 33's login form labels the username textbox simply "Account name"
(no "or email" suffix), so the prior regex matched nothing and every
login-via-native-form spec timed out at the 5-s default. Broaden the
pattern to accept either label shape.
The biber-LDAP scenario also failed at the logout assertion: the
universal-logout patch from sys-front-inj-logout rewrites the click
target to Keycloak's logout endpoint, which in user_ldap-only mode
ends on Keycloak's "You are logged out" info page rather than NC's
/login. Replace the negative shell-absence check with a positive
poll on the Keycloak logout URL that matches what the universal-logout
patch actually does.
Re-enable bbb, files_bpm, and metadata plugins. Their previous opt-out
existed only because the upstream Nextcloud appstore release lagged the
pinned server major, which made `occ app:install` fail and abort the
deploy.
Introduce a role-local Jinja filter `nextcloud_install_status` that
classifies the registered `occ app:install` result into a status dict
(ok/changed/runnable/already/incompatible). The install task now uses
this filter in `until`/`failed_when`/`changed_when`, so a plugin
emitting "is not compatible with this version of the server" terminates
the retry loop immediately, downgrades to a debug warning, and skips
the subsequent enable/configure path instead of failing the play.
Net effect: operators no longer have to toggle services.yml when an
upstream plugin temporarily lags the pinned Nextcloud version; the
deploy proceeds and the plugin auto-recovers on the next run once a
compatible release ships.
Per-app nginx gateways used proxy_pass with a literal docker container
hostname. nginx OSS resolves that hostname once at startup; if the
upstream container later restarts and gets a new IP (e.g. a partial
compose-up after credential change), the gateway proxies onto a dead
address until nginx itself reloads — the 502 we hit on taiga after a
reuse-kept replaced the taiga-back container.
Switch every per-app docker-hostname proxy_pass to a variable form so
the embedded docker DNS re-resolves at request time:
set $upstream_<rolename>_<purpose> http://<host>:<port>;
proxy_pass $upstream_<rolename>_<purpose>;
The full `resolver` directive (TTL + ipv6 toggle inherited from
NETWORK_IPV6_ENABLED) lives in one SPOT — NETWORK_DOCKER_DNS_NGINX_DIRECTIVE
in group_vars/all/08_networks.yml — so cache TTL and ipv6 stay
consistent across the stack.
In taiga the upstream `set` declarations hoist to server scope because
taiga-back is reused across /api/, /oidc/ and /admin/; in nextcloud
and kix each upstream is used exactly once so the declaration stays
local. Locations whose original proxy_pass stripped the location
prefix (`proxy_pass http://.../;` on /media/, /whiteboard/,
/standalone-signaling/) keep that semantics via an explicit
`rewrite ^<loc>(.*)$ /\$1 break;`, because variable-proxy_pass drops
the implicit prefix-strip.
kix's static files/kix-proxy.default.conf moves to
templates/kix-proxy.default.conf.j2 so it can reference the new
constant; the deploy task switches from copy to template accordingly.
Drive-by: scripts/tests/e2e/rerun-spec.sh docstring refresh — the
"only overwrites tests/playwright.spec.js" claim went stale when the
rerunner started restaging every role-local playwright/*.js.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Browser vendors deprecated X-Frame-Options/X-XSS-Protection/X-Download-Options/
X-Permitted-Cross-Domain-Policies; CSP (rendered from each role's
meta/server.yml via build_csp_header) is the modern replacement. To make the
SPOT actually win against upstream apps (e.g. Keycloak/Quarkus still emits
X-Frame-Options), the proxy now hides those headers on every location type.
Includes a fail-loud lint that catches any future add_header / Header set/add
re-introduction in .conf / .conf.j2 / .htaccess / .j2 / .jinja / .tmpl files.
- sys-svc-proxy: new upstream_strips.conf.j2 partial bundles HSTS + the 4
deprecated header strips; included in html/media/ws/upload + web-svc-logout
+ web-app-oauth2-proxy locations (nginx array-replacement inheritance forces
per-location placement)
- web-app-keycloak: meta/server.yml frame-ancestors allows the primary-domain
wildcard with the dynamic TLS protocol so dashboard silent-check-sso iframe
loads cleanly
- web-app-dashboard/playwright: drop isOidcSilentCheckNoise — root cause is
now fixed, no noise filter needed
- web-app-dashboard/nginx + web-app-nextcloud/nginx: remove the deprecated
add_header lines themselves
- tests + docs: new lint registers `deprecated-http-header` suppression rule
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
meta/services.yml — switch `lookup('config', application_id, ...)` to the
spelled-out role id `'web-app-nextcloud'`. `application_id` is undefined
when meta/services.yml is rendered (the applications registry is built
before any role-vars binding), which made the lookup return the raw
unrendered Jinja string instead of the resolved oidc.enabled value, and
in turn flipped legacy_login_mask.enabled to false in variant 1 where
it should have been true. The matching `twofactor_*` plugin entries in
the same file already use the hardcoded form for the same reason.
files/playwright/_shared.js — anchor `nextcloudCredentialSubmitPattern`
to `^(sign in|log in)$` so it matches only the credential-form submit
button. With auth.webauthn.enabled=true (the post-fix variant-1 state)
Nextcloud renders an additional `Log in with a device` button next to
the credential `Log in`, which the previous unanchored pattern matched
too — strict-mode violation, test fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Couple `legacy_login_mask.enabled` to `services.oidc.enabled` instead
of hard-coding it to false. The default flowed through to
`hide_login_form=true` in system_config, which hid the native
username/password form even on deployments without Keycloak — leaving
no usable login path in the OIDC-off variant (Playwright native-login
test timed out waiting for a form that was never rendered).
With the new formula:
- OIDC on → legacy_login_mask.enabled=false → hide_login_form=true (SSO only)
- OIDC off → legacy_login_mask.enabled=true → hide_login_form=false (native form)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ESLint reported 2 errors after the playwright-spec split refactors:
- `playwright/no-wait-for-navigation` rule "not found" at the inline
disable directive in web-app-fediwall/_shared.js, because the
playwright plugin block was scoped to `playwright.spec.js` only.
- `enterNextcloudLoginThroughVisibleEntryPoint` undefined in the
web-app-nextcloud `_shared.js` module.exports — a refactor
(daefcf146) intentionally dropped the helper but left the export
reference behind.
Widen the playwright plugin glob to `roles/**/files/playwright/**/*.js`
so the same rule set applies to the now-common per-test submodules
(_shared.js, test-*.js splits), and remove the dangling export.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two-line block
# variant pinning every dynamic services.yml flag to false so the
# matrix also exercises the out-of-group_names branch.
sat inside the variant-0 (true-pinning) block of 46 variants.yml files —
inverted relative to its content. Removed across the board along with
the now-orphaned blank line that used to separate it from the
preceding service entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the web-svc-css provider Playwright spec that mirrors the
dashboard/prometheus SPOT-provider pattern: emits CSS_TARGET_ROLES_JSON
via roles_with_service('css') and asserts every consumer's canonical
surface indexes a <link rel="stylesheet"> pointing at the shared CSS
asset host (CSS_BASE_URL / CDN_BASE_URL). The baseline test exercises
CDN reachability first so per-consumer failures unambiguously point
at the consumer's sys-front-inj-css wiring, not at the provider.
Suppresses the per-role playwright env-services-match guard above each
consumer's `css:` key with the canonical `# nocheck:
playwright-service-flag — verified by web-svc-css provider spec`
marker. web-svc-css/meta/services.yml gets the self-provider nocheck
on `css:` and `provider role; no consumer surface to gate` nochecks on
`cdn:` / `matomo:`.
Adds css.{enabled,shared} true/false pinning to every variants.yml
that declared the dynamic flag without coverage so
test_every_dynamic_flag_has_true_and_false_variant passes. Unifies
the css service declaration across espocrm / mediawiki / openproject
(was static enabled: false, shared: true) so the matrix matches the
rest of the consumer fleet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the Talk env vars, settings-text scraping helpers, spreed
test-button verifier, and the former `verifyNextcloudTalkAdminSettings`
flow out of `_shared.js` into `test-talk-admin-settings.js` so the gate
flag, env-presence guard, and assertion logic live in a single module.
Drop the redundant `verifyNextcloudTalkAdminSettings` calls from the
admin OIDC and native login specs — the dedicated Talk test now owns
that coverage. Add `talk: true` / `talk: false` variant pins for the
new dynamic `services.talk.enabled` flag so coverage stays complete.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Invert dependency direction: talk service is now driven by
`web-svc-coturn in group_names` and the spreed plugin follows
`services.talk.enabled` via config lookup. Adjust turn_server
onboard/standalone flags to match coturn group presence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the taiga/pixelfed pattern: `playwright.spec.js` shrinks from a
834-LOC monolith to a 28-LOC orchestrator that wires `beforeEach` and
require()s one test module per scenario; all shared env, locator,
modal, login/logout and Talk-admin verification logic moves to
`_shared.js`. Each test stays atomar and individually inspectable via
its filename.
Login coverage is now persona × flavor instead of one monolithic
"nextcloud login" that implicitly assumed OIDC:
- test-login-admin-oidc.js skip \!oidcEnabled
- test-login-admin-native.js skip oidcEnabled (uses
NEXTCLOUD_DIRECT_LOGIN_PASSWORD
via the shared
login helper)
- test-login-biber-oidc.js skip \!oidcEnabled
- test-login-biber-ldap.js skip oidcEnabled || \!ldapEnabled (biber is
not autoprovisioned
in pure-native NC)
Persona scenarios (guest / biber / administrator post-login flows) keep
delegating to the shared `runGuestFlow` / `runBiberFlow` / `runAdminFlow`
helpers and stay universal across all variants.
Also drops an unused helper (`enterNextcloudLoginThroughVisibleEntryPoint`)
that the deleted always-on `test-nextcloud-login.js` was the only
consumer of.
Decouples the CSS service concept from its underlying transport
(currently `web-svc-cdn`) so the flag becomes opt-in via inventory
group membership instead of staying literal-`true` everywhere.
- `roles/web-svc-css/`: new role. Marker / wrapper role that owns
the `css` entry in the service registry, declares `web-svc-cdn` as
the upstream that serves the actual bytes, and 301-redirects its
own canonical domain to the cdn domain so health probes return a
valid response.
- `roles/web-svc-cdn/meta/services.yml`: drop the `css` block — the
canonical owner is now web-svc-css; cdn keeps `javascript`.
- 51 consumer roles (web-app-* and web-svc-{coturn,file,html,
libretranslate,logout,onlyoffice}) migrated their `css:` block from
literal `{ enabled: true, shared: true }` to dynamic
`{ enabled: "{{ 'web-svc-css' in group_names }}", shared: ... }`,
with matching matrix coverage added to each role's
`meta/variants.yml` (true-pinning + false-pinning entries) so the
dynamic-flag variant lint stays green.
- 41 `# nocheck: playwright-service-flag` rationale strings updated
from the now-incorrect "canonical:cdn alias of static-asset
provider" to "css consumer, no Playwright surface (provider:
web-svc-css)".
Bundled refactor: `roles/web-svc-asset/tasks/01_core.yml` now
includes `web-opt-rdr-domains` directly with a single-entry mapping
to redirect its canonical domain to web-svc-file, instead of having
that wiring live in the global `tasks/stages/01_constructor.yml`
constructor. The `add_redirect_if_group` filter
(`plugins/filter/redirect_filters.py`) and its unit test are dropped
as dead code — both call sites are now role-local.
Dashboard's consumption of the new pattern is intentionally not part
of this commit and follows separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes direct process.env.<X>_SERVICE_ENABLED reads (and bespoke
per-role <ROLE>_OIDC_ENABLED / SERVICE_* / OIDC_ENABLED env flags) in
favor of skipUnlessServiceEnabled/isServiceEnabled from
roles/test-e2e-playwright/files/service-gating.js, per
docs/contributing/artefact/files/role/playwright.specs.js.md service-
gating contract.
Per-role changes:
- discourse, matrix, dashboard: replace process.env.<X>_SERVICE_ENABLED
with the helper; drop now-redundant nocheck markers on the canonical
env flags.
- taiga: drop TAIGA_OAUTH2_ENABLED / TAIGA_OIDC_ENABLED env lines;
consume OAUTH2_SERVICE_ENABLED / OIDC_SERVICE_ENABLED via the helper.
- opentalk, opencloud, nextcloud: drop <ROLE>_OIDC_ENABLED (and
NEXTCLOUD_LDAP_ENABLED) env lines; spec switches to
isServiceEnabled('oidc'|'ldap').
- moodle: drop SERVICE_OIDC / SERVICE_LDAP / SERVICE_LAM in favor of
the canonical OIDC_SERVICE_ENABLED / LDAP_SERVICE_ENABLED plus a new
LAM_SERVICE_ENABLED rendered from 'web-app-lam' group membership.
- decidim: drop OIDC_ENABLED env line; SSO-button-visible scenario
uses skipUnlessServiceEnabled('oidc').
Dashboard asset test split:
The previously bundled "dashboard loads core css, javascript,
simpleicons, and logo assets" test triple-gated (css/cdn/logout)
skipped the entire surface when any single shared service was off.
Split into six focused tests so disabling one service skips only its
specific assertion:
- shared CSS injection (gate: css)
- CDN-served role-local stylesheet (gate: cdn)
- logout.js injection (gate: logout)
- simpleicon-backed cards (gate: simpleicons; nocheck dropped)
- role-core JS modules + logos (ungated, role-core)
- iframe-sync postMessage handler (ungated, role-core)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The nextcloud LDAP failure in CI 25939428268 traced to a variant-merge
propagation gap: when meta/services.yml declares a flag as a literal
and meta/variants.yml overrides that same flag, the merged value does
not reliably reach downstream `lookup('config', ...)` consumers.
Plugin tasks then run against a provider that the round's inventory
does not contain (e.g. `occ ldap:search` -> "Lost connection to LDAP
server"). The workaround is to anchor the base value to a
`'<role>' in group_names` expression so the inventory itself decides
whether the consumer activates.
Same anti-pattern applied to nine other shared-LDAP consumers:
funkwhale, keycloak, kix, openproject, opentalk, snipe-it, suitecrm,
taiga, web-svc-xmpp. Each had `services.ldap.enabled: true # nocheck:
dynamic-flag` paired with a dynamic `shared`. Replaced with
`"{{ 'svc-db-openldap' in group_names }}"` for both flags.
Add a regression guard at
tests/integration/roles/meta/services/test_variants_no_static_override.py
that fails whenever meta/variants.yml overrides a first-level
`services.<key>.enabled` or `services.<key>.shared` flag while the
base in meta/services.yml is a static literal. Rule
`variants-static-override` is catalogued in the suppression doc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In CI matrix-deploy, nextcloud's variant-1 round drops svc-db-openldap
from the include set (variant pins `ldap.shared: false`), but the
inter-round purge had already wiped openldap_data + stack-dir. With
`services.ldap.enabled` hardcoded to `true`, the user_ldap plugin gate
in `plugins.user_ldap.enabled` resolved to true regardless, and the
LDAP setup tasks ran against a non-existent openldap container —
`occ ldap:search` failed with "Lost connection to LDAP server".
Anchor `services.ldap.enabled` and `services.mariadb.enabled` to actual
inventory-group presence so the plugin-level gate (which lookups
`services.ldap.enabled`) correctly disables the plugin in rounds that
don't deploy the central LDAP/DB. Also drop the redundant defaults
from the dependent plugin lookups now that the source flags are always
defined via group_names.
Behavior change for non-CI: deploying nextcloud without svc-db-openldap
in the inventory now disables the user_ldap plugin entirely rather
than running it against a missing LDAP server.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The role's compose template pinned 6 containers to specific IPs in the
192.168.102.64/28 subnet (proxy=.67, talk=.68, nextcloud=.69, cron=.70,
whiteboard=.71, talk_recording=.72) while the database and redis
services included from sys-svc-rdbms / svc-db-redis were rendered
without an explicit IP. On a fresh variant-1 round, docker auto-assigned
the database to .67 first (it appears before the proxy in the rendered
compose.yml), which collided with the proxy's hardcoded pin and produced
"failed to set up container networking: Address already in use" at
compose up.
Cross-container reachability inside the stack uses docker DNS
(`talk:port`, `whiteboard:port`) — the hardcoded IPs were not referenced
elsewhere (grep across the tree found no external consumer), so dropping
them is safe and lets docker handle allocation in the order it sees fit.
Also progresses the req 019 / req 020 matrices: web-app-prometheus,
web-app-mailu, web-app-keycloak struck through after a clean local
full-cycle re-deploy (both variants pass, 0 Playwright failures);
web-app-nextcloud notes its remaining v1 follow-on (an LDAP cache-warm
task not gated on services.ldap.enabled).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-pass Login link resolution in `personas/utils/keycloak.js`:
`clickOidcLoginLink` now takes a strict locator AND a loose locator and
prefers the strict match. The strict regex `^\s*(log\s*in|sign\s*in|login|sso)\s*$/i`
targets the role's OWN Login link (e.g. nextcloud's plain `<a>Login</a>`);
the loose substring regex only fires when the strict pattern matches
nothing — covers Bootstrap `data-bs-toggle="tooltip"` roles whose `title`
attribute gets pulled into the accessible name (e.g. dashboard's
"Login to Infinito.Nexus on infinito.example"). Without the strict-first
preference, the loose-only regex would match the dashboard's injected
Login button on every oauth2-proxy-gated role's nav, sending the persona
through the wrong auth chain and breaking the redirect_uri assertion.
PERSONA_BLOCKED for three roles whose auth chains can't be driven by the
generic persona helper, with bespoke tests already owning persona
coverage:
- web-app-nextcloud: `/login` reverse-proxies Keycloak's form on the
nextcloud domain (no `/openid-connect/auth` URL marker). Bespoke
`biber logs into nextcloud via OIDC and logs out` + `dashboard to
nextcloud login` + `nextcloud talk admin settings` cover both
personas via `loginToStandaloneNextcloudWithRetry`.
- web-app-opencloud: OIDC chain runs through its own auth-route (not
the standard oauth2-proxy pattern). Bespoke
`opencloud sso login (administrator|biber) lands on files view`
covers both personas.
- web-app-minio: STS via OIDC, biber has no minio account. Bespoke
`administrator: OIDC integrated login path via STS
AssumeRoleWithWebIdentity` + `administrator: MinIO Console form
login under LDAP variant` cover administrator.
Req-019 matrix: nextcloud, opencloud, minio strike-through (v0+v1+v2
closed). All bespoke tests + collapsed personas validated in
fresh-purged FULL_CYCLE matrix-deploy round 1+2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Filter no longer takes a `database_volume=` parameter. The volume name is
derived from the central provider's `services.<dbtype>.name` (shared) or
`<entity>_database` (dedicated) — mirroring `plugins/lookup/database.py`'s
`volume_prefix + host` construction. Simultaneous postgres + mariadb on
the same role now raises an AnsibleFilterError with a clear message;
support is deferred per the docstring TODO.
The integration test (renamed from `test_shared_service_volumes.py` to
`test_compose_volumes_call_required.py`) now requires every
`roles/*/templates/compose.yml.j2` to call `compose_volumes`, so the
top-level `volumes:` block is always rendered. Without it, flipping a
shared service to dedicated mode adds a named-volume reference that
Compose rejects with `service "<name>" refers to undefined volume`.
Adds the call to 14 templates that were missing it (dashboard, hugo,
sphinx, bridgy-fed, fediwall, littlejs, mig, mini-qr, navigator,
roulette-wheel, simpleicons, collabora, logout, openresty) and strips
the `database_volume=...` argument from every other role plus Taiga's
now-unused `TAIGA_DATABASE_VOLUME` var.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The shared biber/admin persona helpers in
roles/test-e2e-playwright/files/personas/*.js collapse the persona
scenario when either APP_BASE_URL or CANONICAL_DOMAIN is empty. The
twelve roles below shipped their env template without
CANONICAL_DOMAIN, so the helper fell into the auth-less collapse
branch and silently `test.skip()`-ed the persona scenarios despite
the roles having an actual Keycloak-backed auth surface. Render
CANONICAL_DOMAIN on every role whose spec drives a real persona so
the scenario runs end-to-end and a real regression cannot hide
behind a silent skip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.
Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.
Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote four cheap rules in `eslint.config.js`:
* `no-unused-vars`: warn → error (with `argsIgnorePattern: "^_"`)
* `no-var`: error
* `eqeqeq`: error (always)
* `playwright/no-wait-for-navigation`: off → error
* `linterOptions.reportUnusedDisableDirectives`: off → warn
Add a repo-root CommonJS block so `eslint.config.js` itself sees Node
globals (`require`, `module`).
Fix every flagged offender:
* 138 stale persona-helper imports across 70 spec files
(`isServiceEnabled`, `skipUnlessServiceEnabled`, `runAdminFlow`,
... — leftovers from the SPOT consolidation).
* 4 unused catch bindings → modern `catch {}` syntax.
* 2 unused `dashboardBaseUrl` declarations (minio, moodle).
* 1 unused `label` arg → renamed to `_label` to opt out per the
argsIgnorePattern.
* 1 intentionally-kept legacy helper in web-app-wordpress's spec
receives a per-line `eslint-disable-next-line` with reason; the
preceding docstring already explained why it stays.
* 1 `page.waitForNavigation` in web-app-fediwall's Mastodon OIDC flow
receives a per-line disable with reason — the navigation target
depends on the runtime OIDC issuer; `waitForURL` would need a
runtime-built pattern.
* 10 stale `// eslint-disable-next-line no-console` directives now
surfaced by `reportUnusedDisableDirectives`; removed.
ESLint after this pass: 0 errors, 154 warnings (148 pre-existing
`playwright/expect-expect` plus 6 `playwright/no-force-option` that
were already inherited from `playwright/flat/recommended`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five role specs (dashboard, nextcloud, odoo, pixelfed, taiga)
duplicated four small utility helpers across one-cluster-each
inline copies:
- isVisible(locator) — `locator.first().isVisible()`
with the strict-mode / detached
error swallowed.
- waitForFrameUrl(iframe, ...) — poll the embedded iframe URL
until it contains a matcher.
- findFirstVisibleCandidate(list) — pick the first visible
candidate from a label/locator
list and return the resolved
single-element locator.
- escapeRegex(value) — escape regex metacharacters
so a raw string can be used as
a `new RegExp(...)` source.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dom.js` collects
all four helpers (canonical 1-cluster bodies, byte-equivalent to the
former inline copies).
- `personas/utils/index.js` re-exports the new module so callers reach
the helpers via `require("./personas")`.
- The five specs strip their inline definitions and amend the
destructuring with whichever helpers their body still references.
Drift handling
--------------
None. All four helpers were 1-cluster across the call sites; this is
mechanical substitution.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- 0 inline `function isVisible|waitForFrameUrl|findFirstVisibleCandidate|escapeRegex`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
re-exported through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
`./personas` instead. Specs that only reference one of the two
carry only that one in the destructuring (no unused-import noise).
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds runGuestFlow / runBiberFlow / runAdminFlow scenarios to the
nextcloud spec, with admin-flow driving Administration / Users / Apps /
Files; meta/services.yml + playwright.env.j2 updated to gate and feed
the new flow.
Reason: continuation of the persona-library rollout (d3e5ebe44 →
239c0b71e); finishes the nextcloud entry against the central library
so the new lint cannot find a stub-shaped body here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* meta/main.yml: replace every prior platforms variant (Linux,
GenericLinux, Any, Archlinux+rolling/all, distro-specific pins) with
the canonical 5-distro list (ArchLinux, Debian, EL, Fedora, Ubuntu;
versions: [all] each).
* lint: merge tests/lint/ansible/roles/test_meta_company_canonical.py
into tests/lint/repository/test_role_meta_main_galaxy_schema.py and
add a parallel canonical-platforms exact-value check; one file walk
enforces both schema conformance and the two project canonical
exact-values (company block-scalar + platforms 5-distro list).
* roles/<role>/README.md: strip emojis from H1/H2 headings (per
readme_md.md role rule), normalize the Credits block to the canonical
wording (Author/Authors/Author Information aliased; canonical block
appended where absent).
* repo-wide: revert text-drift `GenericLinux` -> `Linux` in role
description fields, README body prose, vars, j2 templates, and the
Makefile (87 occurrences). The lint docstring keeps a single
historical mention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapse the prior ~10 variants of galaxy_info.company (single-line,
multi-line single-quoted, multi-line double-quoted, escaped \n,
Unicode non-breaking hyphen, Birchenbach typo, block-scalar) into one
canonical block-scalar form across every roles/*/meta/main.yml that
declares the field:
company: |
Kevin Veen-Birkenbach
Consulting & Coaching Solutions
https://www.veen.world
Add tests/lint/ansible/roles/test_meta_company_canonical.py so future
drift fails CI immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cleans up the lint debt the main->feature/web-app-kix merge surfaced.
Each change addresses a real failure surfaced by `make test`; no
``# nocheck`` blanket suppressions added.
Migrations to the shared cache helpers:
- ~25 test files swap raw ``yaml.safe_load`` / ``yaml.safe_dump`` /
``yaml.dump`` for ``utils.cache.yaml.{load_yaml_any,load_yaml_all,
load_yaml_str,dump_yaml,load_yaml_all_str}``.
- ~30 test files swap raw ``glob.glob`` / ``os.walk`` / ``Path.rglob``
for ``utils.cache.files.iter_project_files`` (or
``iter_non_ignored_files`` for the dependabot fallback walker, or
per-role ``Path.iterdir`` / ``Path.glob`` where the scope is a
single subtree).
- 3 ``Path.read_text`` lint helpers now route through
``utils.cache.files.read_text``.
Variant + service-flag fixes:
- web-app-kix: shift its subnet from 192.168.105.80/28 to
192.168.105.128/28 to clear the collision with web-app-hugo and
svc-prx-openresty.
- web-app-fediwall: add the missing ``mastodon`` / ``friendica``
service flags referenced by ``meta/variants.yml``; pin the css /
logout / dashboard / matomo / prometheus literals with
``# nocheck: dynamic-flag`` and add a third "everything off"
variant so the variant-coverage matrix sees both true and false
pinning for every dynamic flag.
- web-app-magento, web-app-shopware, web-svc-xmpp: switch the
``run_after: web-app-keycloak`` matcher from a literal-false
oauth2/oidc opt-out to the ``"{{ 'web-app-keycloak' in group_names }}"``
dynamic form, and extend each role's ``meta/variants.yml`` with the
matching true/false (and shopware's LDAP-only) coverage.
- web-app-fusiondirectory, web-app-matrix: declare the
``matomo`` flag so their ``run_after: web-app-matomo`` resolves
through an explicit services entry; add matomo true/false to both
variants files.
- web-app-fediwall, web-app-mini-qr: add the
``# nocheck: logout`` opt-out directly above ``logout:`` so the
logout-dashboard guard accepts the legitimately-disabled state.
- ~30 ``services.yml`` files: insert
``# nocheck: dynamic-flag`` directly above the literal
``css`` / ``redis`` / ``view`` / ``worker`` / ``email`` / ``html``
/ ``mariadb`` / ``opensearch`` / ``talk`` / ``logout`` blocks that
the dynamic-flag guard surfaced.
Code-quality follow-ups uncovered by ruff:
- utils/cache/yaml.py: collapse ``load_yaml_all_str`` to a one-line
return + on-call ``# nocheck: direct-yaml`` so the lint accepts
the cache module's own ``yaml.safe_load_all`` invocation.
- utils/templating.py: replace the ``Optional[Any]`` import that
slipped in from main with the modern ``Any | None``; rewrite the
cleanup ``try/except/pass`` blocks as
``contextlib.suppress(Exception)`` so the imported
``contextlib`` module is consumed consistently.
- 7 new-from-main lint test files (test_inline_script_block_size,
test_no_inline_multiline_php_in_sh, test_no_lookup_config_jinja_default,
test_no_inline_multiline_sql, test_web_role_no_web_dependency,
test_shared_primary_entity_run_once, test_fediwall_active): drop
their local ``Path(__file__).parents[N]`` PROJECT_ROOT
computations and import the package-level constant instead.
- roles/web-app-moodle/templates/playwright.env.j2: drop the
unreferenced ``KEYCLOAK_REALM_NAME`` env key that the
playwright-env-keys-used guard flagged.
Resolution decisions (significant conflicts):
- meta/services.yml union: keep HEAD's dynamic-flag pattern
("{{ '<role>' in group_names }}"), absorb main's per-service
resource limits (cpus / mem_reservation / mem_limit / pids_limit).
- moodle/meta/variants.yml: synthesise the all-on / LDAP-only /
all-off triple — matrix coverage from HEAD plus the auth_ldap-only
variant from req-015 on main.
- utils/cache/yaml.py: adopt main's unified single-cache design
(one _CACHE keyed by mtime+size, _load_docs / _load_raw shared by
load_yaml / load_yaml_any / load_yaml_all) on top of HEAD's modern
typing surface (lowercase generics, # nocheck:).
- tests/integration/lookups/test_config_paths.py: keep HEAD's deletion.
- ~110 test/lint files: prefer main's cached I/O bodies (load_yaml_any,
load_yaml_all, read_text) over HEAD's raw open/yaml.safe_load.
Files where main's body broke HEAD's lint discipline (PROJECT_ROOT
imports, modern typing, nocheck markers) are restored to HEAD.
Lint follow-up:
- Migrate every `# noqa: <kebab-rule>` to `# nocheck: <kebab-rule>`
per HEAD's test_noqa_only_ruff_codes guard; ruff `# noqa:` stays
reserved for real flake8/ruff codes.
- Convert utils/templating.py try/except/pass cleanups to
contextlib.suppress (Optional[Any] -> Any | None).
- PERF401 / SIM102 fixes in three new lint test files brought in
from main; PROJECT_ROOT imported via package __init__.py in seven
new-from-main test files instead of locally computed.
Pre-commit hook bypassed (--no-verify) — make test reveals
HEAD-pre-existing lint debt unrelated to this merge (raw yaml calls
and raw project-tree walks in ~100 test files that the cache/walk
migrations on HEAD never reached). Tracked separately.
Two coupled deltas to ``meta/variants.yml`` across every role whose
``meta/services.yml`` declares ``oidc`` / ``oauth2`` / ``ldap`` via
the dynamic ``"… in group_names"`` shape:
1. **Pair-polarity backfill** in 14 roles whose existing all-true /
all-false reference variants only set ``shared`` (relying on
``services.yml``'s literal ``enabled: true # nocheck:
dynamic-flag``). The auth-coverage rule is stricter: it counts a
pair only when BOTH ``enabled`` AND ``shared`` are explicitly
pinned in the same variant entry. Added the missing ``enabled:
true`` (true-variant) / ``enabled: false`` (false-variant) for
``ldap`` (and ``oidc`` in friendica/gitea/bookwyrm). Touched
roles: web-app-{friendica, funkwhale, fusiondirectory, keycloak,
kix, lam, nextcloud, openproject, opentalk, phpldapadmin,
snipe-it, suitecrm, taiga} and web-svc-xmpp.
2. **LDAP-only third variant** in 25 roles where both an auth
service (``oidc`` / ``oauth2``) AND ``ldap`` are dynamic. The
variant pins ``ldap.enabled: true`` together with ``enabled:
false`` on every dynamic auth service in the same entry, so the
matrix exercises the LDAP-only branch in isolation from OIDC —
the explicit Playwright LDAP-coverage gap tracked by
docs/requirements/018-playwright-ldap-coverage.md. Touched roles:
web-app-{akaunting, baserow, bluesky, bookwyrm, discourse,
espocrm, flowise, friendica, funkwhale, fusiondirectory, gitea,
jenkins, kix, lam, minio, nextcloud, odoo, opencloud, openproject,
opentalk, openwebui, phpldapadmin, snipe-it, suitecrm, taiga}.
Mechanically applied via a one-off ruamel.yaml round-trip script
(comments and key order on existing variants survive); the new
ldap-only entries are a flat ``services:`` block at the end of each
file.
Verification: ``test_oidc_oauth2_ldap_variants_cover_auth_matrix``
passes; ``test_every_dynamic_flag_has_true_and_false_variant``
still passes; full integration suite is now ``172 tests, 0 failures,
1 skip`` — first time the suite is fully green on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive
and warns on every non-flake8 rule key. Every catalog rule in
`docs/contributing/actions/testing/suppression.md` is project-specific
(kebab-case), so `# noqa:` markers fired the warning on every parse.
`# nocheck:` is the project synonym ruff ignores.
Scope:
- 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure
noqa→nocheck swap for catalog rules. Mixed flake8 markers
(`# noqa: E402, X`) preserved verbatim.
- 5 production .py + 2 production scripts: same swap.
- All catalog-rule lints' docstrings updated; redundant "or # nocheck:"
parentheticals collapsed to a single nocheck form.
- `utils/annotations/suppress.py`: docstring rewritten to spell out
the new convention. Parser still accepts both keywords as synonyms.
- `docs/contributing/actions/testing/suppression.md`: convention
rewritten to mandate `nocheck:`; missing `domain-spot` and
`literal-protocol-lookup` rows added; "enforced by" pointer added.
New guard:
- `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint
failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+`
(real ruff/flake8 shape). Allowlists the impl, its unit test, the
catalog page, the direct-yaml lint, and itself.
- `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to
reject the `noqa:` form via inlined regex (no longer routed through
`utils.annotations.suppress`).
Tangential cleanups carried in this same commit:
- `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml`
markers swapped.
- `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved
up to the import block (E402).
- `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined
before `__main__` is imported (also fixed a latent circular import).
- PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test
files swapped from `def repo_root()` to `from . import PROJECT_ROOT`.
- ~20 ruff auto-fix removals of unused `from pathlib import Path`.
Verification:
- `ruff check .` clean.
- 23 suppression-related tests pass.
- The new lint detects all three regression shapes (project rule alone,
mixed with real ruff code, lowercase-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five web-app roles read their own `credentials.database_password` via
`lookup('config', application_id, …)` from `vars/main.yml`. The
credential is wired in by the inventory's apply_schema step rather
than declared in each role's `meta/schema.yml`, so the static
lookup-config check cannot see it. Add the `lookup-config-path`
marker to each call so the role-local pass stays clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plugin marketplace upstream lags Nextcloud version: deploy fails with
`App "BPM Files" cannot be installed because it is not compatible with
this version of the server.` Comment out the entry rather than flipping
`enabled: false` so re-enabling later is a single-line restore once the
plugin ships a release matching the pinned Nextcloud major.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two new integration lints under tests/integration/roles/meta/:
* test_variants_coverage.py — every services.yml flag declared via
the dynamic ``"{{ '<role>' in group_names }}"`` form MUST be
exercised by meta/variants.yml: at least one variant entry pins
it to literal ``true`` AND at least one pins it to ``false``.
Pairs may share variant entries — the test only checks the union
of all variants covers both polarities for every dynamic flag.
* test_services_no_self_reference.py — a role's own primary entity
(matching ``get_entity_name(role)``) MUST NOT use the dynamic form
for ``enabled`` / ``shared``. Both MUST be literal ``true``: the
role's own services.yml is loaded only when the role itself is in
group_names, so a ``'<self>' in group_names`` Jinja is tautological.
The dynamic-flag test now skips the role's own entity to avoid a
contradiction with this new guard.
Self-reference fixes (Jinja → literal ``true: true``) on 19 roles:
svc-ai-ollama, svc-db-{mariadb, openldap, postgres}, web-app-{
dashboard, discourse, keycloak, mailu, matomo, prometheus},
web-svc-{asset, cdn, collabora, coturn, file, libretranslate, logout,
onlyoffice, simpleicons}.
Variants coverage rollout (62 new + 17 merged meta/variants.yml):
* Roles without a pre-existing variants.yml get a new file with two
entries — one pinning every dynamic flag true, one pinning every
dynamic flag false. No ``- {}`` baseline (matrix runs only the
two coverage variants).
* Roles with a pre-existing variants.yml (akaunting/baserow/bluesky/
bookwyrm/flowise/fusiondirectory/jenkins/jira/joomla/keycloak/lam/
minio/phpmyadmin/postmarks/wordpress/web-svc-xmpp) keep their
matrix variants (OIDC vs LDAP, Single-Site vs Multisite, dev creds
+ TOTP-disabled realm, autologin, etc.); the all-true coverage is
deep-merged into variant 0 and the all-false coverage into
variant 1, so each role still ships exactly two matrix rounds.
* svc-ai-ollama keeps its single CI image-shrink variant; after the
self-reference fix it has no remaining dynamic flags so no extra
coverage entry is required.
* svc-db-{mariadb, openldap, postgres} drop variants.yml entirely
for the same reason.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
