`tests/integration/roles/test_playwright_spec_env_gates.py` failed
across 35 roles: every spec that imports `runBiberFlow` /
`runAdminFlow` ships `DASHBOARD_SERVICE_ENABLED=` in its env template,
but the persona helpers consume the dashboard purely via
`process.env.DASHBOARD_BASE_URL` without ever passing the literal
`"dashboard"` to `safeIsEnabled` / `requireService` / friends. The
integration test grep'd for the string-literal call and (correctly)
saw zero matches, so flagged the flag as declared-but-never-gated.
Adds `safeIsEnabled("dashboard");` as a bare-statement gate to
admin.js and biber.js, parallel to the existing `safeIsEnabled
("oauth2"); safeIsEnabled("logout");` lines that already pin those
two flags to the persona surface. The integration test walks every
file under `roles/test-e2e-playwright/files/personas/` once any
persona runner is referenced in the spec, so a single mention covers
all 35 roles in one shot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires JavaScript linting into the project for the first time, scoped to
the Playwright specs and persona helpers under `roles/**/files/**/*.js`.
Setup
-----
- `package.json` + `package-lock.json` pin eslint v9, eslint-plugin-
playwright v2, @eslint/js, globals.
- `eslint.config.js` (flat config) loads `eslint:recommended` and
`playwright/flat/recommended`. Browser globals are injected so
`page.evaluate(() => window.foo)` callbacks lint cleanly. Project-
idiomatic patterns are explicitly turned off with rationale: the
service-gating contract (req 006/019) uses test.skip on disabled
services; persona helpers branch on isVisible/isServiceEnabled before
expect; some specs deliberately use waitForTimeout / networkidle for
OIDC settle. `no-empty` allows the empty-catch idiom; the autofixer
for `prefer-web-first-assertions` and `no-wait-for-navigation` is
unsafe in this codebase (broken diff in dashboard.js), so both are
off pending a hand-migration pass.
- `scripts/lint/javascript.sh` mirrors the markdown.sh shape (skip on
missing, exit 127 with "make install-lint" hint).
- `scripts/lint/autoformat.sh` gets a `run_eslint` worker (treats rc≤1
as OK, like markdownlint).
- `scripts/install/lint.sh` gets `ensure_eslint` (npm-local install in
the repo root, matching the markdownlint pattern), a `javascript`
group, and `install_javascript_tools` is wired into the `all` group.
- `Makefile` exposes `lint-javascript` and folds it into the aggregate
`lint` target.
- `.github/workflows/lint-javascript.yml` follows the lint-makefile.yml
shape (Node 20, npm cache, `npm ci`, `make lint-javascript`); the
ci-orchestrator wires it into the lint pipeline + lint-gate `needs:`.
- `.gitignore` adds `node_modules/`.
Real lint findings fixed in this commit
---------------------------------------
- `roles/web-app-decidim/files/playwright.spec.js` was using
`runBiberFlow` / `runAdminFlow` without ever requiring `./personas`
(would have failed at runtime). Adds the require.
- Same file: two nutzlose `\\'csrf-token\\'` escapes in a double-quoted
query selector — eslint flagged, fixed.
- `roles/web-app-moodle/files/playwright.spec.js`:
`expect(await locator.count()).toBe(0)` →
`await expect(locator).toHaveCount(0)` (Playwright auto-retrying
assertion; safe `prefer-to-have-count` autofix).
Lint baseline: 0 errors, 285 warnings (mostly `no-unused-vars` and
`playwright/expect-expect` — addressable as follow-up cleanup, not
blocking).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tightens "🧹 Lint Galaxy (galaxy-importer)" → "🧹 Lint Galaxy" and
"🧹 Lint Makefile (mbake)" → "🧹 Lint Makefile" so the job titles
are uniform with the other lint-* jobs and don't lock the displayed
name to the underlying tool implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bookwyrm's logout control sits inside a profile-avatar dropdown
whose trigger is icon-only (no accessible name the generic in-app
logout helper can match). Both personas walk all the way through
the OIDC chain and reach the canonical app surface — confirmed via
URL-based auth-surface fallback — but the generic helper cannot
locate the logout button to complete the journey.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments. Document the path back (upstream
accessible-label on the avatar trigger or a per-role logout helper)
in `TODO.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bluesky's social-app shell is a mobile-first React SPA whose logout
control sits inside an icon-only profile menu. The generic in-app
logout helper cannot reach it (no accessible label, no role-based
selector that matches reliably across breakpoints), so the persona
flow walks all the way through the OIDC chain and then fails at the
in-app logout step.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments. The role's bespoke OIDC + LDAP variant
tests (`oauth2-proxy redirects unauthenticated visitors through
Keycloak`, `same oauth2-proxy gate when Keycloak federates user
storage from LDAP`) still verify that both personas authenticate via
the broker and reach the canonical app surface. Document the path
back (per-role logout helper or upstream accessible-label
improvement) in `TODO.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Akaunting's local user table is the source of truth; the role does
not auto-provision OIDC subjects, so a fresh biber identity arriving
from Keycloak hits a blank ledger and never reaches an authenticated
admin surface. The administrator persona has the same problem in
reverse: the bespoke local-admin login form does not accept the OIDC
test fixture's password.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments, and document the path back (wire OIDC
auto-provisioning) in `TODO.md`. Update the req-019 drift matrix
annotation accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coupled changes to `runBiberFlow` / `runAdminFlow`:
1. Move `PERSONA_BIBER_BLOCKED` / `PERSONA_ADMINISTRATOR_BLOCKED`
to position 0 of the flow. A role that opts out now short-circuits
the entire dashboard → app → logout journey instead of walking
through the auth chain and tripping the in-app logout step where
a late opt-out check could not catch it.
2. Add a URL-based fallback to the auth-surface detection: when the
selector-based marker (`getByRole("button", { name: /^(log\s*out…)/i
})`) finds no logout / user-menu control on any frame, accept any
nested frame whose URL has settled on the role's canonical domain
(and is NOT on Keycloak / oauth2-proxy denial / about:blank) as
evidence that the persona reached the authenticated surface.
Iframe-wrapped role surfaces (dashboard tile + role app inside)
and SPAs whose logout sits behind icon-only avatar dropdowns then
count as authenticated.
3. Drop `safeSkipUnlessEnabled("dashboard")` from the top of both
flows. The flag is rendered as `<NAME>_SERVICE_ENABLED=` from the
*provider* perspective in `meta/services.yml` (matomo's services
declare `dashboard.enabled: <web-app-dashboard in group_names>`,
meaning "is the dashboard role part of *this* role's stack").
When matomo deploys solo, the flag rendered as `false` even
though the dashboard URL was reachable, and every persona test in
the matrix run silently `-`-skipped. The auth-less collapse check
immediately below (`if (\!dashboardBaseUrl || \!canonicalDomain)
skip`) is the correct gate — a role that legitimately cannot
start the journey has no `DASHBOARD_BASE_URL` rendered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make `tests/lint/.../test_has_spec.py` raise instead of warn when a
roles/web-app-* role has no `files/playwright.spec.js`, then ship the
27 minimal specs needed to satisfy the new floor.
Each new spec follows `roles/web-app-mig` (TLS + canonical-domain
baseline, HTML content-type, three persona flows) with role-specific
admin-interaction labels seeded from app domain knowledge. The bodies
are intentionally static — `templates/playwright.env.j2` is not
shipped per role yet, so the specs do not run end-to-end on deploy;
detail tightening per role is follow-up work.
Also fix `_body_contains_stub_marker` in `test_no_stub_tests.py`: it
substring-matched `TODO` against `body.upper()`, which flagged
`masTODOn` inside the mastodon spec's body regex. Switched to
word-boundary matching so genuine `// TODO:` markers still trip the
lint while embedded substrings in app names or regex alternations do
not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dashboard service flag in matomo and bigbluebutton was rendered with an
unterminated single quote (`{{ 'web-app-dashboard in group_names }}`),
producing a Jinja-templated false. Closing the quote turns it into a
real `in group_names` lookup. Both variants.yml files now pin
`dashboard.enabled` and `dashboard.shared` to true and false so the
variant-coverage integration test passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdownlint MD056 (table-column-count) doesn't honor pipe-in-code-span
escaping and was counting the `|` inside `` `piwik|matomo` `` as a
fourth cell separator on row 12, breaking the 3-column table. Split
the regex-style alternation across two code spans (`piwik` or `matomo`)
to keep the semantics readable without a backslash-escape that
backticks would render literally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure ruff reflow on tests/lint/ansible/roles/web-app/playwright/
test_persona_strict_mode.py — removes one E303-flagged blank line
and the RET505-flagged `else:` after an unconditional `return`. No
behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Persona helpers learn three new behaviours:
* `clickRoleTileFromDashboard` and `clickSiblingTileFromDashboard`
open every visible Bootstrap dropdown / accordion trigger
(`[data-bs-toggle='dropdown']`, `[aria-expanded='false']`, ...) on
the dashboard before searching for the role tile, so roles that
live inside a collapsed category (e.g. keycloak under "Identity")
are reachable without coupling personas to the dashboard menu shape.
* `assertPrometheusForbiddenForBiber` and
`assertMatomoForbiddenForBiber` switch from fragile iframe DOM
inspection to a direct authoritative probe via
`page.request.get(<role-url>, { maxRedirects: 0 })`. They peek at
the FIRST hop only: 401 / 403 or a redirect into the auth chain
(`/oauth2/start|sign_in|callback`, `openid-connect/auth`) means
denial; any other status is examined further. A first-hop 200 is
acceptable ONLY when the body carries role-specific markers
(`prometheus_build_info` / `<title>Prometheus</title>` for
prometheus; matomo's login-form markers / `piwik|matomo` for
matomo). A 200 with a non-matching body is treated as a
misconfigured proxy or a denial-as-200 surface and fails loudly,
matching req 019 Rule 12.
* `web-app-keycloak`'s spec drops the generic persona-flow runners
(`runGuestFlow` / `runBiberFlow` / `runAdminFlow`) under the
auth-provider exemption: the bespoke "master-realm super
administrator", "normal-realm administrator", and "normal-realm
biber logs in through account interface" tests already cover the
three persona contracts via the realm account UI. The matching env
flags are marked `# nocheck: playwright-service-gate` /
`# nocheck: persona-required-envs` with auth-provider rationales.
Drift matrix in `019-playwright-meta-services-parity.md` strikes
through `web-app-matomo`, `web-app-prometheus`, `web-app-keycloak`,
`web-app-akaunting`, `web-app-baserow`, `web-svc-cdn`, and
`web-svc-simpleicons` after their FULL_CYCLE deploys produced green
Playwright runs (with the well-known kcadm.sh + sys-svc-mail
infrastructure failures documented in `web-app-mailu/TODO.md` per
the autonomy escape clause).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 971fda535: the new "Generate role include files" step
failed with `ModuleNotFoundError: No module named 'utils'` because
cli/build/include/__main__.py imports `utils.cache.yaml` and friends.
Locally scripts/meta/env/python.sh sets `PYTHONPATH:=.` via the
Makefile env layer, but the CI step invokes `bash scripts/setup.sh`
directly and bypasses that layer. Inline `PYTHONPATH=.` on the run
line; verified locally with
`PYTHONPATH=. python3 -c "from utils.cache.yaml import load_yaml"`
and `PYTHONPATH=. python3 -m cli.meta.categories.invokable -s "-"`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The persona-flow runners now fail loudly when biber / administrator
cannot reach an authenticated surface on the role under test. The
previous runtime auto-detection silently `test.skip(...)`'d, hiding
real regressions (broken OIDC mapping, removed logout button,
misconfigured oauth2-proxy, drifted UI selectors) behind a green
deploy.
Roles where the persona legitimately has no access (admin-only roles,
auth-provider roles) MUST declare an explicit env opt-out
`PERSONA_BIBER_BLOCKED=true` / `PERSONA_ADMINISTRATOR_BLOCKED=true`
in `templates/playwright.env.j2` with a documented rationale; without
the flag the helper fails the test with a diagnostic naming the
last-seen URL.
A new lint guard (`test_persona_strict_mode.py`) hard-fails any
future `test.skip(...)` in the persona helpers that is not guarded
by an allowed env opt-out / auth-less-collapse / service-gate
condition, and any `status === 200` branch in the prometheus / matomo
deny-check helpers that does not validate role-specific body markers.
Req 019 Rule 11 (strict-skip contract) and Rule 12 (200-body validation)
plus a matching "Strict failure on un-executable persona cases"
section in `playwright.specs.js.md` document the contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add mcr.microsoft.com to .claude/settings.json's WebFetch allowlist so
the agent can fetch Microsoft Container Registry pages without an
approval prompt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes for the lint-ansible CI job (failing run 25604499233):
1. Add `community.docker` to requirements/requirements.galaxy.yml so
`ansible-galaxy collection install -r ...` pulls it in CI. The
collection is referenced in 9+ files (web-app-discourse handlers,
svc-db-mariadb/openldap, sys-svc-compose/container, web-app-keycloak/
decidim) but was missing from the requirements list, so syntax-check
could not resolve `community.docker.docker_container`.
2. Add a "Generate role include files" step to .github/workflows/
lint-ansible.yml. `tasks/groups/*-roles.yml` are gitignored generated
artefacts (per tasks/groups/.gitignore: `*-roles.yml`) referenced
from tasks/stages/*.yml; without them ansible-lint reports a fatal
load-failure on tasks/groups/desk-roles.yml. The new step runs
`PYTHON=python3 bash scripts/setup.sh` (same path the local `make
setup` target invokes); pyyaml added to the pip install line as
that's the only non-stdlib dep the setup script needs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prepends `# Todos` to roles/web-app-dashboard/TODO.md (matching the
heading shape used by web-app-keycloak/TODO.md and
web-app-mailu/TODO.md) so MD041/first-line-heading-h1 stops failing
`make lint-markdown`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shellcheck flagged three `[[ cond ]] && B || C` chains in
scripts/lint/autoformat.sh (ruff, shellcheck, mbake worker tails) as
SC2015: if `B` exits non-zero, `C` runs even when the test was true.
Rewrites each as `if [[ cond ]]; then B; else C; fi` so the success
and failure branches stay disjoint.
Reason: `make lint-shellcheck` aborted on these three info-level
findings; production-profile lint runs treat them as failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure formatter reflow on tests/lint/ansible/roles/web-app/playwright/
test_persona_required_envs.py from `make autoformat`; no behaviour
change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The reference at scripts/lint/makefile.sh:4 pointed at
github.com/mavyhq/bake which 404s; pypi metadata (mbake project) lists
the canonical homepage as github.com/EbodShojaei/bake.
Reason: tests/external/repository/test_urls_reachable.py was failing
with HTTP 404 on the stale URL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds runGuestFlow / runBiberFlow / runAdminFlow scenarios to the
nextcloud spec, with admin-flow driving Administration / Users / Apps /
Files; meta/services.yml + playwright.env.j2 updated to gate and feed
the new flow.
Reason: continuation of the persona-library rollout (d3e5ebe44 →
239c0b71e); finishes the nextcloud entry against the central library
so the new lint cannot find a stub-shaped body here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow into bigbluebutton and
fediwall, plus matching meta/services.yml gate entries and
playwright.env.j2 env wiring. fediwall's adminInteraction drives a real
management surface so the admin-flow body is not a stub; bigbluebutton
imports isServiceEnabled for follow-on work.
Reason: continuation of d3e5ebe44 / 44d51744a / 735ae5e29 — finishing
the long-tail role list against the central persona library so the
new lint cannot find a stub-shaped body anywhere.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into the remaining 5 roles
(flowise, minio, opentalk, openwebui, wordpress). Each role gets the
spec.js persona scenarios plus matching meta/services.yml gate entries
and playwright.env.j2 env wiring; admin-flow callers pass a
role-specific adminInteraction that drives a real management surface.
Reason: continuation of d3e5ebe44 / 44d51744a — closes the long-tail
role list against the central library so per-role drift cannot
reintroduce the stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into another ~28 roles
(akaunting, baserow, bluesky, bookwyrm, bridgy-fed, decidim, discourse,
fider, friendica, fusiondirectory, gitea, hugo, jenkins, joomla, kix,
matrix, mattermost, mig, moodle, odoo, opencloud, peertube, pixelfed,
postmarks, sphinx, taiga, yourls, web-svc-libretranslate, web-svc-xmpp).
Each role gets the spec.js persona scenarios plus matching
meta/services.yml gate entries and playwright.env.j2 env wiring;
admin-flow callers pass a role-specific adminInteraction that drives
an actual management surface so the body is not a stub.
Reason: continuation of d3e5ebe44 — req 019 Rule 3 demands every
role exercise the canonical guest/biber/administrator flows; this
batch closes the long-tail role list against the central library so
per-role drift cannot reintroduce the stub-shaped bodies the new lint
forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The script lives at cli/administration/inventory/validate/__main__.py
(5 levels below repo root), so the sys.path bootstrap needs parents[4],
not parents[3]; the old value pointed one directory too deep and
broke project-package imports.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that rejects empty/tautological/TODO-marked persona and
contract test bodies in roles/<role>/files/playwright.spec.js, plus
the spec-contract doc updates (playwright.specs.js.md, req-017/018/019)
and the integration-test parity-guard tweak that articulate it.
Reason: previous batch updates produced "contract: <svc> gate is wired"
stubs whose bodies only proved what the gate itself already proved; the
lint catches both the title shape and the body shape so a rename is not
enough.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled doc changes that turn req 019 from "the matrix" into a
self-contained, agent-runnable rollout plan.
playwright.specs.js.md (the per-role spec contract):
- Persona scenarios are MUST-include with `<persona>: <flow>` naming;
additional scenarios MAY be added on top.
- New invariants for browser-context isolation, post-login assertion
hierarchy, admin authorisation hierarchy, and `ignoreHTTPSErrors`.
- New trigger entries for CSP baseline, cross-role round-trip
hosting, and variant-specific scenarios.
- New sub-sections "Test-budget overrides" (when `test.setTimeout`)
and "Cleanup and teardown" (try/finally, idempotent, bounded).
- Env-contract spelled out the minimum-required env-var set and the
promote-to-shared-helper rule for `decodeDotenvQuotedValue` /
`normalizeBaseUrl`.
- OIDC-over-LDAP preference: when both are enabled, the persona
flow uses OIDC; the LDAP-bind path lives in a separate scenario
exercised by the LDAP-only matrix variant.
req 019:
- Vision rephrased: the role's single spec file is the unit of
coverage; (role, included-service) pair MUST be exercised by ≥1
gated step inside it.
- Persona scenarios are mandatory tree-wide (Rule 3); web-svc-* and
the listed auth-less web-app-* roles MAY collapse to a single
baseline.
- New "Closure procedure" section with: Required reading, Autonomy
(no questions, no SERVICES_DISABLED, ignore unrelated failures),
Closure vocabulary (flag-closure vs role-closure),
Per-role flow (now incl. meta/variants.yml adjustment as step 5),
Pattern transfer (post-deploy edits to all later roles, single
deploy per role), Preserving existing tests (purely additive,
rename-only-when-flow-matches), Iteration order (dependency-aware
within tier; explicit auth-less list), Resumability (drift matrix
IS the progress marker), Commits (single commit at end).
- Per-service catalogue dropped as a duplicate; pointed at the
page that owns it.
- Stale test-path references updated to reflect the
`tests/lint/ansible/roles/web-app/playwright/` move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.
In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.
The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.
Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move five role-scoped lint tests out of the catch-all
tests/lint/repository/ tree into the ansible/roles/ tree where
they conceptually belong, dropping the redundant `role_` /
`meta_` filename prefixes:
test_role_file_spot.py → roles/test_file_spot.py
test_role_readme.py → roles/test_readme.py
test_role_meta_info.py → roles/meta/test_info.py
test_role_meta_layout.py → roles/meta/test_layout.py
test_role_meta_main_galaxy_schema → roles/meta/test_main_galaxy_schema.py
Plus regroup two web-app-only Playwright tests under
roles/web-app/playwright/ so the path itself states the scope:
test_playwright_env_keys_used → roles/web-app/playwright/test_env_keys_used.py
test_web_app_roles_have_playwright_spec
→ roles/web-app/playwright/test_has_spec.py
New __init__.py files set PROJECT_ROOT for the new package depths.
External references (suppression catalog entry, design doc,
web-app-joomla README, two integration-test docstrings) updated
to the new paths. Stale path strings in legacy requirement docs
left untouched per the snapshot policy from req 019.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Requirement docs are initial-creation-only artefacts. A live
reference from production docs / code freezes a snapshot that rots
once the per-feature SPOT (a contributor doc, a code module, a
lint test) takes over the live behaviour.
The new lint scans .md / .rst / .py / .yml / .yaml / .j2 / .json /
.sh / .js / .ts files for `requirements/<NNN>-<slug>.md` references
and fails unless one of three exemptions holds:
* the file is named `TODO.md` (anywhere in the tree);
* the line carries the literal token `TODO` (case-sensitive,
word-bounded);
* the file lives under `docs/requirements/` (cross-refs between
requirement docs are legitimate).
Currently surfaces 67 offender lines across 43 files for the team
to clean up by either moving the reference into a TODO.md, marking
the line, or repointing it at the live SPOT.
Also includes a formatter-applied cosmetic cleanup of
test_playwright_spec_env_gates.py (no semantic change).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new integration tests that close the round-trip between a role's
shared-service deps and its Playwright artefacts:
- test_playwright_env_services_match.py: every meta/services.yml entry
with `enabled:` MUST surface as <NAME>_SERVICE_ENABLED= in
templates/playwright.env.j2 (or be marked
`# nocheck: playwright-service-flag`).
- test_playwright_spec_env_gates.py: every <NAME>_SERVICE_ENABLED= line
in the env template MUST be consumed by at least one
requireService / skipUnlessServiceEnabled / isServiceEnabled /
isServiceDisabledReason call in the spec (or marked
`# nocheck: playwright-service-gate`).
Suppression rules added to the catalog. Requirement 019 captures the
policy plus a per-role status matrix of the current Test A drift
across the entire web-app-* / web-svc-* tree (44 roles, ~410 missing
flags) so the rollout has a single tracking surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caps the BFS over each role's shared-service deps:
- `--level 1` returns direct deps only.
- `--level 2` adds their direct deps.
- Omitting the flag keeps the unbounded full-closure behaviour.
Argparse rejects `--level < 1` with rc=2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For each application role, recursively resolve its shared-service
dependency closure and report (name, points, services-csv).
- BFS over each role's meta/services.yml; provider role per service-key
comes from utils.roles.applications.services.registry.
- Default truth predicate is is_explicit_truth so the
'<role>' in group_names Jinja form counts as a dep; --no-group-names
restricts to literal `true` flags only.
- Sort: --sort points (default, asc, name as tiebreaker) | --sort name (asc).
- Tests cover chain resolution, group-names toggle, self-not-counted,
non-application roles skipped.
Plus: roles/desk-gnome/meta/services.yml -- yaml autoformat drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Package renames (dynamic discovery picks up new names automatically):
- cli/build/role_include/ -> cli/build/include/
- cli/build/roles_list/ -> cli/build/list/
- cli/meta/roles/lifecycle_filter/ -> cli/meta/roles/lifecycle/
Domain logic moved out of cli/ (no CLI surface, just a helper class):
- cli/meta/roles/applications/resolution/services/{resolver,errors}.py
-> utils/roles/applications/services/{resolver,errors}.py
- drop dead services/yaml_utils.py + its only test
Bug fixes:
- cli/meta/{callorder -> roles/callorder} dropped its hard-coded
parents[3] PROJECT_ROOT (broke after the directory move and resolved
to <repo>/cli/); it now reuses utils.PROJECT_ROOT.
Help renderer (cli/core/help.py + colors.py):
- DIM color on the description column to separate it from the command name
- Blank line between the long command name and its wrapped description
- Italic invocation under the category heading + leading blank line
- New Style.ITALIC
Doc updates: path references in docs/requirements/00{8,10}*.md plus
soft-wrap fixes in the touched regions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picked up by the make-test setup phase regenerating generated artefacts
during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picked up by the make-test setup phase regenerating generated artefacts
during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* .pre-commit-config.yaml: add 6 scoped lint hooks (lint-action,
lint-ansible, lint-python, lint-shellcheck, lint-markdown, lint-galaxy)
with `files:` patterns so a markdown-only commit no longer triggers
ansible-lint or galaxy-importer. Aggregate make-test, make-test-lint
(md-only branch) and pre-push make-test-external are kept.
* Makefile: drop `lint` from the `make test` dependency chain. Per-file-
type lint stages now live exclusively in pre-commit; running them again
from `make test` would just duplicate work.
* scripts/lint/autoformat.sh: add markdownlint-cli2 --fix and
ansible-lint --fix passes alongside ruff/shfmt/shellcheck.
* roles/svc-db-openldap/meta/main.yml: collapse the wrapped description
into a single line (still under Galaxy's 255-char limit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lint-ansible workflow runs `ansible-playbook playbook.yml --syntax-check`
inside scripts/lint/ansible.sh, which needs every collection the playbook
references to resolve module names. Without them CI would fail with
"couldn't resolve module/action ...". Add an explicit
`ansible-galaxy collection install -r requirements/requirements.galaxy.yml`
step right after the pip install of ansible-core/ansible-lint so the
syntax-check can fully load community.general / hetzner.hcloud /
kewlfft.aur on a fresh runner.
lint-galaxy needs no parallel change — galaxy-importer in --legacy-role
mode treats unresolved modules as warnings, not fatal errors, so the
workflow can stay collection-less.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add the four mandatory H2 sections (Description / Overview / Features /
Credits) to every role README that was missing one. Strategy:
- prose between H1 and the first H2 is wrapped with `## Description`
- synonym headings rename to canonical (Goal/Goals/Use Case →
Overview, Contents/Tasks → Features)
- everything still missing is inserted at its canonical position with
content sourced from `galaxy_info.description`
* Resolve the 18 remaining documentation.md compat findings on the
modified set:
- em-dash separators replaced by parentheses (parenthetical
definitions), commas (clause separators), or colons (label/value)
- `Single Point of Truth (SPOT)` self-label headings renamed to
`User data resolution`
- one full-URL link text shortened to the project name
* tests/lint/repository/test_role_readme.py is now green across all
227 role READMEs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* tests/lint/repository/test_role_readme.py: hard-fails any role README
that violates docs/contributing/artefact/files/role/readme_md.md —
no emojis in headings, required H2s present
(Description/Overview/Features/Credits), required H2 ordering with
Credits last, and the Credits paragraph matching the canonical wording
byte-for-byte. Soft rules (sentence-case, H1 = software name, link
rules, bold-label feature items) are left out as too fuzzy.
* Makefile: revert three remaining `GenericLinux` -> `Linux` text-drift
occurrences in comments that the prior pass missed (no-extension file
was not in the script's extension allowlist).
* test_role_meta_main_galaxy_schema.py: autoformat collapsed the
canonical company string literal onto a single line.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* meta/main.yml: replace every prior platforms variant (Linux,
GenericLinux, Any, Archlinux+rolling/all, distro-specific pins) with
the canonical 5-distro list (ArchLinux, Debian, EL, Fedora, Ubuntu;
versions: [all] each).
* lint: merge tests/lint/ansible/roles/test_meta_company_canonical.py
into tests/lint/repository/test_role_meta_main_galaxy_schema.py and
add a parallel canonical-platforms exact-value check; one file walk
enforces both schema conformance and the two project canonical
exact-values (company block-scalar + platforms 5-distro list).
* roles/<role>/README.md: strip emojis from H1/H2 headings (per
readme_md.md role rule), normalize the Credits block to the canonical
wording (Author/Authors/Author Information aliased; canonical block
appended where absent).
* repo-wide: revert text-drift `GenericLinux` -> `Linux` in role
description fields, README body prose, vars, j2 templates, and the
Makefile (87 occurrences). The lint docstring keeps a single
historical mention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapse the prior ~10 variants of galaxy_info.company (single-line,
multi-line single-quoted, multi-line double-quoted, escaped \n,
Unicode non-breaking hyphen, Birchenbach typo, block-scalar) into one
canonical block-scalar form across every roles/*/meta/main.yml that
declares the field:
company: |
Kevin Veen-Birkenbach
Consulting & Coaching Solutions
https://www.veen.world
Add tests/lint/ansible/roles/test_meta_company_canonical.py so future
drift fails CI immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
'Docker' is not a valid Ansible Galaxy platform; switch all role meta
files to 'Linux' so galaxy metadata validates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
