Move the three persona-contract lints from
`tests/lint/ansible/roles/web-app/playwright/test_persona_*.py` into
their own `persona/` subdirectory and drop the now-redundant
`_persona` filename prefix:
* `test_persona_naming.py` -> `persona/test_naming.py`
* `test_persona_required_envs.py` -> `persona/test_required_envs.py`
* `test_persona_strict_mode.py` -> `persona/test_strict_mode.py`
The new `persona/__init__.py` re-anchors `PROJECT_ROOT` one level
deeper (`parents[7]`). Doc references in requirements 017, 018, and
019 now point at the new paths.
Renames are tracked via `git mv`, so commit history is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fold the recently-landed Playwright lint guards into the three
playwright-related requirement docs so the rules table and per-rule
"Enforced by" column point at the actual mechanical enforcement
rather than at "review (this requirement)".
* 019 Rule 1 now distinguishes the dashboard treatment for
`web-app-*` consumers (per-consumer flag exemption via
`*_TARGET_ROLES_JSON`, owned by the provider's own spec) from the
hard prohibition for non-`web-app-*` roles
(`test_dashboard_integration_scope.py` forbids any truthy
`dashboard.{enabled,shared}` declaration outside `web-app-*`).
* 019 Rule 3 / Rule 11 / Rule 12 now reference
`test_persona_naming.py`, `test_persona_required_envs.py`, and
`test_persona_strict_mode.py` as the enforcing lints.
* 019 "Closure paths" gains a dashboard-scope-exception paragraph:
paths 1 (render flag) and 3 (`# nocheck` annotation) collapse for
non-`web-app-*` roles; only path 2 (drop the entry) or a static
`dashboard: { enabled: false, shared: false }` declaration remain.
* 019 matrix: `dashboard` removed from the "fehlende Flags" column
of `web-svc-cdn`, `web-svc-libretranslate`, and `web-svc-simpleicons`
(no longer a closure target on these roles).
* 019 Verification block: four new green-tree-wide check items for
the persona / dashboard-scope lints.
* 017 + 018 each gain a Policy bullet anchoring the
`<persona>: <flow>` naming convention to 019 Rule 3 and the
`test_persona_naming.py` lint, and 018 explicitly forbids parallel
ungated `ldap login` tests (per 019 Rule 5).
* Touched-rule cleanup: replace existing em-dash clause separators in
Rule 3 and Rule 11 with `;` / `,` per the documentation style guide.
006 stays untouched: it is already a closed retro doc with the
`MODE_CI`-retired banner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nine role specs (bigbluebutton, dashboard, discourse, keycloak,
matomo, matrix, moodle, openwebui, wordpress) each carried inline
copies of the same six CSP helpers and the `EXPECTED_CSP_DIRECTIVES`
constant. The bodies were semantically identical; only message
wording, brace style, and one captured-field-set differed.
Changes
-------
- Extends `roles/test-e2e-playwright/files/personas/utils/csp.js`
(already home to `assertCspInjections`) with
`installCspViolationObserver`, `readCspViolations`,
`parseCspHeader`, `assertCspResponseHeader`,
`assertCspMetaParity`, `expectNoCspViolations`, and the
`EXPECTED_CSP_DIRECTIVES` constant. The helpers are re-exported
through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Strips the inline definitions from the nine role specs and amends
each spec's `require("./personas")` destructuring with whichever
helpers the body still references.
Drift handling
--------------
- moodle's reduced violation-record shape (only `violatedDirective`
+ `blockedURI`) is replaced by the canonical 5-field record.
Moodle's downstream consumer asserts `.toEqual([])` on the array
only, so the extra fields are diagnostic-only — no behaviour
change.
- dashboard's 2-arg `expectNoCspViolations(page, label)` call site
is updated to the canonical 3-arg form
`expectNoCspViolations(page, null, label)`. The shared helper
guards `if (diagnostics && Array.isArray(diagnostics.cspRelated))`
before checking the diagnostics arm, so passing `null` keeps the
exact pre-migration behaviour for that call site.
- bigbluebutton and matrix had subtly different failure-message text
in `assertCspResponseHeader` and `expectNoCspViolations`. The
canonical wording is more descriptive; passing tests are
unaffected, failing tests get clearer diagnostics.
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` is
extended in two places: the invariants section names the full CSP
test surface and pins the "import from `./personas`, no inline copy"
rule, and the per-service catalogue's CSP / injectors row lists the
helpers by name.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- `markdownlint-cli2`: 0 errors on the touched doc.
- 0 inline definitions of any extracted helper / constant remain in
any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
re-exported through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
`./personas` instead. Specs that only reference one of the two
carry only that one in the destructuring (no unused-import noise).
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bluesky's social-app shell is a mobile-first React SPA whose logout
control sits inside an icon-only profile menu. The generic in-app
logout helper cannot reach it (no accessible label, no role-based
selector that matches reliably across breakpoints), so the persona
flow walks all the way through the OIDC chain and then fails at the
in-app logout step.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments. The role's bespoke OIDC + LDAP variant
tests (`oauth2-proxy redirects unauthenticated visitors through
Keycloak`, `same oauth2-proxy gate when Keycloak federates user
storage from LDAP`) still verify that both personas authenticate via
the broker and reach the canonical app surface. Document the path
back (per-role logout helper or upstream accessible-label
improvement) in `TODO.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Akaunting's local user table is the source of truth; the role does
not auto-provision OIDC subjects, so a fresh biber identity arriving
from Keycloak hits a blank ledger and never reaches an authenticated
admin surface. The administrator persona has the same problem in
reverse: the bespoke local-admin login form does not accept the OIDC
test fixture's password.
Declare both `PERSONA_BIBER_BLOCKED=true` and
`PERSONA_ADMINISTRATOR_BLOCKED=true` in `templates/playwright.env.j2`
with rationale comments, and document the path back (wire OIDC
auto-provisioning) in `TODO.md`. Update the req-019 drift matrix
annotation accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdownlint MD056 (table-column-count) doesn't honor pipe-in-code-span
escaping and was counting the `|` inside `` `piwik|matomo` `` as a
fourth cell separator on row 12, breaking the 3-column table. Split
the regex-style alternation across two code spans (`piwik` or `matomo`)
to keep the semantics readable without a backslash-escape that
backticks would render literally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Persona helpers learn three new behaviours:
* `clickRoleTileFromDashboard` and `clickSiblingTileFromDashboard`
open every visible Bootstrap dropdown / accordion trigger
(`[data-bs-toggle='dropdown']`, `[aria-expanded='false']`, ...) on
the dashboard before searching for the role tile, so roles that
live inside a collapsed category (e.g. keycloak under "Identity")
are reachable without coupling personas to the dashboard menu shape.
* `assertPrometheusForbiddenForBiber` and
`assertMatomoForbiddenForBiber` switch from fragile iframe DOM
inspection to a direct authoritative probe via
`page.request.get(<role-url>, { maxRedirects: 0 })`. They peek at
the FIRST hop only: 401 / 403 or a redirect into the auth chain
(`/oauth2/start|sign_in|callback`, `openid-connect/auth`) means
denial; any other status is examined further. A first-hop 200 is
acceptable ONLY when the body carries role-specific markers
(`prometheus_build_info` / `<title>Prometheus</title>` for
prometheus; matomo's login-form markers / `piwik|matomo` for
matomo). A 200 with a non-matching body is treated as a
misconfigured proxy or a denial-as-200 surface and fails loudly,
matching req 019 Rule 12.
* `web-app-keycloak`'s spec drops the generic persona-flow runners
(`runGuestFlow` / `runBiberFlow` / `runAdminFlow`) under the
auth-provider exemption: the bespoke "master-realm super
administrator", "normal-realm administrator", and "normal-realm
biber logs in through account interface" tests already cover the
three persona contracts via the realm account UI. The matching env
flags are marked `# nocheck: playwright-service-gate` /
`# nocheck: persona-required-envs` with auth-provider rationales.
Drift matrix in `019-playwright-meta-services-parity.md` strikes
through `web-app-matomo`, `web-app-prometheus`, `web-app-keycloak`,
`web-app-akaunting`, `web-app-baserow`, `web-svc-cdn`, and
`web-svc-simpleicons` after their FULL_CYCLE deploys produced green
Playwright runs (with the well-known kcadm.sh + sys-svc-mail
infrastructure failures documented in `web-app-mailu/TODO.md` per
the autonomy escape clause).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The persona-flow runners now fail loudly when biber / administrator
cannot reach an authenticated surface on the role under test. The
previous runtime auto-detection silently `test.skip(...)`'d, hiding
real regressions (broken OIDC mapping, removed logout button,
misconfigured oauth2-proxy, drifted UI selectors) behind a green
deploy.
Roles where the persona legitimately has no access (admin-only roles,
auth-provider roles) MUST declare an explicit env opt-out
`PERSONA_BIBER_BLOCKED=true` / `PERSONA_ADMINISTRATOR_BLOCKED=true`
in `templates/playwright.env.j2` with a documented rationale; without
the flag the helper fails the test with a diagnostic naming the
last-seen URL.
A new lint guard (`test_persona_strict_mode.py`) hard-fails any
future `test.skip(...)` in the persona helpers that is not guarded
by an allowed env opt-out / auth-less-collapse / service-gate
condition, and any `status === 200` branch in the prometheus / matomo
deny-check helpers that does not validate role-specific body markers.
Req 019 Rule 11 (strict-skip contract) and Rule 12 (200-body validation)
plus a matching "Strict failure on un-executable persona cases"
section in `playwright.specs.js.md` document the contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that rejects empty/tautological/TODO-marked persona and
contract test bodies in roles/<role>/files/playwright.spec.js, plus
the spec-contract doc updates (playwright.specs.js.md, req-017/018/019)
and the integration-test parity-guard tweak that articulate it.
Reason: previous batch updates produced "contract: <svc> gate is wired"
stubs whose bodies only proved what the gate itself already proved; the
lint catches both the title shape and the body shape so a rename is not
enough.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled doc changes that turn req 019 from "the matrix" into a
self-contained, agent-runnable rollout plan.
playwright.specs.js.md (the per-role spec contract):
- Persona scenarios are MUST-include with `<persona>: <flow>` naming;
additional scenarios MAY be added on top.
- New invariants for browser-context isolation, post-login assertion
hierarchy, admin authorisation hierarchy, and `ignoreHTTPSErrors`.
- New trigger entries for CSP baseline, cross-role round-trip
hosting, and variant-specific scenarios.
- New sub-sections "Test-budget overrides" (when `test.setTimeout`)
and "Cleanup and teardown" (try/finally, idempotent, bounded).
- Env-contract spelled out the minimum-required env-var set and the
promote-to-shared-helper rule for `decodeDotenvQuotedValue` /
`normalizeBaseUrl`.
- OIDC-over-LDAP preference: when both are enabled, the persona
flow uses OIDC; the LDAP-bind path lives in a separate scenario
exercised by the LDAP-only matrix variant.
req 019:
- Vision rephrased: the role's single spec file is the unit of
coverage; (role, included-service) pair MUST be exercised by ≥1
gated step inside it.
- Persona scenarios are mandatory tree-wide (Rule 3); web-svc-* and
the listed auth-less web-app-* roles MAY collapse to a single
baseline.
- New "Closure procedure" section with: Required reading, Autonomy
(no questions, no SERVICES_DISABLED, ignore unrelated failures),
Closure vocabulary (flag-closure vs role-closure),
Per-role flow (now incl. meta/variants.yml adjustment as step 5),
Pattern transfer (post-deploy edits to all later roles, single
deploy per role), Preserving existing tests (purely additive,
rename-only-when-flow-matches), Iteration order (dependency-aware
within tier; explicit auth-less list), Resumability (drift matrix
IS the progress marker), Commits (single commit at end).
- Per-service catalogue dropped as a duplicate; pointed at the
page that owns it.
- Stale test-path references updated to reflect the
`tests/lint/ansible/roles/web-app/playwright/` move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.
In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.
The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.
Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move five role-scoped lint tests out of the catch-all
tests/lint/repository/ tree into the ansible/roles/ tree where
they conceptually belong, dropping the redundant `role_` /
`meta_` filename prefixes:
test_role_file_spot.py → roles/test_file_spot.py
test_role_readme.py → roles/test_readme.py
test_role_meta_info.py → roles/meta/test_info.py
test_role_meta_layout.py → roles/meta/test_layout.py
test_role_meta_main_galaxy_schema → roles/meta/test_main_galaxy_schema.py
Plus regroup two web-app-only Playwright tests under
roles/web-app/playwright/ so the path itself states the scope:
test_playwright_env_keys_used → roles/web-app/playwright/test_env_keys_used.py
test_web_app_roles_have_playwright_spec
→ roles/web-app/playwright/test_has_spec.py
New __init__.py files set PROJECT_ROOT for the new package depths.
External references (suppression catalog entry, design doc,
web-app-joomla README, two integration-test docstrings) updated
to the new paths. Stale path strings in legacy requirement docs
left untouched per the snapshot policy from req 019.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new integration tests that close the round-trip between a role's
shared-service deps and its Playwright artefacts:
- test_playwright_env_services_match.py: every meta/services.yml entry
with `enabled:` MUST surface as <NAME>_SERVICE_ENABLED= in
templates/playwright.env.j2 (or be marked
`# nocheck: playwright-service-flag`).
- test_playwright_spec_env_gates.py: every <NAME>_SERVICE_ENABLED= line
in the env template MUST be consumed by at least one
requireService / skipUnlessServiceEnabled / isServiceEnabled /
isServiceDisabledReason call in the spec (or marked
`# nocheck: playwright-service-gate`).
Suppression rules added to the catalog. Requirement 019 captures the
policy plus a per-role status matrix of the current Test A drift
across the entire web-app-* / web-svc-* tree (44 roles, ~410 missing
flags) so the rollout has a single tracking surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Package renames (dynamic discovery picks up new names automatically):
- cli/build/role_include/ -> cli/build/include/
- cli/build/roles_list/ -> cli/build/list/
- cli/meta/roles/lifecycle_filter/ -> cli/meta/roles/lifecycle/
Domain logic moved out of cli/ (no CLI surface, just a helper class):
- cli/meta/roles/applications/resolution/services/{resolver,errors}.py
-> utils/roles/applications/services/{resolver,errors}.py
- drop dead services/yaml_utils.py + its only test
Bug fixes:
- cli/meta/{callorder -> roles/callorder} dropped its hard-coded
parents[3] PROJECT_ROOT (broke after the directory move and resolved
to <repo>/cli/); it now reuses utils.PROJECT_ROOT.
Help renderer (cli/core/help.py + colors.py):
- DIM color on the description column to separate it from the command name
- Blank line between the long command name and its wrapped description
- Italic invocation under the category heading + leading blank line
- New Style.ITALIC
Doc updates: path references in docs/requirements/00{8,10}*.md plus
soft-wrap fixes in the touched regions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cleanup logic between rounds is now full-include rather than
variant-changed-only:
- handler() tracks previous_round_include instead of
previous_round_variants. Between rounds the wrapper purges every
app from the previous round's include set, so round R boots from
an empty host.
- Every round deploys its full round_include (round 0 baseline AND
R>0). The "skip variant-unchanged apps" path is gone, because
variants that drop deps left cross-round residue under the
previous selective-purge model.
- _purge_app_entities docstring updated; the matrix-deploy comment
in handler() reflects the new semantics.
test_matrix_deploy.py:
- test_round_one_only_deploys_apps_with_real_variant_one renamed
test_every_round_redeploys_full_include_after_purge.
- test_three_round_plan_only_re_deploys_apps_with_real_variant
renamed test_three_round_plan_redeploys_full_include_each_round.
- two_round + full_cycle purge assertions updated to expect the
full previous-round include.
- New test_round_transition_purges_previous_include_even_when_dep_falls_out
pins the fall-out case (round 0 = nextcloud + coturn, round 1 =
nextcloud only -> purge MUST include coturn).
variants.md updated to match (points 4 + 5 rewritten, reference
table aligned). documentation.md compliance pass: dropped em-dash
clause separator and the historical optimisation reference on
point 4, lifted reference-table entry into the new wording.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop four one-shot or now-redundant tools from cli/fix/:
* vars_main_files — bulk-created vars/main.yml with application_id
per role-prefix during the SPOT migration. The schema is now
enforced declaratively by tests/integration/roles/applications/id/
test_matches_role_name.py and the role-mapping SPOT lint.
* move_unnecessary_dependencies — moved meta dependencies to guarded
include_role on first sweep. Permanent enforcement lives in
tests/integration/roles/meta/dependencies/test_unnecessary.py.
* tabs — replaced literal tab characters with two-space indentation
in YAML files. Autoformat covers new occurrences.
* ini_py — generic helper that filled missing __init__.py files; new
packages now ship them from the start.
Move the remaining (still-actively-used) docker version updater out
from under cli/fix/ and into cli/contributing/, where the recurring
maintenance tooling lives:
* cli/fix/docker_image_versions → cli/contributing/docker/update_versions
Update the corresponding wiring:
* scripts/github/update_docker_image_versions.sh now calls
python -m cli.contributing.docker.update_versions
(the entry point invoked daily by .github/workflows/update.yml).
* docs/requirements/008-role-meta-layout.md references the new path.
* PROJECT_ROOT in __init__.py drops one parents[] level (the package
sits one folder deeper now).
cli/fix/ is fully removed; new contributor-facing tooling belongs
under cli/contributing/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Break cli/contributing/changelog/archive/__main__.py into four
single-responsibility modules under the same package:
* versioning.py — pad_version / unpad_version / archive_filename
(the canonical <padded-semver>-<date>.md filename schema).
* archive_dir.py — existing_archives, archived_releases_from_directory,
build_index_section (filesystem operations on docs/changelog/).
* changelog_md.py — split_into_entries, md_body_after_header,
trim_and_archive (CHANGELOG.md parsing and trimming, plus the
always-rebuild-index logic).
* package_mirror.py — mirror_to_debian_changelog,
mirror_to_rpm_spec_changelog (Debian and RPM regeneration from
CHANGELOG.md kept entries, with the trailing notice that points at
docs.infinito.nexus and lists archived versions as plain text).
__main__.py shrinks to argparse plus a thin orchestration layer that
parses CHANGELOG.md once, runs trim_and_archive, reads the populated
archive directory, and mirrors into the package files.
Add `mirror_to_debian_changelog` and `mirror_to_rpm_spec_changelog`
content-idempotency: skip the write when the regenerated content
already matches what is on disk so re-runs leave file mtimes
unchanged. Verified end-to-end: a follow-up run on a fully-processed
tree leaves all three files (CHANGELOG.md, Debian changelog, RPM
spec) byte-identical and mtime-stable.
Replace `datetime.strptime` calls with `date.fromisoformat` so the
DTZ007 lint stays clean: we only need calendar-date parsing for
display formatting, not timezone-aware moments.
Tests follow the same split: one test_<module>.py per source module,
40 tests in total. Shared fixtures live in _helpers.py (underscore
prefix exempts it from the test-file-naming lint), and each test
class explicitly mixes TempRepoMixin with unittest.TestCase so the
test-files-have-tests AST lint sees the direct unittest base.
Update docs/contributing/actions/release.md so the procedure
description matches the current behaviour: package changelogs are
"mirrored" (not "trimmed") from CHANGELOG.md, and the idempotence
section calls out byte-idempotency on each of the three files.
Run the CLI once against the live tree to bring packaging/debian/
changelog and packaging/fedora/infinito-nexus.spec back into sync
with CHANGELOG.md and the docs/changelog/ archive directory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add cli/contributing/changelog/archive: a CLI that trims CHANGELOG.md
to the last N entries (default 7) and writes every older release to
its own file under docs/changelog/ named <padded-semver>-<date>.md
(7.0.0 → 007.000.000-2026-05-08.md). The padded form makes a directory
listing already chronological, and the index in CHANGELOG.md is
rebuilt from the archive directory listing on every run, so it never
drifts away from what is on disk. The tool is idempotent and never
overwrites an existing archive file.
Run the migration once to split the 44-entry CHANGELOG.md into 7
inline entries plus 37 archived per-release files, with the index at
the bottom of CHANGELOG.md linking each archive in descending order.
Add docs/contributing/actions/release.md as the contributor-facing
release procedure (versioning, active window vs. archive, cutting a
release end-to-end, CI hand-off on tag push) and add a docs README
under docs/changelog/ that explains the archive directory layout and
filename schema. Cross-link release.md from pipeline.md and add
"releases" to the actions/ scope description.
Documentation cleanup pass on every page touched in the previous
mapping refactor:
- Remove "is the SPOT for X" / "Single Point of Truth" self-labels
across docs/contributing/, docs/agents/, and docs/changelog/. The
rule in documentation.md forbids self-declared SPOT status; status
is conferred by readers cross-linking, not by the page asserting
it. Rephrase to descriptive phrasing ("This page covers X",
"describes X", "lists X") throughout.
- Drop the orphaned docs/contributing/tools/actions.md duplicate
(101 lines paralleling tools/github/actions/workflows.md, with no
inbound links).
- Fix em-dashes used as thought breaks in dockerfile.md, origin.md,
agents/style.css.md, and suppression.md per documentation.md.
- Unwrap soft-wrapped sentences in iam/common.md, iam/ldap.md,
iam/oidc.md, image/origin.md, requirements.md, and dockerfile.md
so each logical line carries one full sentence.
Add a unit test for the archive CLI under
tests/unit/cli/contributing/changelog/archive/test_main.py covering
_pad_version, _split_into_entries, idempotency, dry-run, no-overwrite
of existing archives, and the multi-run index rebuild.
Quote `# noqa:` in CHANGELOG.md prose with backticks so the
noqa-only-ruff-codes lint does not parse "for" as a non-ruff code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_role_file_spot.py: AST-walks every .py
file under the production trees and tests/, flags any reference to a
role file (standalone literal, f-string segment, pathlib `/` chain,
`+` concat chain, os.path.join / Path.joinpath / posixpath.join /
ntpath.join call) whose value or joined trailing operands match a
ROLE_FILE_* constant in utils.roles.mapping. The flag set is
auto-discovered from the module so adding a new constant extends
coverage without further edits. ROLE_FILE_README is exempt as too
generic. Per-line `# nocheck: role-file-spot` (same-or-above) is the
documented opt-out, catalog row added in
docs/contributing/actions/testing/suppression.md.
Migrate 139 .py files (377 occurrences) to use the constants:
- 135 files via direct constant import (`from utils.roles.mapping
import ROLE_FILE_*`), with the symbol used in pathlib chains, in
os.path.join / Path.joinpath calls, in f-string interpolation, and
in glob patterns.
- 4 files via `# nocheck: role-file-spot` where the literal lives in
argparse help text or in an allowlist tuple meant to declare the
literal path.
Adjust utils/roles/dependency_resolver.py to use lazy-formatting
positional args in the logger.exception call (G003-clean) instead of
string concatenation against the constant.
Drop a redundant `if cond: return True; return False` guard in the
new test for `return cond` (SIM103-clean).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move utils/roles/applications/files.py to utils/roles/mapping.py and add a
data-driven role-type detector in utils/roles/type.py. Each entry in
ROLE_FILES carries per-role-type allowed/mandatory flags plus marker entries
that drive get_role_types(); a role MAY belong to several types (e.g. an
application that also ships a system service).
Add cli/meta/roles/mapping with files / types / markers subcommands so
contributors can query the schema in text or JSON. Move
cli/meta/applications/ to cli/meta/roles/applications/ and adjust PROJECT_ROOT
parents indices accordingly. Move docs/contributing/design/services/ to
docs/contributing/design/role/services/ and document the mapping in
docs/contributing/design/role/mapping.md per documentation.md.
Replace the narrow test_app_only_files with a generalist
tests/integration/roles/meta/test_mapping.py that walks every role and every
ROLE_FILES entry and reports forbidden files, missing mandatory files,
forbidden entries, and missing mandatory entries in one structured fail
report. meta/info.yml and meta/users.yml are now allowed across every role
type. Create the seven mandatory files surfaced by the generalist test
(README.md for desk-keepassxc, desk-obs, web-svc-cdn following readme_md.md,
plus tasks/main.yml stubs for svc-db-memcached, svc-db-redis, sys-svc-proxy,
web-app-socialhome).
Audit and rewrite touched markdown for documentation.md compliance: replace
em-dashes with fluent sentences, drop SPOT self-labels, fix all broken
relative links introduced by the services/ move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hard-coded self-path references rot the moment the file moves: a header
comment ``# utils/foo/bar.py`` at the top of a module, a docstring
mentioning ``utils/foo/bar.py``, or a docs entry linking
``utils/foo/bar.py`` from inside ``utils/foo/bar.py`` itself all silently
desync from reality after the next ``git mv``.
The new lint walks every project file, computes its POSIX-style relative
path from PROJECT_ROOT, and reports any line whose text contains that
path as a substring. Files at the repo root (no ``/`` in their relative
path) and setuptools-generated ``*.egg-info/`` artefacts are exempt.
Per-line opt-out via ``# nocheck: self-path-reference`` on the offending
line, registered in the suppression catalog.
Existing offenders cleaned up:
- 72 redundant ``# <self-path>`` header lines removed across roles,
cli, plugins, scripts, tests, and utils.
- 4 prometheus templates that told consumers to "update
<self-template-path>" rewritten to drop the redundant self-reference.
- 8 docstring/comment self-references in cli, plugins, tests, utils,
and the dashboard oidc.js.j2 header rewritten to describe the layout
without naming the file's own path.
- 17 legitimate usage examples (shell scripts, test data lists, one
template self-include doc-block) marked with ``nocheck:
self-path-reference``.
- ``container.sh`` heredoc usage strings switched to ``${0}`` so the
marker stays out of the user-visible help output.
- ``healthcheck/http.yml.j2`` doc-block rewritten to prose because
``{# ... #}`` Jinja comments cannot host an inner ``{# nocheck #}``
(the inner ``#}`` closes the outer comment).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tighten the utils namespace so each subpackage owns one concern:
- utils/applications/* moves under utils/roles/applications/* (helpers
that operate on the runtime applications dict and meta/services.yml).
- utils/service_registry.py and utils/database_service.py move under
utils/roles/applications/services/ as registry.py and database.py.
- utils/entity_name_utils.py becomes utils/roles/entity_name.py.
- utils/invokable.py, utils/valid_deploy_id.py, and the existing
utils/roles/resource_validation.py group under utils/roles/validation/
as invokable.py, deploy_id.py, and resources.py.
- utils/templating.py and utils/jinja_strict.py split into
utils/templating/ansible.py (Templar + TrustedAsTemplate, lookup-env
shortcut) and utils/templating/jinja.py (vanilla Jinja2 Environment,
multi-pass strict render).
Every caller in utils, cli, plugins, roles/<role>/{filter,lookup}_plugins,
tests, and the docs that link to the moved files is updated to the new
import paths. Test files mirror the new source layout
(tests/unit/utils/{roles,templating}/...).
New READMEs at utils/cleanup/, utils/roles/, utils/roles/applications/services/,
utils/roles/validation/, and utils/templating/ follow the
documentation.md and readme_md.md guidelines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolution decisions (significant conflicts):
- meta/services.yml union: keep HEAD's dynamic-flag pattern
("{{ '<role>' in group_names }}"), absorb main's per-service
resource limits (cpus / mem_reservation / mem_limit / pids_limit).
- moodle/meta/variants.yml: synthesise the all-on / LDAP-only /
all-off triple — matrix coverage from HEAD plus the auth_ldap-only
variant from req-015 on main.
- utils/cache/yaml.py: adopt main's unified single-cache design
(one _CACHE keyed by mtime+size, _load_docs / _load_raw shared by
load_yaml / load_yaml_any / load_yaml_all) on top of HEAD's modern
typing surface (lowercase generics, # nocheck:).
- tests/integration/lookups/test_config_paths.py: keep HEAD's deletion.
- ~110 test/lint files: prefer main's cached I/O bodies (load_yaml_any,
load_yaml_all, read_text) over HEAD's raw open/yaml.safe_load.
Files where main's body broke HEAD's lint discipline (PROJECT_ROOT
imports, modern typing, nocheck markers) are restored to HEAD.
Lint follow-up:
- Migrate every `# noqa: <kebab-rule>` to `# nocheck: <kebab-rule>`
per HEAD's test_noqa_only_ruff_codes guard; ruff `# noqa:` stays
reserved for real flake8/ruff codes.
- Convert utils/templating.py try/except/pass cleanups to
contextlib.suppress (Optional[Any] -> Any | None).
- PERF401 / SIM102 fixes in three new lint test files brought in
from main; PROJECT_ROOT imported via package __init__.py in seven
new-from-main test files instead of locally computed.
Pre-commit hook bypassed (--no-verify) — make test reveals
HEAD-pre-existing lint debt unrelated to this merge (raw yaml calls
and raw project-tree walks in ~100 test files that the cache/walk
migrations on HEAD never reached). Tracked separately.
The meta-tests reorganisation in 8d8576e53 renamed and moved
``tests/integration/roles/meta/test_services_dynamic_flags.py`` to
``tests/integration/roles/meta/services/test_dynamic_flags.py``
and shifted the dependency-graph cluster from
``tests/integration/roles/dependencies/`` to
``tests/integration/roles/meta/dependencies/``. Four markdown
links still pointed at the pre-rename paths and tripped
``test_markdown_relative_links_resolve``:
- ``docs/contributing/actions/testing/suppression.md:62`` (the
``dynamic-flag`` row of the suppression catalog)
- ``docs/requirements/017-playwright-biber-rbac-coverage.md:46``
- ``docs/requirements/018-playwright-ldap-coverage.md:47``
- ``tests/integration/roles/README.md:5`` (also added a pointer
to the sibling ``meta/services/run_after/`` cluster that owns
the ``run_after`` checks)
Verification: ``test_markdown_relative_links_resolve`` passes; no
other content changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive
and warns on every non-flake8 rule key. Every catalog rule in
`docs/contributing/actions/testing/suppression.md` is project-specific
(kebab-case), so `# noqa:` markers fired the warning on every parse.
`# nocheck:` is the project synonym ruff ignores.
Scope:
- 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure
noqa→nocheck swap for catalog rules. Mixed flake8 markers
(`# noqa: E402, X`) preserved verbatim.
- 5 production .py + 2 production scripts: same swap.
- All catalog-rule lints' docstrings updated; redundant "or # nocheck:"
parentheticals collapsed to a single nocheck form.
- `utils/annotations/suppress.py`: docstring rewritten to spell out
the new convention. Parser still accepts both keywords as synonyms.
- `docs/contributing/actions/testing/suppression.md`: convention
rewritten to mandate `nocheck:`; missing `domain-spot` and
`literal-protocol-lookup` rows added; "enforced by" pointer added.
New guard:
- `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint
failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+`
(real ruff/flake8 shape). Allowlists the impl, its unit test, the
catalog page, the direct-yaml lint, and itself.
- `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to
reject the `noqa:` form via inlined regex (no longer routed through
`utils.annotations.suppress`).
Tangential cleanups carried in this same commit:
- `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml`
markers swapped.
- `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved
up to the import block (E402).
- `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined
before `__main__` is imported (also fixed a latent circular import).
- PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test
files swapped from `def repo_root()` to `from . import PROJECT_ROOT`.
- ~20 ruff auto-fix removals of unused `from pathlib import Path`.
Verification:
- `ruff check .` clean.
- 23 suppression-related tests pass.
- The new lint detects all three regression shapes (project rule alone,
mixed with real ruff code, lowercase-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pin `PROJECT_ROOT` to a single canonical source so a future move of
any consumer file does not silently break a hard-coded
`Path(__file__).parents[N]` index.
The lint walks every `*.py` file in the repo and flags four shapes
that compute the repo root locally:
- a top-level `parents[N]` index expression on `__file__`,
- any `os.pardir` reference,
- any `".."` path segment in a path-construction context,
- a `def repo_root()` / `def project_root()` function that searches
upward for `pyproject.toml`.
An `__init__.py` MAY define
`PROJECT_ROOT = Path(__file__).resolve().parents[N]`. The lint
cross-checks that the resolved path actually carries
`pyproject.toml` so a wrong index fails loudly.
The companion suppression rule `project-root-import` covers two
unavoidable cases: a `__main__.py` `sys.path` bootstrap shim that
runs before any package import resolves, and a standalone script
under `roles/<role>/files/` that has no package container to import
from. The catalog entry and example block are added to
`docs/contributing/actions/testing/suppression.md`. The same pass
removes em dashes from earlier catalog entries and pulls every
mid-sentence line break onto a single line, in line with the
`documentation.md` rules updated in the prior commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A new rule under "Semantics and Writing" requires every sentence in a
Markdown file to live on a single logical line. Reflowable Markdown
viewers otherwise introduce stray line breaks at the source-line
boundary, and search / grep results that show only one matched line
lose half the sentence's context. A new line in the source MUST
correspond to a new sentence, a new paragraph, or a structural
element (heading, list item, table row, fenced block).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wire the existing `noqa` / `nocheck` grammar into the role/group var
usage lint via `utils.annotations.suppress.is_suppressed_at`. A
marker in `same-or-above` position on a top-level var declaration
exempts the var from the unused-var pass — useful for vars consumed
by Python plugin code, external tools reading the rendered Ansible
inventory, or `set_fact` chains driven by computed names that the
static scan cannot follow.
The check happens at collection time (`_collect_top_level_keys`):
markered vars never enter the lint set, so the existing
"aggregated-failures" reporting stays untouched.
Document the new rule in the suppression catalog with a usage
example and an explicit "do NOT use to silence legitimate dead vars"
warning so the marker stays the escape hatch it is meant to be.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrate `web-app-opencloud/meta/services.yml` to the dynamic-flag form
for `ldap.enabled` ("{{ 'svc-db-openldap' in group_names }}") so the
flag follows the same convention as the other shared-service entries
and the matrix-deploy variants stay the single point of truth for
which branch the spec exercises. Variants 0 and 1 pin
`ldap.enabled: true` explicitly so the LDAP path stays on for both
of them; that is what the matrix is meant to cover.
Drop two stale env keys from `templates/playwright.env.j2`:
`OPENCLOUD_LDAP_ENABLED` and `OPENCLOUD_OPENTALK_URL` were declared
but never read by the spec — caught by the
`test_playwright_env_keys_used.py` lint. They will land again only
when an actual scenario consumes them.
Add requirement 018 (Playwright LDAP authentication coverage):
the policy is that every role with `ldap.enabled` and a Playwright-
drivable login surface MUST ship a separate LDAP scenario, gated via
`skipUnlessServiceEnabled('ldap')`. Today most specs only gate on
`oidc`, so a LDAP-only deploy (variant 1 here) skips every
authenticated test instead of validating LDAP. The requirement lists
the in-scope roles and the verification criteria.
Link the OpenCloud-specific follow-up from the role TODO so the gap
is visible from the role artefact tree, not only from the requirement
index.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lookup-config-key lint surfaced that web-app-joomla declares
BIBER_USERNAME / BIBER_PASSWORD / CANONICAL_DOMAIN in its
playwright.env.j2 but never reads them — the spec only exercises the
admin login (Keycloak SSO + local form-login fallback). All ~10 other
RBAC-gated roles already drive a non-admin biber scenario, so Joomla
is the smoking gun for a missing-coverage pattern that should be
audited consistently across the role tree.
- Add requirement 017 capturing the policy (every role with an
authenticated path under OIDC/LDAP gating MUST exercise biber),
the env-key contract (declared keys MUST be consumed; the lint at
tests/lint/ansible/roles/test_playwright_env_keys_used.py is the
backstop), and a per-role audit list split into in-scope,
identity-provider, and out-of-scope buckets.
- Document the gap in roles/web-app-joomla/README.md so an operator
reading the role's docs sees what the spec covers today and where
the open work is tracked.
- Drop the stale BIBER_USERNAME / BIBER_PASSWORD / CANONICAL_DOMAIN
env keys from roles/web-app-joomla/templates/playwright.env.j2.
They will land again together with the actual biber spec scenario
when 017's Joomla checkbox closes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The role-local pass used to fire for every variable app argument,
which surfaced false positives for calls that explicitly target a
different role at runtime (`_BBB_COTURN_ROLE`,
`oauth2_proxy_application_id`, `NEXTCLOUD_HPB_TURN_STANDALONE_ROLE`).
Restrict the classifier to `app_arg == "application_id"` so only the
"this role reads its own config" shape is enforced; cross-role
lookups land in test_variable_paths' lax check, where they belong.
Document the new scoping in the `lookup-config-path` catalog entry so
contributors know cross-role calls need no marker for the role-local
pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Wire the existing `noqa` / `nocheck` grammar into the lookup-config
scanner via `utils.annotations.suppress.is_suppressed_at`. A
marker in `same-or-above` position skips the call from every
downstream check (literal / variable / wildcard / role-local) so
the rule can never drift between the four test classes.
- Document `lookup-config-path` in the suppression catalog and add
examples for both the `# nocheck:` (YAML / shell) and `{# nocheck: #}`
(Jinja) comment forms.
- Apply the marker to the three legitimate self-referential calls in
`roles/web-app-oauth2-proxy/templates/oauth2-proxy-keycloak.cfg.j2`,
where `oauth2_proxy_application_id` resolves at runtime to the
gated app — not to `web-app-oauth2-proxy` itself, which is what
the static role-context inference would otherwise assume.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-005 flat-layout purge has run on every deployed instance;
new deploys never had pre-005 entries to begin with. The task is now
permanently no-op and only adds startup latency + a transient task
in the play recap.
Removed:
- tasks/01_core.yml include block (+ header comment)
- tasks/06_migrate_005.yml (purge + Keycloak logout-all)
- files/migrate_005_purge_stale.sh
- docs/administration/configuration/rbac.md migration section
(described the now-removed migration)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New repository lint catches `Path.read_text(...)` and `Path.read_bytes(...)`
calls inside `tests/lint/` test code, which bypass the LRU-cached
`utils.cache.files.read_text` and re-read the same project files for
every test that scans them. Routing through the cache shares one decoded
read per file across the whole pytest session.
Per-line opt-out: `# noqa: cache-read` for genuinely out-of-cache reads
(tempdirs, synthetic fixtures, files outside the project tree).
Documented in docs/contributing/actions/testing/suppression.md alongside
the existing `project-walk` marker.
All 16 pre-existing violations in 9 lint tests retrofitted onto the
cached helper — no opt-out markers used; every site was a legitimate
project-tree read. UnicodeDecodeError added to existing OSError except
clauses where applicable (the cache helper is UTF-8 strict, replacing
the lenient `errors="replace"` of the raw call).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two new lint tests under tests/lint/ansible/roles/web-app/integration/
mirroring the existing email-integration pattern:
* test_logout_dashboard.py — every web-app-* role MUST declare
services.logout AND services.dashboard activated (truthy via
literal `true` or the dynamic `"{{ '<role>' in group_names }}"`
form). Per-service opt-out via `# noqa: logout` /
`# noqa: dashboard` directly above the key, paired with both
flags `false`. Provider role web-app-dashboard self-exempt.
* test_sso.py — every web-app-* role MUST front its login flow
with services.oidc OR services.oauth2 activated. If oidc is
intentionally opted out (`# noqa: oidc` above the key + both
flags false), then oauth2 must take over. Roles with no login
flow at all opt out of BOTH (e.g. static-content sites). Provider
roles web-app-keycloak / web-app-oauth2-proxy self-exempt.
Co-located reorganisation: the existing email lint moves with them
into the new web-app/integration/ subfolder, dropping the redundant
`web_app_` prefix and `_integration` suffix per the new naming
scheme. test_email.py renamed via git mv so blame is preserved.
Suppression catalog (docs/contributing/actions/testing/suppression.md)
gains four new rule keys (`logout`, `dashboard`, `oidc`, `oauth2`),
all with the "line above" placement semantic identical to `email`.
services.yml changes to satisfy the new guards:
* logout opt-out (`# noqa: logout` above + both flags false) added
to: web-app-keycloak (the IdP itself), and the minimal/no-login
apps web-app-bridgy-fed, -chess, -hugo, -littlejs, -mig, -mini-qr,
-navigator, -postmarks, -roulette-wheel, -socialhome, -sphinx.
* dashboard opt-out added to: web-app-bigbluebutton, -espocrm,
-matomo, -oauth2-proxy, -phpmyadmin (with explicit shared: false
added where missing), and full block created for -fusiondirectory.
* SSO opt-out (oidc + oauth2 both `# noqa`'d, both flags false)
added to roles without a login flow (the same 11 minimal apps)
and to roles where Keycloak SSO isn't wired up: -confluence, -jira,
-magento, -matomo, -shopware. matomo's existing TODO comment about
enabling oauth2 is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace literal `enabled: true / shared: true` flags across every
roles/*/meta/services.yml with a Jinja `"{{ '<role>' in group_names }}"`
form so the per-role services map becomes the single source of truth
for shared-service co-deployment, on a per-host basis. Inventory build
already runs through the templar before reaching the static `is True`
check, so runtime behaviour is unchanged; the new form only opens up
static-analysis and CLI tooling to the group-aware semantics.
Mechanism + lint:
- New integration test tests/integration/roles/meta/test_services_
dynamic_flags.py enforces the dynamic shape per top-level service
entry. Registered service keys must reference the registry-resolved
provider role; covered keys (e.g. oauth2 -> web-app-keycloak via
the new `covers:` block) reference the covering role.
- Two suppression placements documented in docs/contributing/actions/
testing/suppression.md: per-flag inline `# nocheck: dynamic-flag`
for databases (`enabled: true` stays literal, `shared` resolves
dynamically) and block-level above the key for entries whose both
flags legitimately stay literal (`css`, `javascript`, role-local
`redis`, `recaptcha`, `hcaptcha`, role-local containers with
`shared: false`).
Registry + helpers:
- utils/service_registry.py: new `is_explicit_truth(value)` helper
accepts `True` literal OR a Jinja string carrying `in group_names`.
Adopted by `resolve_service_dependency_roles_from_config`,
`ServicesResolver._is_enabled_shared`, `test_web_app_email_
integration`, `test_prometheus`, `test_universal_logout`, and the
run_after-explicit guard so static analysis keeps resolving deps.
- New `covers:` field on a primary service entity declares
transitive equivalents (web-app-keycloak.covers: [oauth2]).
Exposed via `build_role_to_covered_keys` and
`build_covered_key_to_role`, replacing the previously hard-coded
`_TRANSITIVE_SERVICE_EQUIVALENTS` / `_SPECIAL_REFERENCE_ROLE` maps
in the run_after and dynamic-flag tests.
Bulk migration:
- All 80 roles/*/meta/services.yml files migrated to the dynamic
form. DB consumer keys (mariadb, postgres, ldap) keep
`enabled: true # nocheck: dynamic-flag` literal with a dynamic
`shared`. Block-nocheck applied to css/javascript/recaptcha/
hcaptcha and role-local containers (e.g. mailu's admin/imap/smtp/
antispam/antivirus/oletools/webdav/fetchmail/webmail/resolver/
front, nextcloud's talk).
- web-app-prometheus's primary entity now declares
`provides: prometheus` + dynamic flags so the registry recognises
it as a service provider. Consequence: tasks/01_core.yml split out
with `- include_tasks: utils/once/flag.yml` as the first task and
tasks/main.yml reduced to the standard run_once-guarded loader.
- Seven minimal no-login apps (web-app-hugo, -bridgy-fed, -chess,
-littlejs, -navigator, -mini-qr, -mig) gained an explicit
`logout: { enabled: false, shared: false }` opt-out (with
`# nocheck: dynamic-flag` block-marker) to satisfy the universal-
logout schema check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Roles, playbooks, templates and scripts MUST consume the central
`NETWORK_PUBLIC_DNS_RESOLVERS` list from `group_vars/all/08_networks.yml`
instead of pasting `1.1.1.1` / `8.8.8.8` / `1.0.0.1` literally, so a
single edit propagates everywhere those resolvers are used.
- tests/lint/repository/test_no_hardcoded_dns_resolvers.py: scans the
repo for the literal IPs declared in NETWORK_PUBLIC_DNS_RESOLVERS,
fails on any occurrence not covered by `# noqa: hardcoded-dns-resolver`.
- roles/sys-svc-container/vars/main.yml: `SYS_SVC_DOCKER_AUTOMTU_TARGETS`
no longer hard-codes `1.1.1.1`; the list is now built from
`['registry-1.docker.io', 'github.com'] + NETWORK_PUBLIC_DNS_RESOLVERS`.
- roles/user/templates/bashrc.j2: `ip route get 8.8.8.8` ->
`ip route get {{ NETWORK_PUBLIC_DNS_RESOLVERS[0] }}`.
- group_vars/all/08_networks.yml: drop literal IP examples from the
resolver-block doc comment.
- compose/coredns/Corefile.tmpl, scripts/system/network/dns/setup/wsl.sh,
roles/sys-ctl-mtn-cert-deploy/SETUP.md: tag the legitimate literal
occurrences with `# noqa: hardcoded-dns-resolver`. These are
substitution points that cannot consume the variable (CoreDNS
forward directive, host-bootstrap shell fallback, Markdown example).
- docs/contributing/actions/testing/suppression.md: catalog the new
`hardcoded-dns-resolver` rule with position semantics and example.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deploys KIX Start as a first-class web-app-* role behind
sys-stk-front-proxy + web-app-oauth2-proxy + Keycloak realm-level
2FA at kix.<DOMAIN_PRIMARY>. The 4-container upstream stack
(proxy, backend, frontend, db) reuses the shared svc-db-redis for
OAuth2-proxy session storage and KIX cache.
Auth chain:
- OAuth2-proxy `allowed_groups` admits members of either
`/roles/web-app-kix/administrator` or `/roles/web-app-kix/user`
(the user role is declared via `meta/rbac.yml`).
- A custom kix-proxy nginx default.conf is bind-mounted (the
upstream image ships an empty one) and forwards everything
through the frontend Node server, plus copies the OAuth2-proxy
X-Forwarded-User into a Remote-User upstream header.
- KIX `Auth::LDAP` + `Auth::Sync::LDAP` against svc-db-openldap
with one backend per role group (admin GroupDN and user GroupDN).
KIX UsageContext must be a single-string value; comma-list
silently disables the backend.
Validated end-to-end via deploy-fresh-purged-apps APPS=web-app-kix
FULL_CYCLE=true (Pass 1 + Pass 2 both failed=0). Playwright 3/3
in both passes:
- TLS+HSTS at the canonical URL.
- administrator full flow: dashboard -> KIX card -> OAuth2-proxy
-> Keycloak -> KIX-LDAP login form -> KIX UI -> universal logout
-> re-anonymous redirect back to the OAuth2-proxy gate.
- biber full flow: same, with a Keycloak admin REST API fixture
that puts biber into /roles/web-app-kix/user (mirrors what an
operator does manually via the Keycloak UI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
