Reorder every target block in the Makefile so the definitions land in
strict ASC order — the order broke during the cascading domain-prefix
renames (install-agent, security-apparmor-*, etc.) and now matches what
test_makefile_targets_sorted_ascending requires.
Rewrite the two for-loops in test_phony_declarations as list-comprehension /
list.extend forms so ruff PERF401 stops complaining and the file passes
autoformat.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Place a single `.PHONY: <target>` line directly above each target's
comment block instead of clustering all phony declarations at the top
of the Makefile. The `help.sh` parser already resets its comment state
on lines beginning with `.`, so the PHONY entry must sit ABOVE the
comment (not between comment and target) to keep `make help` accurate.
Add the previously-missing PHONY entries for `dotenv` and `dotenv-force`
so every task target carries one.
Introduce `tests/lint/filesystem/makefile/test_phony_declarations.py`
to enforce two invariants going forward:
- every target has a matching `.PHONY: <target>` above its comment, and
- every `.PHONY: <target>` references an actually-defined target (no orphans).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four follow-ups uncovered by \`make test\` after 3c9db6142:
* tests/lint/ansible/variables/test_role_and_group_vars_used:
TLS_ENABLED is declared in group_vars/all/02_tls.yml but never
read via \`{{ TLS_ENABLED }}\`; it is consumed by
plugins/lookup/{tls,email,cert}.py, which the YAML/Jinja-scoped
lint cannot see. Tag the declaration with the project's standard
\`# nocheck: unused-var\` marker and document the consumer.
* tests/lint/filesystem/python/test_no_raw_project_walk: the new
test_no_ansible_env_lookups.py used \`Path.rglob("*")\` for its
walk, which is the exact shape the no-raw-walks lint forbids in
test code. Switch to \`iter_project_files(extensions=_SCAN_EXTS)\`
with a relative-path prefix filter; same coverage, shared cache.
* tests/unit/cli/administration/deploy/development/test_inventory_vars_file_spot:
the new lint test legitimately names the dev inventory path
(\`inventories/development/default.yml\`) in its failure message.
Add it to the SPOT-drift-guard allowlist with a justification.
* tests/integration/ansible/jinja/test_variable_definitions: the
debug task added in 0884eb4e2 prints \`ansible_env\` and \`vars\`,
both Ansible magic variables. Add them to the test's reserved-name
list so the static analyzer stops flagging them as undefined.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
\`lookup('env', 'INFINITO_*')\` calls in group_vars / roles couple
deployment-agnostic Ansible code to default.env, which only exists in
dev/CI. Production deploys never source .env, so the lookups silently
fall back to their group_vars defaults and the inventory's intent gets
lost on the way (see the IP4 host_vars-overlay incident fixed in
8980b00b1).
Migrated keys (group_vars/roles -> inventory):
- DOMAIN_PRIMARY (00_general.yml)
- BUILDKIT_ENABLED (00_general.yml)
- CA_CONTAINER_ENABLED (02_tls.yml)
- defaults_networks.internet.{ip4,ip6,dns} (08_networks.yml)
- TEST_E2E_PLAYWRIGHT_KEEP (roles/test-e2e-playwright/defaults/main.yml)
group_vars and the role default now carry literal fallback values;
the dev/CI inventory at \`inventories/development/default.yml\` pulls
the live values from .env via straight \`lookup('env', 'INFINITO_*')\`
calls without any \`| default(...)\` filter, because load.sh guarantees
the keys are set whenever .env exists.
The convention is enforced by the new lint test
\`tests/lint/filesystem/env/test_no_ansible_env_lookups.py\`, which
scans group_vars/, tasks/, roles/, and playbook.yml for any
\`lookup('env', 'INFINITO_*')\` against a default.env key and fails
with a pointer to the dev inventory as the injection site.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.
Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
switched from globbing env/*.env to targeting default.env directly;
test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
STATIC_ENV + every docstring/error message
Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make test-unit` (docker runner) reached run.sh inside the container via
`bash --login`, which never sources the project's env-loader. PYTHON stayed
unset, so the test invocation fell back to the system python3 — which has
none of the runtime deps (ruamel, ansible, bcrypt, requests, certifi,
humanfriendly), producing 132+ ModuleNotFoundError errors.
run.sh now sources scripts/meta/env/load.sh itself (idempotent via
INFINITO_ENV_LOADED), so the env layer applies regardless of how the shell
was spawned.
While load.sh's detect.sh (now scripts/meta/env/python.sh) correctly walks
/opt/venvs and ~/.venvs to find the venv, an ambient VIRTUAL_ENV from a
different project (e.g. /root/.venvs/pkgmgr in pkgmgr's container) would
hijack the resolution. python.sh now scans the project-named venv candidates
first and only falls back to VIRTUAL_ENV when no project venv exists.
Also moves scripts/meta/python/detect.sh to scripts/meta/env/python.sh so the
env-related helpers live under one directory; updates the load.sh source
pragma and the lint-test path that scans it for INFINITO_* registrations.
`make test-unit` goes from 1262 tests / 132 errors to 1999 tests / 0 errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:
* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
env var → defaulted to infinito.localhost, which never resolved
because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
(env-builder, role default, run_one.yml docker exec, playwright.config.js,
lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
compose forwards it as build-arg, volume mount path, and container env;
every Python CLI + shell helper + CI workflow now reads it strictly
(os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
/opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
container) through compose.yml's environment block.
Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls VENV/PYTHON/PIP/PYTHONPATH out of the dotenv handler pipeline
into scripts/meta/python/detect.sh. load.sh now sources detect.sh and
prepends ${VENV}/bin to PATH above the INFINITO_ENV_LOADED guard, so
every BASH_ENV re-entry re-resolves PYTHON the moment a venv
materialises mid-chain — the same fix as b79ed2869 but at the SPOT,
which retires the reload-trick from ansible.sh, python.sh and
pre-commit.sh.
Also renames INVENTORY_VARS_FILE → INFINITO_INVENTORY_VARS_FILE under
the new env-prefix lint, drops the 5 dead INFINITO_ENV_*_LOADED unsets
left over from the multi-loader era, and adds a lint guarding the
INFINITO_ prefix invariant on env/*.env keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/install/venv.sh
-----------------------
INFINITO_VENV_BASE is a static-default convention; the active VENV
may end up outside that base (e.g. \$HOME/.venvs/<name> when
/opt/venvs is not writable, or a wholly custom override). Probing
writability on the legacy base triggered a spurious sudo branch
whenever VENV lived elsewhere -- fatal in no_new_privs sandboxes
where sudo cannot escalate. Switch the writability + mkdir + chown
chain to \$(dirname "\${VENV}") so the script prepares the directory
it is actually about to create the venv in.
tests/lint/ansible/roles/web-app/playwright/test_app_base_url_uses_role.py
-------------------------------------------------------------------------
New lint: every role's templates/playwright.env.j2 must declare
APP_BASE_URL as a reference to the role's own surface. APP_BASE_URL
feeds the shared guest-persona helper and Playwright's global
baseURL; a role that hardcodes a sibling role name (most commonly
'web-app-dashboard') sends the guest test to the wrong site and
breaks any matrix variant where the sibling is not deployed
(ERR_SSL_UNRECOGNIZED_NAME_ALERT).
The 16 web-app role templates that violated the contract switch
from sibling lookups (`lookup('tls', 'web-app-dashboard', ...)`)
to the canonical `lookup('tls', application_id, ...)` form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a meta-layout lint that enforces a `lifecycle:` declaration on the
primary entity of every invokable role (those matched by the paths in
roles/categories.yml). Missing lifecycle is a hard error because the
lifecycle-filtered deploy tooling (`cli meta roles applications type
--lifecycles ...`) silently drops a role without it, so the role would
quietly disappear from any selection.
Additionally emit a GitHub Actions warning annotation (via
utils.annotations.message.warning) for each invokable role whose
lifecycle sits outside the CI-tested set {alpha, beta, rc, stable}. This
is informational only and never fails the test, so the project can keep
shipping pre-alpha / unsupported apps while making it visible at PR
review time which apps lack deploy-test coverage. The CI-tested set
mirrors scripts/meta/resolve/apps.sh:80 (single source of truth).
Allowed-value enforcement remains with TestLifecycleAllowedValues in
tests/lint/ansible/roles/meta/test_layout.py -- this new test only
covers existence and CI-coverage signaling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the shell helpers under scripts/github/ into five subfolders
that match how the workflow files consume them:
runner/ host-runner hygiene (disk, swap, Docker daemon, diagnostics)
cancel/ run cancellation on PR-close and branch-delete events
resolve/ derivation of workflow inputs and outputs from repo state
update/ scheduled Update: Versions pipeline
release/ release-time gates
Drop the now-redundant subfolder prefix from each filename, e.g.
dump_runner_state.sh becomes dump_state.sh and cancel_pull_request_runs.sh
becomes pull_request_runs.sh. Each subfolder carries a README that
describes its scope per documentation.md rules; the top-level README
points at the subfolders without inventorying files.
Propagate the new paths through ten workflow files, three docs
(workflow.md, pipeline.md, secrets.md), and two tests. Fix a real
runtime bug in enlarge_swap.sh, which still referenced the
pre-rename dump_runner_state.sh path. Extend the inline-block-size
lint hint to describe the new subfolder convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Browser vendors deprecated X-Frame-Options/X-XSS-Protection/X-Download-Options/
X-Permitted-Cross-Domain-Policies; CSP (rendered from each role's
meta/server.yml via build_csp_header) is the modern replacement. To make the
SPOT actually win against upstream apps (e.g. Keycloak/Quarkus still emits
X-Frame-Options), the proxy now hides those headers on every location type.
Includes a fail-loud lint that catches any future add_header / Header set/add
re-introduction in .conf / .conf.j2 / .htaccess / .j2 / .jinja / .tmpl files.
- sys-svc-proxy: new upstream_strips.conf.j2 partial bundles HSTS + the 4
deprecated header strips; included in html/media/ws/upload + web-svc-logout
+ web-app-oauth2-proxy locations (nginx array-replacement inheritance forces
per-location placement)
- web-app-keycloak: meta/server.yml frame-ancestors allows the primary-domain
wildcard with the dynamic TLS protocol so dashboard silent-check-sso iframe
loads cleanly
- web-app-dashboard/playwright: drop isOidcSilentCheckNoise — root cause is
now fixed, no noise filter needed
- web-app-dashboard/nginx + web-app-nextcloud/nginx: remove the deprecated
add_header lines themselves
- tests + docs: new lint registers `deprecated-http-header` suppression rule
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`application_id` is not bound when the applications registry is built
(meta/*.yml is rendered before any role-vars binding), so
`lookup('config', application_id, ...)` returns the raw unrendered
Jinja string instead of the resolved value. The lint test enumerates
`roles/*/meta/*.yml`, picks each line that contains a Jinja expression
(`{{ ... }}` / `{% ... %}`), and fails if `application_id` appears as
a variable inside it (kwarg form `application_id='web-app-…'` is
correctly excluded).
The failure message points at every offending file:line and tells the
author to spell out the role id explicitly
(`lookup('config', 'web-app-nextcloud', …)`).
Also reduces the gitea credentials.native_metrics_token description to
the value's meaning ("Bearer token authorizing scrape requests to
Gitea's /metrics endpoint") and drops the inventory-path / lookup-
mechanism text that previously tripped the lint until the Jinja-only
filter went in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New lint
--------
``tests/lint/ansible/modules/test_compose_repository_url_ref.py`` walks
every dict in every project ``.yml``/``.yaml`` and asserts:
1. ``compose_repository_url`` MUST be a Jinja expression — keeps repo
coordinates in ``meta/services.yml`` (looked up via vars) instead
of pasted into task files.
2. ``compose_repository_ref`` MUST also be a Jinja expression for the
same SPOT reason.
3. Every ``compose_repository_url`` MUST be paired with a
``compose_repository_ref`` in the same dict — pinning the clone
source without pinning the ref leaves checkouts at upstream HEAD
(non-reproducible deploys).
Violations fixed
----------------
The lint surfaced three offenders, all migrated to the established
``repository:``/``ref:`` SPOT pattern under the role's primary entity
in ``meta/services.yml`` (akaunting/bigbluebutton/roulette-wheel
already use this shape):
- **web-app-taiga**: added ``repository:`` + ``ref:`` to the ``taiga``
service block; tasks now use ``TAIGA_REPOSITORY`` / ``TAIGA_REF``
vars; restores the previously missing ref pinning.
- **web-app-openproject**: added ``repository:`` + ``ref:`` to the
``openproject`` service block (the ref keeps the
``stable/{{ OPENPROJECT_VERSION }}`` Jinja form, just SPOT-relocated).
- **web-app-littlejs**: added ``repository:`` + ``ref:`` to the
``littlejs`` service block; the existing
``LITTLEJS_REPOSITORY_BASE`` (used by ``nav_bottom.html.j2`` for the
human-readable upstream link) is now derived from the new
``LITTLEJS_REPOSITORY`` via ``regex_replace('\\.git$', '')`` so the
literal hostname is gone too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move `rm -rf /var/lib/apt/lists/*` before `apt-get update` in every
Debian-based Dockerfile so a stale base-image-baked InRelease
(Valid-Until expired) cannot abort the build layer with exit 100. The
trailing rm at end-of-RUN never ran because `apt-get update` aborted
first; the leading rm forces a fresh fetch from the configured source.
Affected: moodle, bridgy-fed, bookwyrm (2 RUN), postmarks, taiga
backend, svc-db-openldap, svc-db-postgres, jenkins, joomla, mediawiki,
odoo, suitecrm (2 RUN), wordpress.
Add tests/lint/docker/dockerfile/test_apt_update_purge_lists.py to
enforce the contract repo-wide: every `RUN` containing `apt-get
update` MUST `rm -rf /var/lib/apt/lists/*` first in the same RUN, and
MUST NOT pass `Acquire::Check-Valid-Until=false` (which would silence
apt's replay-protection rather than refresh the list).
Rename roles/web-app-taiga/templates/Dockerfile.{back,front}.j2 to
Dockerfile.{backend,frontend}.j2 and update the matching tasks/
main.yml template refs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bespoke `pixelfed oidc login (biber)` test always timed out at the
1200s Ansible-task wrapper because pixelfed's first-time OIDC user-
creation runs Laravel's `email:strict,filter_unicode,dns,spoof`
validator. Egulias `DNSCheckValidation` hard-rejects any TLD in its
RESERVED_DNS_TOP_LEVEL_NAMES list (`test`, `example`, `invalid`, …)
BEFORE any DNS lookup, so `biber@infinito.example` always fails and
Laravel auto-redirects `back()` to the Keycloak Referer — breaking the
SSO callback chain and never returning to pixelfed.
The fix mutates biber's Keycloak profile to `biber@infinito.nexus` for
the duration of this spec (real TLD, real MX) and restores the original
afterwards via the Keycloak admin API. Verified empirically against
the live stack: `pixelfed oidc login (biber)` was 1200s timeout, is
now 20s pass; the `afterAll` restorer puts `biber@infinito.example`
+ `emailVerified=true` back cleanly.
Also splits the spec into one file per test so each scenario stays
atomar and individually inspectable:
- `playwright.spec.js` orchestrator: `test.use` + lifecycle + `require()`
- `_shared.js` env, locator candidates, login/logout flow,
keycloak admin helpers, `beforeAll`/`afterAll`
- `test-oidc-login-biber.js`
- `test-oidc-login-administrator.js`
- `test-native-login-administrator.js` new — exercises pixelfed's
stock email+password `/login` form (no infinito-specific selectors),
gated to run only when OIDC is disabled
- `test-guest-persona.js`
- `test-biber-persona.js`
- `test-administrator-persona.js`
The deploy's existing `with_fileglob: "*.js"` copies the companions
into `tests/`; Playwright's testMatch only discovers `playwright.spec.js`
so the `test-*.js` modules are pure `require()` targets, not auto-
discovered specs.
Updates two lints to scan the whole `files/playwright/` directory so
both monolithic and split layouts pass:
- `tests/lint/ansible/roles/web-app/playwright/persona/test_naming.py`
aggregates persona titles across all sibling `.js` files.
- `tests/integration/roles/playwright/test_spec_env_gates.py`
globs `*.js` for service-gate helper calls instead of reading only
the main spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `utils/github/playwright_summary.py` parses every
`playwright-junit.xml` under a given artefact directory and prints a
markdown table to stdout with one row per test case: App, Test,
Status (🟢 passed / 🔴 failed / 🔵 skipped), Duration (sub-second
precision), Message. Sub-second formatting keeps useful per-test
signal that integer rounding would erase. Edge cases (missing dir,
no junit files, all-unparseable) emit a friendly placeholder note
instead of crashing. Covered by 26 unit tests in
`tests/unit/utils/github/test_playwright_summary.py`.
- `test-deploy-server.yml` runs the script with `if: always()` after
the deploy step and pipes its output into `$GITHUB_STEP_SUMMARY`, so
every matrix job's run page surfaces a per-app test table without
needing to download artefacts.
- Playwright config adds the `github` reporter so failing tests also
emit inline `::error file=...,line=...::`-annotations directly on the
workflow run page and in PR diffs.
- New lint `tests/lint/filesystem/test_plugin_paths_files.py` enforces
that files under the Ansible plugin source directories declared in
`ansible.cfg` (`action_plugins`, `filter_plugins`, `lookup_plugins`,
`library`, `module_utils`) are Python source files (README.md
exempt). Paths are read live from `ansible.cfg` so a future config
edit automatically extends the lint's coverage. Counterpart to the
existing `test_scripts_folder_files.py` lint that enforces
`scripts/` contains only `.sh`.
Known limitation (pre-existing pipeline behaviour, not introduced by
this change): when a role runs through multiple matrix variants, the
container path `/var/lib/infinito/logs/test-e2e-playwright/<app_id>/`
has no variant component, so each variant overwrites the previous
variant's reports. The summary therefore reflects only the LAST
variant per role. A future change to namespace the reports per
variant (or copy them after each variant pass) will make the table
cover every variant without touching the script or workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Matcher in web-app-prometheus's parametrized scrape test now also
reads `labels.app`. Native scrape jobs (set `app="web-app-<id>"` via
each role's `templates/prometheus.yml.j2` fragment) now match alongside
blackbox-healthz targets. A single iteration asserts up=1 for both
surfaces per consumer, so per-app native tests become redundant.
- Scenario I (`/metricz`) iterates `prometheusTargetRoles` and asserts
each consumer's `app="<role_id>"` label is present in the
lua-resty-prometheus output. Subsumes the per-app metricz-label spot
checks.
- Remove four redundant per-app prometheus tests from web-app-gitea and
web-app-mattermost playwright specs (metricz-label + native-scrape
up=1), drop the orphaned `PROMETHEUS_BASE_URL=` declarations from
their env.j2 files, and clean up `OIDC_ISSUER_URL=` in gitea's env.j2
(only used by the removed scenarios).
- Drop the dead `const prometheusBaseUrl` declarations plus their
eslint-disable shut-up comments from web-app-fider, web-app-odoo, and
web-app-taiga playwright specs, along with the matching
`PROMETHEUS_BASE_URL=` blocks in those roles' env.j2.
- Correct the persona-required-envs lint: only `MATOMO_BASE_URL` is
actually consumed (by `personas/utils/csp.js`). `DASHBOARD_BASE_URL`
and `PROMETHEUS_BASE_URL` were aspirationally required by a lint
whose docstring described persona-collapse behaviour that no persona
implements. Update `_REQUIRED_URLS` and the docstring, and drop the
unused `\!dashboardBaseUrl` token from `_ALLOWED_SKIP_GUARDS` in
test_strict_mode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New bidirectional contract test ensures `services.prometheus.native_metrics.enabled`
and `roles/<role>/templates/prometheus.yml.j2` stay consistent across the roles tree,
replacing per-role spot checks for gitea/mattermost/matrix. Document the `native-metrics`
suppression rule and apply it to web-app-mattermost (prepared-but-default-off fragment).
- Move every test in `tests/integration/meta/config/` into the per-domain layout under
`tests/integration/roles/meta/{services,server}`. Group prometheus tests under
`services/prometheus/`, repositories tests under `services/repositories/`, the
`server.csp` shape test under `server/`. Move `test_non_empty.py` one level up
(scans all meta/*.yml, not just services.yml).
- Delete `tests/integration/meta/spots/` — both files were only `assertIn` over file
contents (brittle, no behavioural coverage, knowledge belongs in code comments or
proper integration tests).
- Harden `test_project_root_import.py` to catch chained `Path(__file__)…parent.parent.…`
walks (the attribute-access twin of `parents[N]` that previously slipped through).
Refactor 6 surfaced sites to import canonical `PROJECT_ROOT` / `ROLES_DIR`, plus
delete the now-redundant `_find_roles_dir` / `_find_project_root` helpers in
`native_metrics_apps` lookup and `invokable_paths` filter plugin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The first reuse-kept deploy after introducing `database_query` failed
with `couldn't resolve module/action 'database_query'`: ansible only
searches for custom modules along the `library` config path, and the
module had landed under `plugins/modules/` (collection-style layout)
without that path being registered in ansible.cfg.
Realign with the repo's existing convention — `library/cert_check_exists.py`
and `library/docker_bridge_gateway.py` are already there — by relocating
the module to `library/database_query.py`. Make the search path explicit
in ansible.cfg (`library = ./library`) instead of relying on ansible's
implicit playbook-adjacent discovery, so the resolver stays predictable
across CLI / playbook / role-include entry points.
Move the unit suite into `tests/unit/library/` alongside the moved
module, fix the doc-string reference in the lint guard that points
callers at the canonical module location, and drop the now-empty
`tests/unit/plugins/modules/` package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces controller-side `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` calls with a project-local `database_query`
Ansible module that wraps `container exec -i <container> <psql|mysql>`
and pipes SQL via stdin. The module routes via the DB container's own
network so the same task works identically in shared mode (central
`svc-db-*` container) and non-shared mode (per-role `<entity>-database`
container) — the `container` field of `lookup('database', …)` is
shared-aware and the module just reads it. SQL payloads come from a
file (`query_file:`, preferred) or as a raw string (`query:`, escape
hatch). `named_args:` substitutes `%(name)s` placeholders with engine-
appropriate escaping (postgres single-quote-doubling, mariadb backslash
escapes), so role tasks stop string-interpolating Jinja into SQL.
New backends extend the module's `_ENGINES` dispatch table; everything
else is engine-agnostic. The module ships at `plugins/modules/` (Ansible
collection-style layout) — the shebang lint test learned to recognise
that path as a valid module location alongside the existing per-role
`library/` layout.
Two lint tests enforce the convention:
* `tests/lint/ansible/modules/test_no_raw_db_query_module_usage.py`
forbids `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` in role tasks. Opt out per-line with
`# nocheck: database-query` (used by `svc-db-postgres/tasks/02_init.yml`
where the role manages its own DB and the `database` lookup would be
self-referential).
* `tests/lint/ansible/modules/test_no_raw_database_query_string.py`
forbids inline `query:` on `database_query:` tasks. Opt out with
`# nocheck: database-query-raw` (no opt-outs in the current tree —
every SQL payload reaches the module through a `.sql` file under
`files/sql/`).
Role migrations:
* `web-app-peertube` — 3 tasks migrated, 3 new `.sql` files,
`templates/upsert-oidc-plugin.sql.j2` deleted in favour of
`files/sql/upsert-oidc-plugin.sql` + `named_args:`.
* `web-app-openproject` — 3 tasks migrated, 3 new `.sql` files for
the LDAP-auth-source check/update/insert chain.
* `web-app-odoo` — 33 tasks migrated, 9 new `.sql` files for the
LDAP / admin / init-database flows; 17 pre-existing OIDC `.sql`
files renamed snake_case → kebab-case and relocated from
`files/oidc/` to `files/sql/oidc/` (via `git mv` so rename history
stays intact).
Unit tests cover every dispatch-table entry, the named_args escaper
(strings, ints, floats, bools, None, single-quote and backslash
injection vectors), and the password-redactor that hides
PGPASSWORD/MYSQL_PWD assignments from the module's returned `cmd`
field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `test_semver_pinning.py` under `tests/lint/ansible/services/`.
Recursively walks every `roles/*/meta/services.yml`, surfaces a
warning for every `image:`+`version:` pair and every `repository:`+`ref:`
pair whose value is not semver-compatible. Output via the shared
annotations helper (GitHub Actions `::warning::` plus pytest's
`warnings.warn`), with a single trailing suppression hint when any
finding fires. Never fails the test session.
Suppression reuses the existing markers:
- `# nocheck: docker-version` for `version:` pins
- `# nocheck: repository-version` for `ref:` pins
Both accept same-line or line-above placement. The existing external
update checks (`utils/update/docker.suppressed_services` and
`utils/update/repository.suppressed_ref_lines`) are loosened from
strict `line-above` to the default `same-or-above` for consistent
marker semantics across all three consumers.
Updates `docs/contributing/actions/testing/suppression.md` so the
catalog lists both consumers per marker and adds the previously
missing `repository-version` row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the bespoke ``_emit`` that hand-rolled ``::warning file=...::``
directives written to ``sys.__stderr__`` and switches to the shared
``utils.annotations.message.warning`` helper so the PR-side annotation
shape matches every other lint:
::warning title=Deprecated pkgmgr role include,file=X,line=Y::body
Before, GitHub rendered the annotation with no title column and a body
that duplicated the location prefix (``X:Y: deprecated 'pkgmgr' ...
Line: name: pkgmgr-install``). After, the title column carries the
rule name, ``file=`` / ``line=`` carry the location, and the body
contains just the message plus the offending snippet in parentheses.
The local pytest warnings-summary line keeps its ``path:line: body``
shape via ``warnings.warn`` so terminal output stays scannable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Modules imported by tests run inside the same pytest process as the
tests themselves. Raw ``path.read_text(...)`` calls in those modules
bypass the cache the tests just populated. ``roles/`` stays excluded
(deployed scripts have no ``utils/`` on PYTHONPATH); ``utils/cache/``
stays excluded (cache implementation itself).
Five conversions to ``utils.cache.files.read_text``:
- ``utils/handler/yaml.py`` (YAML loader, read-only)
- ``utils/annotations/summarize.py`` (CI log scanner, with
``UnicodeDecodeError`` fallback replacing ``errors="replace"``)
- ``cli/meta/roles/applications/resolution/combined/__main__.py``
(single-shot CLI JSON input)
- ``cli/core/help.py`` (read_folder_title + read_folder_description;
same README is read many times across CLI help iteration)
Eight ``# nocheck: cache-read`` markers, each annotated inline:
- ``utils/claude/settings_sort.py`` (read then rewrite sorted)
- ``utils/docker/version_updater.py`` (read then rewrite config)
- ``cli/administration/deploy/development/network.py`` (tests mock
``Path.read_text`` directly; caching would bypass the mock)
- ``cli/administration/deploy/development/log_utils.py`` (append flow)
- ``cli/contributing/changelog/archive/changelog_md.py``,
``__main__.py``, ``package_mirror.py`` (file mutated in same flow)
Two callers refactored to variable extraction so ruff's 88-char wrap
keeps the ``# nocheck`` marker on the ``read_text(...)`` AST span line
(``cli/.../network.py`` and ``package_mirror.py:105``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Widens the cache-read lint guard from ``tests/lint/`` to every ``.py``
under ``tests/``. Integration tests show the same compounding cost
pattern (one role's compose template / nginx conf / scrape fragment
read across many shape tests) that motivated the original lint for
lint/.
Converts 91 violations across 28 test files:
- **Cache (utils.cache.files.read_text)** for project-tracked reads
and single-read tempdir fixtures, e.g. test_prometheus.py (32x),
test_keycloak_import_flow.py (11x), test_dashboard_oidc.py (3x),
test_changelog_md.py (8x), test_package_mirror.py (8x), plus
test_logging.py / test_summarize.py / test_version_updater.py /
test_get_url_retry.py / test_pull_specific_host.py.
- **Cache + UnicodeDecodeError handler** for project-walk scanners
that previously used ``errors="replace"`` / ``errors="ignore"`` to
defend against non-UTF-8 files (compose volumes, password quoting,
template image, mailu dependency, URL reachability). Wraps each
read in ``try/except (OSError, UnicodeDecodeError): continue``,
which has the same observable effect (skip undecodable files).
- **Refactor away the binary heuristic** in test_no_raw_docker.py:
``read_bytes() + b"\x00" not in raw`` becomes
``read_text() / except UnicodeDecodeError``. Same files skipped,
with cache benefit.
- **Variable extraction** in test_changelog_md.py / test_package_mirror.py
for cases where ``self.assertEqual(path.read_text(...), expected)``
exceeded ruff's 88-char limit. Ruff was wrapping the marker onto a
line outside the AST span of the read_text call.
- **Compute-from-setup** for two test methods that previously read
``self.changelog`` twice (before/after) to verify a no-op:
``before`` is now derived from the known input, dropping the first
read.
Five ``# nocheck: cache-read`` markers remain, each annotated inline
with the reason: idempotency tests where the second read must hit
disk to catch non-idempotent regressions, and tests with intermediate
``write_text`` whose subsequent read would return stale cache data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Counterpart to `test_env_keys_used.py` — that one catches dead env
declarations (env declared, spec never reads), this one catches the
inverse failure mode (spec reads, env never declares).
The kix `DASHBOARD_BASE_URL` regression from CI run 25868178119 made
every persona test crash on the preamble assertion
`expect(dashboardBaseUrl, "DASHBOARD_BASE_URL must be set").toBeTruthy()`.
The existing env→spec lint didn't catch it because the failure was
in the opposite direction. This test scans `roles/<role>/files/playwright.spec.js`,
extracts every `process.env.X` / `process.env["X"]` / `readEnv("X")`
reference, and fails loudly if X is not declared in the sibling
`roles/<role>/templates/playwright.env.j2`.
Scope is narrowed to ONLY `playwright.spec.js` — other JS files
under `<role>/files/` (e.g. `login-broker/server.js` in web-app-bluesky)
are runtime app code whose `process.env` reads come from compose env,
not the test contract.
Excludes Node.js / Playwright / test-runner runtime names
(`NODE_ENV`, `NODE_TLS_REJECT_UNAUTHORIZED`, `PLAYWRIGHT_KEEP_ALL`,
`PWDEBUG`, etc.) via `_RUNTIME_ENV_NAMES`.
Verified: passes against the current 44 specs that ship an env
template, fails with a precise diagnostic when `DASHBOARD_BASE_URL`
is removed from `roles/web-app-kix/templates/playwright.env.j2`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `tests/lint/repository/test_pkgmgr_deprecated.py` and the matching
`pkgmgr-deprecated` rule in the suppression catalog. The check never
fails the suite; each unmarked `name: pkgmgr` / `name: pkgmgr-install`
line is surfaced as a `DeprecationWarning` locally and as a GitHub
Actions `::warning file=…,line=…::…` annotation in CI. The warning
message points at the available pip migrations (`kpmx`, `docoli`,
`git+infinito-nexus-core`). Per-line opt-out via
`# nocheck: pkgmgr-deprecated`.
Also compacts the suppression catalog page: drops the examples
section, replaces prose-style bullet lists with tables, shortens
catalog `Effect` cells, and aligns with `documentation.md` (no em
dashes, no soft-wraps inside sentences, link texts use filenames
instead of full paths).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tightens tests/lint/ansible/roles/test_compose_resource_limits in two
steps. First, the iteration scope is reduced from "every role with a
meta/services.yml" to "every role whose folder name starts with an
invokable-category prefix" (resolved via plugins.filter.invokable_paths.
get_invokable_paths). Non-invokable categories like sys-* never ship a
top-level compose service of their own, so leaving them out removes a
class of structural noise. Second, the warn-only sentinel
(`assertGreater(len(findings), 0)`) is replaced with a hard failure
that lists every missing key — a missing limit is now a build break.
Declares min_storage / cpus / mem_reservation / mem_limit / pids_limit
on the 48 invokable primary entities that previously lacked one or more
of the keys. Container roles (svc-db-{memcached,redis},
web-app-{bigbluebutton,matrix,oauth2-proxy}) get sizing tiers matched
to their workload: BBB heavy (4 cpus / 6g / 4096 pids), Matrix mid-high
(1.5 cpus / 2g / 2048 pids), redis / memcached / oauth2-proxy small.
Native-on-host roles (desk-*, drv-*, gen-*, svc-bkp-*, svc-net-*,
svc-opt-*, web-opt-rdr-*, web-svc-legal, web-app-socialhome) carry
approximate footprints of the underlying package(s); the keys do not
bind a container at runtime (no compose template includes them) but
document expected host consumption and satisfy the strict lint.
Pre-existing curated min_storage values stay (BBB's 50GB recording
budget, redis 1GB, oauth2-proxy 1GB, drv-intel 0GB, …) — only
placeholder shapes ("0GB" / "1GB" with quotes) get overwritten.
Restructures tests/lint/ansible/ into five concern-keyed subdirectories
(jinja/, modules/, services/, shell/, sql/) plus the pre-existing roles/
and variables/. Each new subdir ships its own __init__.py exporting
PROJECT_ROOT and a scope-focused README.md following the project's
documentation conventions (RFC 2119 keywords, emoji-after-headline,
one-sentence-per-line, no historical context).
12 existing tests move into their concern bucket via git mv so history
follows the file. Adds tests/lint/ansible/variables/test_dotted_path_keys_exist.py
which catches the bug class where a Jinja dotted path's head resolves
via group_vars/all/ or the consuming role's own vars/defaults but a
deeper segment does not exist as a key in the dict tree (e.g. the
previous `LDAP.SERVER.REACH_DOMAIN` typo in web-app-odoo).
Touches three external references so cross-links keep resolving:
- docs/contributing/actions/testing/suppression.md (run-once, shared)
- roles/sys-svc-compose/tasks/utils/network/README.md (network-via-util)
- tests/lint/ansible/sql/test_no_inline_multiline_sql.py (self-path exclude)
Adds tests/lint/ansible/test_no_jinja_default_on_spot_path.py: scans every
Jinja `{{ ... | default(...) }}` block under roles/, tasks/, group_vars/,
identifies the actual default target via a paren-aware left-walk from the
matched `|`, and fails when the target's dotted path resolves cleanly
against any group_vars/all/*.yml dict tree. Role vars/defaults are
intentionally out of scope (Ansible loads them only when the owning role
is in the play, so cross-role defaulting is a legitimate caller guard,
not SPOT decoupling). Per-line opt-out: `# nocheck: spot-default`.
Fixes the 21 violations the test surfaces:
- group_vars/all/00_general.yml: drop `| default('')` on CAPTCHA self-checks
- group_vars/all/02_tls.yml: drop `| default(...)` on DOMAIN_PRIMARY in CA subject
- roles/web-app-{listmonk,keycloak,espocrm}/meta/schema.yml: drop defaults on CAPTCHA.*
- roles/web-app-suitecrm/templates/env.j2: map empty SECURITY -> 'none' via `or`,
not via the default filter (SPOT value `""` is always set)
- roles/web-app-opentalk/tasks/main.yml: drop default on MODE_ASSERT (always bool)
- roles/user-workstation/tasks/01_core.yml: drop default on MASK_CREDENTIALS_IN_LOGS
- roles/test-e2e-playwright/tasks/run_one.yml: drop defaults on TLS_MODE + CA_TRUST.*
Also corrects the dotted-path bugs in roles/web-app-odoo/vars/main.yml that
referenced non-existent LDAP keys (`LDAP.SERVER.REACH_DOMAIN`,
`LDAP.USERS.SERVICE_ACCOUNT.*`, `LDAP.ATTRIBUTES.ID`, `LDAP.SERVER.SSL`) to
the canonical paths actually defined in group_vars/all/12_ldap.yml.
Two follow-ups after c7bacbab5 removed `inventory/variants.py`:
* docs/contributing/design/variants.md and docs/requirements/020-*.md
still referenced the deleted submodule, breaking
`test_markdown_relative_links_resolve`. The design doc drops the
removed entries from the reference table; the postmortem turns the
historical mention into plaintext with a pointer to the removal
commit and the current package layout.
* tests/lint/repository/test_markdown_links.py and
test_project_root_import.py walked the worktree via
`iter_project_files`, which does NOT honour `.gitignore`. In the
`make exec` container path the markdown lint's primary
`git ls-files` codepath fails and falls back to that walker, surfacing
19 broken links inside vendored skill READMEs under `.agents/` and
`.claude/skills/` plus 2 `parents[N]` violations in installed skill
scripts. Both lints now use `iter_non_ignored_files` so gitignored
trees (skills install output, build artefacts) are skipped in every
environment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/roles/meta/test_oauth2_contract.py — when a
role's meta/services.yml declares oauth2 with `enabled` not literal
false (Jinja-templated `"{{ 'web-app-keycloak' in group_names }}"`
counts as truthy), the test now requires:
* oauth2.origin.host (non-empty string)
* oauth2.origin.port (non-empty scalar)
* <entity>.ports.local.oauth2 (int)
These three fields are consumed at deploy time by
sys-stk-front-proxy and web-app-oauth2-proxy. Their absence
previously surfaced only as opaque `lookup('config', ...) failed`
errors mid-deploy — exactly the trap that hit web-app-shopware on the
Bundle 2 V0 deploy. web-app-shopware's services.yml is updated to
satisfy the new contract (oauth2: 16503 + origin: { host: web, port:
"8000" }).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/roles/test_compose_no_raw_image.py (strict mode):
every `image:` line in roles/<role>/templates/compose.yml.j2 must
resolve through a Jinja substitution so the underlying registry /
repository / tag is declared in the role's meta/services.yml instead of
being pinned in the compose template.
Fix the 8 pre-existing violations the lint surfaces:
- web-svc-coturn: gateway nginx → services.gateway.{image,version}
- web-svc-simpleicons: local build tag → services.application.{image,version}
- web-app-mailu: clamav antivirus → services.antivirus.{image,version}
- web-app-taiga (5×): events, async-rabbitmq, events-rabbitmq, protected,
gateway → top-level services.<name>.{image,version}
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The env-keys lint previously matched only direct `process.env.<KEY>`
references. Persona helpers under roles/test-e2e-playwright/files/
personas/ read most env keys via a shared `readEnv("KEY")` wrapper, so
keys consumed only through the wrapper were flagged as unreferenced.
Extend the pattern so the wrapper counts as a real consumer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits the network-create helper out of the all-in-one wrapper so roles
can pick the right entry point:
utils/network/create.yml network-only, derives name + subnet from
`network_role_id` (use when the role still
needs to do work between the network create
and the compose-up, or when pre-creating
foreign networks in a loop).
utils/network/routine.yml former wrapper, now delegates to create.yml
and re-flushes sys-svc-compose right after
(use for self-contained single-container
services).
A new lint
`tests/lint/ansible/test_network_create_via_util.py` forbids ad-hoc
`community.docker.docker_network` calls or `docker network create` /
`container network create` shell-outs anywhere in `roles/` outside
`utils/network/create.yml`. The existing 5 callers (mariadb, postgres,
ollama, openldap, web-app-prometheus) move to the new layout and shed
the now-redundant `docker_network_name` / `docker_network_subnet`
caller-side vars; that retires 5 unused role constants
(MARIADB_NETWORK / MARIADB_SUBNET / POSTGRES_SUBNET / OLLAMA_NETWORK /
OPENLDAP_NETWORK).
The `web-app-prometheus` task gains a pre-create loop over
`lookup('native_metrics_apps')` so the deploy unblocks the bootstrap
deadlock observed on test-deploy-server CI run 25634428082 for
web-app-gitea and web-app-matrix (`network <X> declared as external,
but could not be found`): prometheus is loaded as a shared service
*before* the target role, so its `external: true` references to the
target's network needed pre-creation to satisfy compose-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.
Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.
Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops `MAX_LINES` from 12 to 11 in `test_inline_script_block_size.py`,
catching the five blocks that were sitting exactly on the previous
threshold. Pure-bash blocks move to dedicated `.sh` files invoked via
`ansible.builtin.script:` or `run: bash …`; the keycloak heredoc with a
templated JSON payload splits into a `set_fact` plus a folded
`printf | kcadm … -f -` one-liner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the three persona-contract lints from
`tests/lint/ansible/roles/web-app/playwright/test_persona_*.py` into
their own `persona/` subdirectory and drop the now-redundant
`_persona` filename prefix:
* `test_persona_naming.py` -> `persona/test_naming.py`
* `test_persona_required_envs.py` -> `persona/test_required_envs.py`
* `test_persona_strict_mode.py` -> `persona/test_strict_mode.py`
The new `persona/__init__.py` re-anchors `PROJECT_ROOT` one level
deeper (`parents[7]`). Doc references in requirements 017, 018, and
019 now point at the new paths.
Renames are tracked via `git mv`, so commit history is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/roles/web-app/playwright/test_persona_naming.py
to fail the build whenever a `web-app-*/files/playwright.spec.js`
omits one of the three persona scenarios required by the SPOT
contract in docs/contributing/artefact/files/role/playwright.specs.js.md
without an explicit suppression.
Two suppression channels are recognised, both already documented in
the SPOT contract:
- PERSONA_<PERSONA>_BLOCKED=true rendered in the role's
templates/playwright.env.j2 (req 019 Rule 11 explicit role-
contract opt-out).
- Auth-less collapse exception: roles that ship no
templates/playwright.env.j2 are not orchestrated by
test-e2e-playwright and the persona contract does not apply.
Without one of those, every web-app-* role MUST contain
`test("guest: …")`, `test("biber: …")`, and `test("administrator: …")`.
Reason
------
The previous commit fixed seven roles that were silently violating
this rule. Without an automated guard, the same regression class
returns whenever a future role is added or an existing spec's test
title is renamed away from the `<persona>:` prefix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make `tests/lint/.../test_has_spec.py` raise instead of warn when a
roles/web-app-* role has no `files/playwright.spec.js`, then ship the
27 minimal specs needed to satisfy the new floor.
Each new spec follows `roles/web-app-mig` (TLS + canonical-domain
baseline, HTML content-type, three persona flows) with role-specific
admin-interaction labels seeded from app domain knowledge. The bodies
are intentionally static — `templates/playwright.env.j2` is not
shipped per role yet, so the specs do not run end-to-end on deploy;
detail tightening per role is follow-up work.
Also fix `_body_contains_stub_marker` in `test_no_stub_tests.py`: it
substring-matched `TODO` against `body.upper()`, which flagged
`masTODOn` inside the mastodon spec's body regex. Switched to
word-boundary matching so genuine `// TODO:` markers still trip the
lint while embedded substrings in app names or regex alternations do
not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure ruff reflow on tests/lint/ansible/roles/web-app/playwright/
test_persona_strict_mode.py — removes one E303-flagged blank line
and the RET505-flagged `else:` after an unconditional `return`. No
behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
