Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Generalises the postgres image build to be driven by a declarative
extension→library registry instead of a hardcoded POSTGRES_VECTOR_ENABLED
flag. Same Dockerfile.j2 now serves both the central svc-db-postgres
image (libraries aggregated across all deployed consumers) and per-
consumer images for shared:false postgres consumers whose extensions
need libraries not in the postgis/postgis base (e.g. pgvector).
postgres.yml.j2 switches between image: and build: based on whether
custom libraries are needed. New postgres_libraries lookup filters
base-image extensions out and returns recipes for the rest. Dedicated
task path extracted into dedicated.yml + dedicated_postgres.yml so the
block-with-when size limit stays satisfied and the postgres-type check
happens once.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sys-svc-rdbms now renders an extensions.sql into a per-consumer
.initdb.d/ directory and bind-mounts it into the dedicated postgres
container at /docker-entrypoint-initdb.d/. CREATE EXTENSION runs as
POSTGRES_USER (= bootstrap superuser) on first initdb so non-trusted
extensions succeed. Same declarative source (service_extensions
lookup, services.yml extensions field) drives both shared:true
(02_init.yml loop) and shared:false (initdb.d) paths. database lookup
exposes a new initdb_dir field; service_extensions lookup loses its
shared:true guard (shared:false is now a real path). Refactored
repeated lookups in sys-svc-rdbms tasks into vars/main.yml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The assertion `assertNotIn("{{", result["password"])` flaked when
`strong_password` happened to emit two `{` adjacent in the random
password. The intent is to verify the Jinja template was rendered,
so match a full `{{...}}` block instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- new utils.inventory.bundle_apps resolver aggregates and dedupes role
groups from inventories/bundles/{servers,workstations}/<name>/inventory.yml
- new make targets deploy-bundles (fresh-purged) and redeploy-bundles
(reuse-kept), driven by BUNDLES=... env, wrapped by
scripts/tests/deploy/local/deploy/{bundles,redeploy-bundles}.sh
- unit tests for the resolver under tests/unit/utils/inventory/
- document the new targets in docs/contributing/tools/make.md,
docs/contributing/actions/testing.md, inventories/bundles/README.md
- collapse multi-line Makefile comments to one-liners for consistency
- disable nextcloud bbb plugin auto-enable (commented out)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each consumer (bookwyrm/mobilizon/discourse) now declares its required
postgres extensions in its own meta/services.yml under postgres.extensions
and pins shared: true. svc-db-postgres' 02_init.yml drops the hardcoded
postgis-gate and unconditional vector steps and loops over the new
lookup keyed by database_consumer_id. The lookup fast-fails if a role
declares extensions without pinning shared: true.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Avoid Playwright HTTP 500 race: the all-in-one Baserow container's entrypoint
can still be running Django migrations while the playbook continues. Run
`compose exec … manage.py migrate --noinput` with retries before bootstrapping
the admin so auth_user / core_workspace exist, then restart the service. Inline
the previous helper scripts as YAML tasks (drop migrate_via_compose_env.py
entirely; replace bootstrap_admin_via_compose_env.py with a small Django shell
snippet loaded via lookup('file', …)). Use the `compose` wrapper instead of raw
`docker compose` to keep the engine swappable. Unit-test the bootstrap snippet.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:
- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull → compose_pull_repository
- docker_repository_path → compose_repository_path
``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.
Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.
Breaking change for any external override that still sets the old
names — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auxiliary compose calls inside a role's task list (e.g. taiga's
`compose -f compose-inits.yml run --rm manager migrate` before
flush_handlers) used to crash with
service "<X>" has neither an image nor a build context specified:
invalid compose project
whenever the on-disk compose.ca.override.yml still declared services
from a previous matrix-deploy variant that the freshly-rendered
compose.yml no longer ships. The handler-driven regeneration only
fires at flush_handlers — too late for that call.
Teach the compose wrapper's `detect_compose_files()` to drop a stale
override (services in override that are absent from compose.yml) from
its auto-detected file list and warn on stderr. The next handler run
regenerates the override on disk and subsequent compose calls pick
up the fresh one normally; the skipped call runs without CA injection,
which is harmless for the docker-internal-DB migrations that own
this code path.
Empirically verified against the live taiga stale state (database
service in override, not in compose.yml v2): the patched
/usr/bin/compose drops the override from the merged project, the
migrate task proceeds, and no validation crash occurs. 7 new unit
tests cover the stale-detection helper plus the wrapper's skip/keep
branches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Modern MariaDB images (>=11) dropped the `mysql` symlink, so the
hardcoded `mysql` binary in the mariadb engine spec started failing
with rc=127 ("executable file not found") — visible e.g. in the
Pixelfed users-table check looping through its `until` retries.
Switch `_ENGINES["mariadb"]` to an ordered `binaries=("mariadb",
"mysql")`, try them in sequence, and only fall back when the OCI
runtime reports the exact binary we tried missing — engine-side
rc=127s (SQL errors etc.) are not masked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reason: the lookup only consulted services.ldap.enabled, so even with OIDC
fully disabled it would still return "sociallogin" (or "oidc_login" when
LDAP was on). Nextcloud's services.yml then enables the matching OIDC
plugin, which redirects to Keycloak — but the Keycloak client's
redirectUris correctly excludes apps with oidc.enabled=false, so the
handoff dies with "Invalid parameter: redirect_uri". Caused all 3
nextcloud variant-1 Playwright errors (`nextcloud login`,
`nextcloud talk admin settings`, `biber logs into nextcloud via OIDC`).
The lookup now short-circuits to "" when oidc.enabled is falsy, which
makes all three plugin gates (oidc_login, sociallogin, user_oidc) compare
false and 06_setup_plugin.yml's disable branch take over. Existing
flavor-selection paths (oidc.enabled=true) are unchanged. Adds two
regression tests covering the new short-circuit and updates the
`_apps` test helper to default oidc.enabled=true so existing
flavor-selection tests stay focused on the ldap/flavor axis.
Reason: an unrendered `{{ lookup('env', 'DOMAIN') | default(...) }}` value would
otherwise propagate verbatim into CURRENT_PLAY_DOMAINS / CURRENT_PLAY_REDIRECT_DOMAINS
and end up as an nginx `server_name` — emitting `{{`/`}}` into a vhost makes nginx
fail with `directive "server_name" is not terminated by ";"`, breaking all roles
that share the openresty validation (observed on web-svc-asset/-legal/-file).
The filter now eagerly resolves domain_primary via the existing
`render_ansible_strict` env-lookup fallback, and `_add_default_domain` fails
loudly if any Jinja marker still survives. Adds two regression tests covering
the env-lookup path and the fail-fast guard.
The `image` lookup inferred `role_id` from `role_name` when only
`service_name` was passed. Inference broke lazy re-evaluation: a default
like `HEALTH_CSP_IMAGE: "{{ lookup('image', 'csp-checker', 'ref') }}"`
defined in sys-ctl-hlth-csp got re-evaluated inside sys-service's
systemctl.service.j2 template (via `system_service_tpl_exec_start`),
where `role_name='sys-service'`. The lookup then searched
`roles/sys-service/meta/services.yml` for `csp-checker` and aborted with
`no image mapping found for role 'sys-service' service 'csp-checker'`.
CI run 25976825584 surfaced this on six apps (web-app-{bridgy-fed,chess,
mig,hugo,mini-qr,postmarks}) whose deploy reached sys-svc-webserver-core
outside the service-loader preload context, which is what actually
triggers the sys-ctl-hlth-csp -> sys-service include chain.
Make `role_id` mandatory in the plugin signature, remove `_infer_role_id`,
and update the three callsites that relied on inference
(sys-ctl-hlth-csp, sys-svc-container, test-e2e-playwright) to pass the
owning role id explicitly. A migration guard catches the removed
`(service_name, want)` form and emits a clear error instead of the
confusing "missing services.yml" file-not-found message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Matrix-deploy orchestrator now pushes `VARIANT_INDEX` (0-based round
id) alongside the existing `ASYNC_ENABLED` extra var for both PASS 1
(sync) and PASS 2 (async) runs. Test-e2e-playwright's archive step
consumes both to derive a per-(variant, pass) destination so a
subsequent round cannot overwrite the previous one's reports.
- New `roles/test-e2e-playwright/files/archive_playwright_reports.sh`
moves the playwright-junit.xml / playwright-report/ / test-results/
outputs into `variant-<N>/<sync|async>/` under the role's reports
dir, idempotently (re-running the same tuple replaces rather than
fails). `run_one.yml` invokes the script via `ansible.builtin.script`
so the YAML stays a single 7-line task.
- `VARIANT_INDEX: 0` lives in `defaults/main.yml` to keep the variable
defined for dev / non-matrix invocations. Role defaults are not
considered SPOT, so the `| default(0)` belt-and-suspenders inside
the script invocation stays valid against the spot-default lint.
- Summary script now renders an 8-column table: Time | App | Variant |
Pass | Test | Status | Duration | Message. Per-test ISO timestamps
are derived from the JUnit `<testsuite timestamp>` attribute plus the
cumulative duration of preceding cases; file mtime is the fallback
when the attribute is absent. App / Variant / Pass are extracted
from the artefact path layout produced by the archive script.
- Matrix-deploy unit tests refreshed to expect the new
`extra_ansible_vars` shape; summary unit tests gain coverage for
`_to_iso_z`, `_parse_iso_timestamp`, the four supported path
layouts, and the file-mtime fallback (55 tests total).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `utils/github/playwright_summary.py` parses every
`playwright-junit.xml` under a given artefact directory and prints a
markdown table to stdout with one row per test case: App, Test,
Status (🟢 passed / 🔴 failed / 🔵 skipped), Duration (sub-second
precision), Message. Sub-second formatting keeps useful per-test
signal that integer rounding would erase. Edge cases (missing dir,
no junit files, all-unparseable) emit a friendly placeholder note
instead of crashing. Covered by 26 unit tests in
`tests/unit/utils/github/test_playwright_summary.py`.
- `test-deploy-server.yml` runs the script with `if: always()` after
the deploy step and pipes its output into `$GITHUB_STEP_SUMMARY`, so
every matrix job's run page surfaces a per-app test table without
needing to download artefacts.
- Playwright config adds the `github` reporter so failing tests also
emit inline `::error file=...,line=...::`-annotations directly on the
workflow run page and in PR diffs.
- New lint `tests/lint/filesystem/test_plugin_paths_files.py` enforces
that files under the Ansible plugin source directories declared in
`ansible.cfg` (`action_plugins`, `filter_plugins`, `lookup_plugins`,
`library`, `module_utils`) are Python source files (README.md
exempt). Paths are read live from `ansible.cfg` so a future config
edit automatically extends the lint's coverage. Counterpart to the
existing `test_scripts_folder_files.py` lint that enforces
`scripts/` contains only `.sh`.
Known limitation (pre-existing pipeline behaviour, not introduced by
this change): when a role runs through multiple matrix variants, the
container path `/var/lib/infinito/logs/test-e2e-playwright/<app_id>/`
has no variant component, so each variant overwrites the previous
variant's reports. The summary therefore reflects only the LAST
variant per role. A future change to namespace the reports per
variant (or copy them after each variant pass) will make the table
cover every variant without touching the script or workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New bidirectional contract test ensures `services.prometheus.native_metrics.enabled`
and `roles/<role>/templates/prometheus.yml.j2` stay consistent across the roles tree,
replacing per-role spot checks for gitea/mattermost/matrix. Document the `native-metrics`
suppression rule and apply it to web-app-mattermost (prepared-but-default-off fragment).
- Move every test in `tests/integration/meta/config/` into the per-domain layout under
`tests/integration/roles/meta/{services,server}`. Group prometheus tests under
`services/prometheus/`, repositories tests under `services/repositories/`, the
`server.csp` shape test under `server/`. Move `test_non_empty.py` one level up
(scans all meta/*.yml, not just services.yml).
- Delete `tests/integration/meta/spots/` — both files were only `assertIn` over file
contents (brittle, no behavioural coverage, knowledge belongs in code comments or
proper integration tests).
- Harden `test_project_root_import.py` to catch chained `Path(__file__)…parent.parent.…`
walks (the attribute-access twin of `parents[N]` that previously slipped through).
Refactor 6 surfaced sites to import canonical `PROJECT_ROOT` / `ROLES_DIR`, plus
delete the now-redundant `_find_roles_dir` / `_find_project_root` helpers in
`native_metrics_apps` lookup and `invokable_paths` filter plugin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Decouples the CSS service concept from its underlying transport
(currently `web-svc-cdn`) so the flag becomes opt-in via inventory
group membership instead of staying literal-`true` everywhere.
- `roles/web-svc-css/`: new role. Marker / wrapper role that owns
the `css` entry in the service registry, declares `web-svc-cdn` as
the upstream that serves the actual bytes, and 301-redirects its
own canonical domain to the cdn domain so health probes return a
valid response.
- `roles/web-svc-cdn/meta/services.yml`: drop the `css` block — the
canonical owner is now web-svc-css; cdn keeps `javascript`.
- 51 consumer roles (web-app-* and web-svc-{coturn,file,html,
libretranslate,logout,onlyoffice}) migrated their `css:` block from
literal `{ enabled: true, shared: true }` to dynamic
`{ enabled: "{{ 'web-svc-css' in group_names }}", shared: ... }`,
with matching matrix coverage added to each role's
`meta/variants.yml` (true-pinning + false-pinning entries) so the
dynamic-flag variant lint stays green.
- 41 `# nocheck: playwright-service-flag` rationale strings updated
from the now-incorrect "canonical:cdn alias of static-asset
provider" to "css consumer, no Playwright surface (provider:
web-svc-css)".
Bundled refactor: `roles/web-svc-asset/tasks/01_core.yml` now
includes `web-opt-rdr-domains` directly with a single-entry mapping
to redirect its canonical domain to web-svc-file, instead of having
that wiring live in the global `tasks/stages/01_constructor.yml`
constructor. The `add_redirect_if_group` filter
(`plugins/filter/redirect_filters.py`) and its unit test are dropped
as dead code — both call sites are now role-local.
Dashboard's consumption of the new pattern is intentionally not part
of this commit and follows separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- plugins/lookup/status.py: `list(str)` on `APPLICATIONS_WHITELIST` (CSV from `-e`) split it into chars, breaking every consumer of `.whitelist` / `.running` (web-app-roles when-clause skipped all apps, keycloak per-app LDAP mappers never created).
- test-e2e-playwright/discover_playwright_roles: tolerate Python list-repr strings from non-native-Jinja2 stringification of `"{{ lookup('status').groups }}"` — otherwise `TEST_E2E_PLAYWRIGHT_APPS` resolves to `[]` and no spec runs.
- Unit tests cover both regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- plugins/lookup/roles_with_service.py: resolve url.base via the tls
sublookup and return canonical_url alongside id/canonical_domain.
- web-app-matomo/files/playwright/playwright.spec.js: parameterised
matomo-tracker injection assertion per consumer over
MATOMO_TARGET_ROLES_JSON, using target.canonical_url.
- web-svc-logout: new files/playwright/playwright.spec.js + templates/
playwright.env.j2; checks universal-logout injection on every consumer
declared in LOGOUT_TARGET_ROLES_JSON. meta/services.yml gains nocheck
markers explaining why this provider has no user-facing surface to
assert on.
- tests/unit/plugins/lookup/test_roles_with_service.py: cover the new
canonical_url field via a stubbed tls sublookup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`container inspect mariadb` is polymorphic and matched the same-named
external Docker network (created by sys-svc-compose for shared DBs)
instead of the container, producing "map has no entry for key 'State'"
template errors on every retry until the wait task timed out.
- plugins/lookup/database.py: add `network` field (dbtype when shared,
consumer entity otherwise) so roles get the DB network for both modes.
- roles/web-app-mediawiki/tasks/main.yml: switch to typesafe
`container container inspect` against the new MEDIAWIKI_DB_CONTAINER.
- roles/web-app-mediawiki/vars/main.yml: derive MEDIAWIKI_NETWORK_DATABASE
and MEDIAWIKI_DB_CONTAINER from lookup('database', ...).
- tests: cover the new `network` field for empty/local/shared cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The first reuse-kept deploy after introducing `database_query` failed
with `couldn't resolve module/action 'database_query'`: ansible only
searches for custom modules along the `library` config path, and the
module had landed under `plugins/modules/` (collection-style layout)
without that path being registered in ansible.cfg.
Realign with the repo's existing convention — `library/cert_check_exists.py`
and `library/docker_bridge_gateway.py` are already there — by relocating
the module to `library/database_query.py`. Make the search path explicit
in ansible.cfg (`library = ./library`) instead of relying on ansible's
implicit playbook-adjacent discovery, so the resolver stays predictable
across CLI / playbook / role-include entry points.
Move the unit suite into `tests/unit/library/` alongside the moved
module, fix the doc-string reference in the lint guard that points
callers at the canonical module location, and drop the now-empty
`tests/unit/plugins/modules/` package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces controller-side `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` calls with a project-local `database_query`
Ansible module that wraps `container exec -i <container> <psql|mysql>`
and pipes SQL via stdin. The module routes via the DB container's own
network so the same task works identically in shared mode (central
`svc-db-*` container) and non-shared mode (per-role `<entity>-database`
container) — the `container` field of `lookup('database', …)` is
shared-aware and the module just reads it. SQL payloads come from a
file (`query_file:`, preferred) or as a raw string (`query:`, escape
hatch). `named_args:` substitutes `%(name)s` placeholders with engine-
appropriate escaping (postgres single-quote-doubling, mariadb backslash
escapes), so role tasks stop string-interpolating Jinja into SQL.
New backends extend the module's `_ENGINES` dispatch table; everything
else is engine-agnostic. The module ships at `plugins/modules/` (Ansible
collection-style layout) — the shebang lint test learned to recognise
that path as a valid module location alongside the existing per-role
`library/` layout.
Two lint tests enforce the convention:
* `tests/lint/ansible/modules/test_no_raw_db_query_module_usage.py`
forbids `community.postgresql.postgresql_query` /
`community.mysql.mysql_query` in role tasks. Opt out per-line with
`# nocheck: database-query` (used by `svc-db-postgres/tasks/02_init.yml`
where the role manages its own DB and the `database` lookup would be
self-referential).
* `tests/lint/ansible/modules/test_no_raw_database_query_string.py`
forbids inline `query:` on `database_query:` tasks. Opt out with
`# nocheck: database-query-raw` (no opt-outs in the current tree —
every SQL payload reaches the module through a `.sql` file under
`files/sql/`).
Role migrations:
* `web-app-peertube` — 3 tasks migrated, 3 new `.sql` files,
`templates/upsert-oidc-plugin.sql.j2` deleted in favour of
`files/sql/upsert-oidc-plugin.sql` + `named_args:`.
* `web-app-openproject` — 3 tasks migrated, 3 new `.sql` files for
the LDAP-auth-source check/update/insert chain.
* `web-app-odoo` — 33 tasks migrated, 9 new `.sql` files for the
LDAP / admin / init-database flows; 17 pre-existing OIDC `.sql`
files renamed snake_case → kebab-case and relocated from
`files/oidc/` to `files/sql/oidc/` (via `git mv` so rename history
stays intact).
Unit tests cover every dispatch-table entry, the named_args escaper
(strings, ints, floats, bools, None, single-quote and backslash
injection vectors), and the password-redactor that hides
PGPASSWORD/MYSQL_PWD assignments from the module's returned `cmd`
field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Widens the cache-read lint guard from ``tests/lint/`` to every ``.py``
under ``tests/``. Integration tests show the same compounding cost
pattern (one role's compose template / nginx conf / scrape fragment
read across many shape tests) that motivated the original lint for
lint/.
Converts 91 violations across 28 test files:
- **Cache (utils.cache.files.read_text)** for project-tracked reads
and single-read tempdir fixtures, e.g. test_prometheus.py (32x),
test_keycloak_import_flow.py (11x), test_dashboard_oidc.py (3x),
test_changelog_md.py (8x), test_package_mirror.py (8x), plus
test_logging.py / test_summarize.py / test_version_updater.py /
test_get_url_retry.py / test_pull_specific_host.py.
- **Cache + UnicodeDecodeError handler** for project-walk scanners
that previously used ``errors="replace"`` / ``errors="ignore"`` to
defend against non-UTF-8 files (compose volumes, password quoting,
template image, mailu dependency, URL reachability). Wraps each
read in ``try/except (OSError, UnicodeDecodeError): continue``,
which has the same observable effect (skip undecodable files).
- **Refactor away the binary heuristic** in test_no_raw_docker.py:
``read_bytes() + b"\x00" not in raw`` becomes
``read_text() / except UnicodeDecodeError``. Same files skipped,
with cache benefit.
- **Variable extraction** in test_changelog_md.py / test_package_mirror.py
for cases where ``self.assertEqual(path.read_text(...), expected)``
exceeded ruff's 88-char limit. Ruff was wrapping the marker onto a
line outside the AST span of the read_text call.
- **Compute-from-setup** for two test methods that previously read
``self.changelog`` twice (before/after) to verify a no-op:
``before`` is now derived from the known input, dropping the first
read.
Five ``# nocheck: cache-read`` markers remain, each annotated inline
with the reason: idempotency tests where the second read must hit
disk to catch non-idempotent regressions, and tests with intermediate
``write_text`` whose subsequent read would return stale cache data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`3c1d5d821` solved the cross-round backbone-abriss problem by making
`include` equal the union of every variant's closure for every round.
Side-effect: every matrix round then ALSO deployed the full union, so
`08_deploy_performance.sh` saw `dashboard.infinito.example` still
answering HTTP 200 after a matomo-only `deploy-fresh-purged-apps` —
the deploy step was no longer matomo-only at all.
Split the two concerns instead:
- `include` (PlanEntry[3]) goes back to the per-round variant-closure.
Both `init`/`builder` (`--include` for the inventory bake) and
`deploy` (`--id` for ansible) consume it. Each round therefore
deploys ONLY its own variant-closure again — the pre-`3c1d5d821`
semantics.
- `purge_set` (PlanEntry[4], new) carries the union across all rounds.
`deploy.py` uses it for the inter-round wipe instead of the previous
round's include. Result: between rounds the WHOLE matrix footprint
is purged, not just the previous round's variant slice — a variant
pinning `services.<X>.shared: false` cannot leave its bundled
provider on the host for a sibling round that wires the same service
shared. The backbone-abriss `3c1d5d821` was hunting stays fixed:
whichever round needs the backbone re-deploys it immediately after
the wipe (purge runs at the start of each round, not the end).
`previous_round_include` tracking is gone. The first iteration skips
the purge (nothing to wipe yet), which also covers `total_rounds == 1`
and `--variant N` pinned single-folder modes cleanly.
Unrelated, but staged together because it is part of the same
investigation:
- `roles/dev-nix/templates/nix.conf.j2`: enable `keep-derivations` /
`keep-outputs`, so `nix-collect-garbage -d` (run by the aggressive
prune step) does not drop the build closure of pkgmgr's flake; this
was making every deploy re-fetch the build inputs from
cache.nixos.org and overshoot pkgmgr's update timeout.
- `roles/pkgmgr/tasks/01_core.yml`: raise the pkgmgr update timeout
from 600s to 1800s and lower the retry count from 6 to 3, so a slow
fetch on a single try no longer pushes total wallclock past three
hours.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the no-op `docker restart` recovery with `POST
/service/rest/v1/repositories/{name}/invalidate-cache` against each
apt-format proxy repo (Debian, Ubuntu, …) discovered dynamically
from Nexus' REST API. Container restart never re-validated cached
content; the new path makes Nexus actually re-fetch from upstream
on the next request and keeps repo definitions, blobstore, and
admin credentials intact.
Co-located changes:
- fresh-purged-app.sh: source scripts/meta/env/all.sh so the cache
override env (incl. INFINITO_PACKAGE_CACHE_FRONTEND_IP) is set
for the deploy CLI — without this, apt traffic bypasses Nexus.
- utils/nexus/list_apt_proxy_repos.py: stdin JSON → stdout name list,
exit 1 on malformed input / no matches; used by the bash helper
for discovery (parser lives outside the heredoc for testability).
- tests/unit/utils/nexus/test_list_apt_proxy_repos.py: 10 cases
covering filter rules, empty/invalid input, non-dict entries,
missing/empty names, and a realistic live payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 16f323319 added a strict variant-only include rule + conflict
detector in `inventory/variants.py`, gated behind the presence of any
primary's `meta/variants.yml`. In practice the rule never paid off:
* The CI failure that motivated it (matomo's matrix transition leaving
a stale dashboard nginx vhost -> 502) is already addressed by the
nginx-purge primitive shipped in commit bfd04e4a1.
* The existing `CombinedResolver` path already implements the "any
role says true or `in group_names` -> pull" semantics via
`utils.roles.applications.services.registry.is_explicit_truth`, so
every needed service still gets pulled and services that all roles
mark False stay out of the include.
* The strict literal-equality conflict detector was a false-positive
factory: 79/80 variant-equipped primaries failed it against the
current repo state because roles legitimately disagree on `enabled`
flags for shared services.
This commit removes the entire variant-only branch and restores the
single-path planner. Round-index clamping for non-primary deps is left
exactly as before 16f323319 -- only the additions are reverted.
The file-split refactor from 16f323319 is intentionally kept; only the
`variants.py` submodule, the corresponding `test_variants.py`, and the
variant-only gate test class in `test_planner.py` are removed. The
`__init__.py` re-exports + `__all__` are pruned to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A set_fact run_once: true at role scope in svc-db-postgres/tasks/01_core.yml
did not survive the matrix-deploy round split: rounds whose include set did
not contain svc-db-postgres failed at template-render time with
'POSTGRES_ALLOWED_AVG_CONNECTIONS' is undefined.
- Hoist POSTGRES_VCPUS, POSTGRES_TOTAL_RAM_MB, POSTGRES_MAX_CONNECTIONS,
POSTGRES_ALLOWED_AVG_CONNECTIONS, POSTGRES_SUPERUSER_RESERVED_CONNECTIONS,
POSTGRES_SHARED_BUFFERS*, POSTGRES_WORK_MEM*, POSTGRES_MAINTENANCE_WORK_MEM*,
POSTGRES_DELAY to group_vars/all/17_resource.yml (lazy Jinja evaluation
per round).
- Drop the set_fact from tasks/01_core.yml; the value is now derived
globally on every play.
- Relocate the split_postgres_connections filter plugin from
roles/svc-db-postgres/filter_plugins/ to plugins/filter/ so it is always
loaded via the global filter_plugins = ./plugins/filter declaration in
ansible.cfg.
- Move the matching unit test from
tests/unit/roles/svc-db-postgres/filter_plugins/ to
tests/unit/plugins/filter/ to mirror the plugin's new location.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes ship together because the variant-only matrix-deploy logic
was written into the legacy 697-line inventory.py and then re-split:
* Source package (max 240 lines per submodule, well under the 300 budget):
- spec.py DevInventorySpec + PlanEntry data types
- payload.py _resolve_variant_payloads + _bake_overrides
- legacy_resolver.py CombinedResolver fallback (default-mode path)
- variants.py strict variant-only include resolution
- planner.py plan_dev_inventory_matrix + filter_plan_to_variant
- builder.py build_dev_inventory + build_dev_inventory_matrix
- __init__.py public-API re-exports + back-compat shims
* Test package mirrors the source 1:1 with a shared _fixtures.make_spec
to avoid duplicating the DevInventorySpec defaults across files.
* New variant-only matrix mode (gated on any primary shipping
meta/variants.yml): only services whose merged variant payload pins
both enabled and shared to literal True become inventory groups;
Jinja flags never qualify (group_names does not exist yet at build
time); run_after only orders the include, never expands it;
cross-primary disagreement on a (service-key, round) raises ValueError.
Default mode (no variants.yml on any primary) keeps the legacy
CombinedResolver path untouched.
* Docs in design/{inventory,variants}.md updated to point at the new
per-submodule paths; test_nginx_vhosts.py's Test classes now also
inherit directly from unittest.TestCase so the AST-level lint in
test_all_test_files_have_tests.py picks them up (transitive base via
NginxVhostsTestBase alone is not enough).
Coverage: 458 cli unit tests, 349 utils unit tests, 99 dev-deploy tests,
all green; ruff check clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matrix-deploy variant transitions left per-domain vhost files behind:
when a service dropped out of a round's include set, sys-svc-compose's
docker prune removed its container but the vhost under
<NGINX_DIR>/conf.d/servers/{http,https}/<domain>.conf survived, making
the next round's HTTPS probe see 502 instead of TLS-error 000 (5 failed
test-development jobs on run 25705903504).
The inter-round purge in cli.administration.deploy.development.deploy
already invokes scripts/tests/deploy/local/purge/entity.sh with the
previous round's full include — the missing piece was an entity-keyed
nginx primitive next to db/compose/dir. Add it:
- utils.roles.entity_apps.apps_for_entity: reverse of get_entity_name,
walks roles/*/vars/main.yml to map entity -> [application_id]
- utils.cleanup.nginx_vhosts: resolves entity -> apps -> canonical+alias
domains (via build_applications_from_roles + iter_app_domains so it
stays ansible-free) and removes the matching vhost files
- scripts/container/purge/entity/nginx.sh: thin bash wrapper,
symmetric to db/compose/dir
- scripts/container/purge/apps.sh: drive nginx.sh inside the entity
loop, alongside db/compose/dir
Tests: 14 new unit tests covering the reverse-lookup edge cases and
the purge iterator (existing-only / aliases / multi-entity / no-op CLI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Filter no longer takes a `database_volume=` parameter. The volume name is
derived from the central provider's `services.<dbtype>.name` (shared) or
`<entity>_database` (dedicated) — mirroring `plugins/lookup/database.py`'s
`volume_prefix + host` construction. Simultaneous postgres + mariadb on
the same role now raises an AnsibleFilterError with a clear message;
support is deferred per the docstring TODO.
The integration test (renamed from `test_shared_service_volumes.py` to
`test_compose_volumes_call_required.py`) now requires every
`roles/*/templates/compose.yml.j2` to call `compose_volumes`, so the
top-level `volumes:` block is always rendered. Without it, flipping a
shared service to dedicated mode adds a named-volume reference that
Compose rejects with `service "<name>" refers to undefined volume`.
Adds the call to 14 templates that were missing it (dashboard, hugo,
sphinx, bridgy-fed, fediwall, littlejs, mig, mini-qr, navigator,
roulette-wheel, simpleicons, collabora, logout, openresty) and strips
the `database_volume=...` argument from every other role plus Taiga's
now-unused `TAIGA_DATABASE_VOLUME` var.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`lookup('database', <consumer>, 'image')` previously returned the bare
dbtype (`postgres`), so an embedded dedicated database
(`postgres.shared=false`) rendered `image: postgres:<svc-db-postgres
version>` even when the central provider shipped a different image
(`svc-db-postgres` declares `postgis/postgis`). `compose pull` then
failed with `manifest unknown` because `postgres:17-3.5` does not
exist on Docker Hub. Look the image up from the central provider's
`services.<dbtype>.image` and fall back to the bare dbtype only when
the provider declares none, so dedicated instances ship the same
image as the shared provider.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.
`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four additive changes to `infinito meta roles applications complexity`:
* Render two new columns for the one-hop view — `embeds_direct` /
`consumers_direct` — alongside the existing transitive `embeds` /
`consumers`. The direct fields are independent of `--level`: a
reader can always see what the role embeds / is-embedded-by
without its indirect tail.
* Render a `total` column that sums the four numeric columns. It
serves as a coarse "overall touch points" score; `--sort total`
surfaces the most-coupled roles.
* `--order {asc,desc}` (default `asc`) controls sort direction.
Applies to every `--sort` choice; `--sort total --order desc`
puts the hubs at the top of the table.
* `--filter SUBSTRING` (case-insensitive) keeps only rows whose
role name contains SUBSTRING. The complexity scores are computed
against the full role tree first, then filtering applies to the
rendered rows only — a role's transitive counts still reflect the
whole codebase.
The tuple shape grew by appending the new fields, so existing
index 0..4 assertions in the unit tests stay valid; new
assertions cover the direct-vs-transitive distinction and the
total computation.
Bonus: trailing-newline fix on `roles/web-svc-file/meta/services.yml`
(the autoformat surfaced a missing EOL after the recent dashboard
cleanup).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Surface the reverse direction of the shared-service dependency graph
in `infinito meta roles applications complexity`. Each row now also
counts and lists the application roles that transitively embed *this*
role (the "consumers" / "consumed-by" closure), with the same
`--level` depth cap as the forward direction.
* Forward closure (column `embeds`, list `services`) and reverse
closure (column `consumers`, list `consumed_by`) share a single
BFS over (forward, reverse) adjacency maps built once per
invocation.
* `--sort` accepts `embeds` (default), `consumers`, or `name`. The
former `points` label is renamed to `embeds` because the column
header reads as a sentence with the role name (e.g. `web-app-foo
embeds 12, consumers 78`), which is sharper than a generic
"points" score.
* `--format cli` (default) renders three numeric columns — name,
embeds, consumers — for a compact terminal view.
`--format json` emits the full payload with the resolved role
lists for each direction.
* Tests cover the chain case in both directions and verify that
`--level` caps the reverse-direction BFS the same way.
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.
In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.
The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.
Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caps the BFS over each role's shared-service deps:
- `--level 1` returns direct deps only.
- `--level 2` adds their direct deps.
- Omitting the flag keeps the unbounded full-closure behaviour.
Argparse rejects `--level < 1` with rc=2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For each application role, recursively resolve its shared-service
dependency closure and report (name, points, services-csv).
- BFS over each role's meta/services.yml; provider role per service-key
comes from utils.roles.applications.services.registry.
- Default truth predicate is is_explicit_truth so the
'<role>' in group_names Jinja form counts as a dep; --no-group-names
restricts to literal `true` flags only.
- Sort: --sort points (default, asc, name as tiebreaker) | --sort name (asc).
- Tests cover chain resolution, group-names toggle, self-not-counted,
non-application roles skipped.
Plus: roles/desk-gnome/meta/services.yml -- yaml autoformat drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Package renames (dynamic discovery picks up new names automatically):
- cli/build/role_include/ -> cli/build/include/
- cli/build/roles_list/ -> cli/build/list/
- cli/meta/roles/lifecycle_filter/ -> cli/meta/roles/lifecycle/
Domain logic moved out of cli/ (no CLI surface, just a helper class):
- cli/meta/roles/applications/resolution/services/{resolver,errors}.py
-> utils/roles/applications/services/{resolver,errors}.py
- drop dead services/yaml_utils.py + its only test
Bug fixes:
- cli/meta/{callorder -> roles/callorder} dropped its hard-coded
parents[3] PROJECT_ROOT (broke after the directory move and resolved
to <repo>/cli/); it now reuses utils.PROJECT_ROOT.
Help renderer (cli/core/help.py + colors.py):
- DIM color on the description column to separate it from the command name
- Blank line between the long command name and its wrapped description
- Italic invocation under the category heading + leading blank line
- New Style.ITALIC
Doc updates: path references in docs/requirements/00{8,10}*.md plus
soft-wrap fixes in the touched regions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TC (flake8-type-checking): typing-only imports moved into
``if TYPE_CHECKING:`` blocks across 8 source files via auto-fix
(117 hits). Reduces runtime import surface and rules out circular
imports for type-only deps. Project-wide ``from __future__ import
annotations`` (FA, already enabled) keeps annotations as strings,
so the moved imports never need to load at runtime.
N (pep8-naming): 32 hits handled.
Renames (semantic + lint):
- ``isSmallerThenMaximumBackupSize`` -> ``is_larger_than_maximum_backup_size``
(snake_case + fixes both the typo "Then -> Than" and the
semantic mismatch: the function returns ``current > maximum``)
- ``isDirectoryDeletable`` / ``deleteVersion`` /
``getAmountOfIteration`` / ``deleteIteration`` -> snake_case
- 8 test functions with ALL-CAPS env-var fragments
(``test_..._DOMAIN_PRIMARY_...`` etc.) -> snake_case
- Mock alias ``VH`` -> ``vh`` in test_host_vars.py
Exception class renames (``N818`` Error suffix):
- ``TimeLimitExceededException`` -> ``TimeLimitExceededError``
- ``AttemptException`` -> ``AttemptError``
- ``PathNotFound`` -> ``PathNotFoundError`` (propagated across
4 callers under tests/integration/lookups/config/)
- ``_ExitJson`` / ``_FailJson`` -> ``*_Error``
Refactor:
- ``METADATA = {...}`` local constant in two functions of
``plugins/filter/invokable_paths.py`` lifted to module-level
``_METADATA_KEYS``.
- ``from typing import TYPE_CHECKING`` in
``tasks/utils/migrate_meta_layout/__main__.py`` moved above
the ``sys.path.insert`` hack to satisfy E402 (collateral from
the TC auto-fix that landed it after the path mutation).
Per-file ignores in pyproject.toml for the legitimate cases the
rule cannot accept:
- ``pull-specific-host.py`` -> ``N999`` (Ansible-invoked role-level
script, hyphenated filename pinned by the surrounding stack).
- 4 matrix-deploy files -> ``N806`` for the ``include_R`` /
``raw_include_R`` round-index notation; the uppercase ``R``
here mirrors the formula in the docstring rather than naming
a constant.
- ``plugins/filter/value_generator.py`` -> ``N806`` for the
lazy-loaded class binding ``ValueGenerator =
_value_generator_cls()``, which retains the imported class
identifier per PEP 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cleanup logic between rounds is now full-include rather than
variant-changed-only:
- handler() tracks previous_round_include instead of
previous_round_variants. Between rounds the wrapper purges every
app from the previous round's include set, so round R boots from
an empty host.
- Every round deploys its full round_include (round 0 baseline AND
R>0). The "skip variant-unchanged apps" path is gone, because
variants that drop deps left cross-round residue under the
previous selective-purge model.
- _purge_app_entities docstring updated; the matrix-deploy comment
in handler() reflects the new semantics.
test_matrix_deploy.py:
- test_round_one_only_deploys_apps_with_real_variant_one renamed
test_every_round_redeploys_full_include_after_purge.
- test_three_round_plan_only_re_deploys_apps_with_real_variant
renamed test_three_round_plan_redeploys_full_include_each_round.
- two_round + full_cycle purge assertions updated to expect the
full previous-round include.
- New test_round_transition_purges_previous_include_even_when_dep_falls_out
pins the fall-out case (round 0 = nextcloud + coturn, round 1 =
nextcloud only -> purge MUST include coturn).
variants.md updated to match (points 4 + 5 rewritten, reference
table aligned). documentation.md compliance pass: dropped em-dash
clause separator and the historical optimisation reference on
point 4, lifted reference-table entry into the new wording.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop four one-shot or now-redundant tools from cli/fix/:
* vars_main_files — bulk-created vars/main.yml with application_id
per role-prefix during the SPOT migration. The schema is now
enforced declaratively by tests/integration/roles/applications/id/
test_matches_role_name.py and the role-mapping SPOT lint.
* move_unnecessary_dependencies — moved meta dependencies to guarded
include_role on first sweep. Permanent enforcement lives in
tests/integration/roles/meta/dependencies/test_unnecessary.py.
* tabs — replaced literal tab characters with two-space indentation
in YAML files. Autoformat covers new occurrences.
* ini_py — generic helper that filled missing __init__.py files; new
packages now ship them from the start.
Move the remaining (still-actively-used) docker version updater out
from under cli/fix/ and into cli/contributing/, where the recurring
maintenance tooling lives:
* cli/fix/docker_image_versions → cli/contributing/docker/update_versions
Update the corresponding wiring:
* scripts/github/update_docker_image_versions.sh now calls
python -m cli.contributing.docker.update_versions
(the entry point invoked daily by .github/workflows/update.yml).
* docs/requirements/008-role-meta-layout.md references the new path.
* PROJECT_ROOT in __init__.py drops one parents[] level (the package
sits one folder deeper now).
cli/fix/ is fully removed; new contributor-facing tooling belongs
under cli/contributing/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Break cli/contributing/changelog/archive/__main__.py into four
single-responsibility modules under the same package:
* versioning.py — pad_version / unpad_version / archive_filename
(the canonical <padded-semver>-<date>.md filename schema).
* archive_dir.py — existing_archives, archived_releases_from_directory,
build_index_section (filesystem operations on docs/changelog/).
* changelog_md.py — split_into_entries, md_body_after_header,
trim_and_archive (CHANGELOG.md parsing and trimming, plus the
always-rebuild-index logic).
* package_mirror.py — mirror_to_debian_changelog,
mirror_to_rpm_spec_changelog (Debian and RPM regeneration from
CHANGELOG.md kept entries, with the trailing notice that points at
docs.infinito.nexus and lists archived versions as plain text).
__main__.py shrinks to argparse plus a thin orchestration layer that
parses CHANGELOG.md once, runs trim_and_archive, reads the populated
archive directory, and mirrors into the package files.
Add `mirror_to_debian_changelog` and `mirror_to_rpm_spec_changelog`
content-idempotency: skip the write when the regenerated content
already matches what is on disk so re-runs leave file mtimes
unchanged. Verified end-to-end: a follow-up run on a fully-processed
tree leaves all three files (CHANGELOG.md, Debian changelog, RPM
spec) byte-identical and mtime-stable.
Replace `datetime.strptime` calls with `date.fromisoformat` so the
DTZ007 lint stays clean: we only need calendar-date parsing for
display formatting, not timezone-aware moments.
Tests follow the same split: one test_<module>.py per source module,
40 tests in total. Shared fixtures live in _helpers.py (underscore
prefix exempts it from the test-file-naming lint), and each test
class explicitly mixes TempRepoMixin with unittest.TestCase so the
test-files-have-tests AST lint sees the direct unittest base.
Update docs/contributing/actions/release.md so the procedure
description matches the current behaviour: package changelogs are
"mirrored" (not "trimmed") from CHANGELOG.md, and the idempotence
section calls out byte-idempotency on each of the three files.
Run the CLI once against the live tree to bring packaging/debian/
changelog and packaging/fedora/infinito-nexus.spec back into sync
with CHANGELOG.md and the docs/changelog/ archive directory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
