Commit Graph
29 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 f0ecf3e337 style(update): apply the formatting the gate produced
The nocheck markers were written as long single lines; ruff splits the
call across three lines and leaves the marker behind the closing paren.
Committing the formatter's own output keeps the tree and the history in
step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:51:25 +02:00
kevinveenbirkenbachandClaude Opus 5 94baeb7502 fix(lint): meet the repository rules the collection work broke
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.

File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.

restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 623ecc0248 feat(update): retune the preferred collection source from measured latency
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.

A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.

History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:59:45 +02:00
kevinveenbirkenbachandClaude Opus 5 e992175fcc feat(update): keep the pinned Ansible collections from ageing unnoticed
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.

Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.

The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:58:35 +02:00
kevinveenbirkenbachandClaude Opus 5 5e6884b16d feat(update): keep pinned pip requirements from ageing unnoticed
The version cron updates docker image tags and repository refs, and tests/external/update covers docker, addons and repository. Neither sees a version inside an Ansible task variable, and dependabot reads pyproject.toml rather than roles/. A pin set once therefore froze and nothing thawed it: backup-docker-to-local has not moved since it was written, and the automtu pin one commit back would have gone the same way.

utils/update/pip mirrors the docker module: collect the pins, ask PyPI, rewrite in place. Only exact == pins are collected, because a range or a VCS reference states an intent an updater must not overrule. The external check warns rather than fails, like its docker sibling, so a slow PyPI cannot redden a run and a release the repository has deliberately not taken is not a regression.

The cron job is the docker job with one script swapped, so a changed pin arrives as its own PR under update/pip-pins.

Executed: the collector finds both pins in the repository; a synthetic file rewrites from 1.0.0 to 1.1.1 with the indent, quoting and neighbouring line untouched; automtu==1.0.0 would be reported while a bare name and a git+ reference are passed over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 01:44:51 +02:00
kevinveenbirkenbachandClaude Opus 5 fc0881ff33 fix(env): point python at the OS trust store
The package-cache frontend terminates TLS for package traffic and signs
with its own CA: probed from inside the test container, pypi.org arrives
as subject CN=pypi.org issued by CN=Infinito Package-Cache Frontend CA.
package-frontend-ca.sh installs that CA into the OS store, which is why
apt, pip, curl and urllib work — but requests reads certifi, which has
never heard of it, so test-external failed its reachability precondition
with CERTIFICATE_VERIFY_FAILED. Upgrading certifi does not help; the
issuer is not a public root.

load.sh is where the runner puts the venv on PATH, so it is also where the
interpreter carrying the pip certifi is chosen; pointing REQUESTS_CA_BUNDLE
and SSL_CERT_FILE at the store the stack deliberately fills keeps one trust
decision instead of two. A caller-set value still wins.

The bundle location was about to become a fourth copy: dev-nix and the
network diagnose CLI each carried their own candidate list, and the two had
already drifted apart in their fourth entry. All of them now read
group_vars CA_BUNDLE_CANDIDATES — Ansible directly, the shell through a
generated env key, python through read_group_paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 18:55:58 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 f4a713d827 perf(registry): probe the image mirror before Docker Hub and cache sizes
The min_storage gate measures every declared pin on each run, and every measurement was an anonymous manifest request to Docker Hub, which meters those per IP. A full run exhausted the pull budget, so the following local deploy could no longer pull at all and the gate itself stopped measuring: 96 of 165 images answered indeterminately and only warned.

Read the GHCR mirror first and fall back to the upstream registry only for images it does not carry. The copy is byte-identical, so the measured size is unchanged. Persist measured sizes under the temporary directory; store positive results only, because a cached indeterminate answer would be remembered as fact and the gate would stop measuring for good.

Lift the mirror path formula out of the provider into utils so the writer and the reader share one definition, resolving the namespace from GITHUB_REPOSITORY, then remote.pushDefault, then origin. pushDefault has to win: origin may point at an upstream that hosts no mirror. Parse .git/config by hand because configparser rejects the duplicate keys git writes and the test container ships no git binary.

Measured over all 165 declared pins: no requests to Docker Hub, 277 to the mirror, 9 to registries it does not carry, and 164 of 165 images measured.

Leave manifest_exists on the upstream registry. It answers whether the declared pin is still pullable from where the deploy pulls it, which a mirrored copy cannot attest; it takes the cache only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 17:22:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1c5230302 Merge branch 'main' into feature/swarm-nfs
Conflicts resolved:
- .claude/.gitignore: union of ignore patterns
- plugins/lookup/rbac_group_path.py: drop both unused conflicting imports
  (body fetches apps via lookup_loader)
- roles/sys-svc-compose/handlers/main.yml: keep HEAD's refactor (compose-up
  handlers live in compose.yml, included per DEPLOYMENT_MODE)
- utils/cache/base.py: keep HEAD's merge_with_defaults/FILE_TOKENS imports

Post-merge fixes for main's newly-enabled lints:
- pyproject.toml: grandfather 5 files under the BLE001 ratchet
- justify 6 pre-existing security # noqa markers (S108/S311/S701)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:41:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ce6780575 refactor(meta): split server.yml into csp/domains/networks.yml
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:

  csp:      -> meta/csp.yml      -> applications.<app>.csp
  domains:  -> meta/domains.yml  -> applications.<app>.domains
  networks: -> meta/networks.yml -> applications.<app>.networks

Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.

- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
  applications.<app>.{csp,domains,networks} across templates, vars, tasks,
  lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
  build_ldap_role_entries, service-registry overlay discovery,
  roles_with_service, and the file-based readers utils/domains/list.py +
  utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
  ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
  server.yml still carries a split field; auto-fails from 2026-12-28 as a
  delete reminder.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc77d9d265 fix(security): resolve all open code-scanning alerts and guard against recurrence
Resolve the 80 open CodeQL/code-scanning alerts on this branch.

Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
  extracting the env/arg helpers into a leaf module `env.py`, giving the
  load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
  cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
  exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.

Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.

False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).

Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
  package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
  current production offenders grandfathered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:42:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8a586ad373 feat(lint): validate generated distro changelogs + harden the mirror generator
Adds a lint-packages target/workflow that parses the generated packaging metadata with each ecosystem's native parser (dpkg-parsechangelog, rpmspec, bash -n for the PKGBUILD), failing on any parser stderr so warnings cannot slip through. Wired into the make lint fan-out and ci-orchestrator's lint gate; an install group provisions the parsers (best-effort, cross-distro).

Hardens package_mirror so the generated changelogs parse without warnings: every RPM body line gets a '- ' continuation prefix (a column-0 '*' was read as a new %changelog entry -> 'bad date'), and the archived-releases notice is folded into the oldest kept entry instead of trailing after the final entry (free text there made dpkg-parsechangelog warn). Regenerates packaging/debian/changelog and the Fedora spec accordingly.

Adds a test-lint check (test_package_changelog_completeness) asserting both package changelogs list every CHANGELOG.md + docs/changelog release and that their inline window matches CHANGELOG.md, plus unit regressions for the RPM continuation prefix and the Debian footer fold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 21:36:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 376822ebe4 fix(changelog): indent Debian changelog bodies so dpkg-buildpackage parses them
format_debian_entry indented only the first body line per entry, leaving the rest at column 0. dpkg's changelog parser treats un-indented change lines as badly formatted, loses the entry boundary and reports 'cannot parse maintainer email address ""', failing the .deb build in build-ci-images. Every non-empty body line is now indented by two spaces.

trim_and_archive additionally dropped the '# Changelog' H1 preamble on every rewrite (it rebuilt content from the version entries only), which would break the MD041 markdown lint on the next trim. The preamble before the first version header is now preserved.

Regenerated packaging/debian/changelog and packaging/fedora/infinito-nexus.spec from CHANGELOG.md (untrimmed). Also adds the missing YAML document-start to web-app-pihole/tasks/main.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 21:36:30 +02:00
kevinveenbirkenbach e685f5f8e4 Merge branch 'main' into fix/ci-deploy-server-failures 2026-06-13 17:18:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5dd03210c5 fix(changelog): use asterisk bullets in Older Releases index
The auto-generated "Older Releases" index in CHANGELOG.md emitted dash
bullets while the rest of the file uses asterisks, failing markdownlint
MD004 (ul-style, consistent) with 49 errors. Switch the archive index
generator to asterisk bullets, convert the existing entries, and align
the unit test assertions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 17:18:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 61477fb55d refactor(mirror): extract the wait-for-mirrors logic into a cli module
scripts/meta/wait/mirrors.sh embedded its logic as a `python - <<PY` heredoc
plus a bash skopeo/retry loop, and the heredoc imported iter_role_images from
utils.docker.image_discovery — a module that does not exist (it is
utils.docker.image.discovery). That raised ModuleNotFoundError, so the process
substitution yielded zero refs and the script logged "Nothing to wait for" and
exited 0, silently turning the fork-PR "wait for mirrored images" safeguard
into a no-op.

Move the whole thing into cli/contributing/mirror/wait/ (sibling to sync and
resolver): an argparse entrypoint that enumerates role images via
iter_role_images, computes each GHCR destination via GHCRProvider, and polls
GHCRProvider.tag_exists with --attempts/--sleep-seconds until all are present
or it times out. mirrors.sh becomes a thin env->args wrapper invoking
python -m cli.contributing.mirror.wait, matching scripts/meta/mirror/sync_missing.sh.

Behaviour is identical to the previous bash, minus the broken import.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 13:42:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 519010c439 refactor(requirements): move archive CLI from cli/ to kpmx (pkgmgr)
The fully-checked-Markdown archive helper used to live under
cli/contributing/requirements/archive/ in this repo. It is a generic
spec-folder-management primitive (numbered NNN-topic.md files,
task-list completeness check, README ## Archive section append) and
now ships in pkgmgr.actions.archive so every kpmx-managed repository
can rely on the same convention.

Changes here:

* tests/lint/repository/documentation/test_requirements_completeness.py
  imports the archive primitives from pkgmgr.actions.archive instead
  of the vendored cli.contributing.requirements.archive, and points
  contributors at `pkgmgr archive docs/requirements` in the failure
  message.
* cli/contributing/requirements/README.md becomes a stub that
  redirects readers to the kpmx archive subcommand.
* The vendored cli/contributing/requirements/archive/ package and its
  unit tests under tests/unit/ are deleted.
* The pkgmgr role pin bumps to v1.15.0 to pick up the new archive
  subcommand on deployed hosts.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 08:06:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a23a67f21a chore(lint): satisfy ruff TC/SIM/S checks + cache-read exemption
* TC003: move `pathlib.Path` into `TYPE_CHECKING` blocks across the
  new `requirements/archive` package and its completeness lint test.
* TC001: same treatment for `VaultHandler` in
  `credentials/emit.py` (annotation-only).
* SIM105/S110: replace `try/except/pass` in
  `current_play_redirect_domains.py` with
  `contextlib.suppress(Exception)`.
* `archive/__main__.py`: mark `README_PATH.read_text(...)` with
  `# nocheck: cache-read` — CLI rewrites the README in the same flow,
  mirroring the `changelog/archive` pattern.
* Drive-by ruff-format condensations in the archive unit tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 15:42:54 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9b8bceec62 feat(requirements): add archive CLI + completeness lint
* `python -m cli.contributing.requirements.archive` moves the H1 of
  every fully-checked `docs/requirements/NNN-*.md` into the
  `## Archive` block of the directory README and deletes the file.
  Files with any remaining `- [ ]` are skipped. Split SRP-wise into
  discovery / inspect / readme / __main__.
* New lint `test_requirements_completeness` warns per unchecked
  acceptance criterion (GitHub annotation) and fails when any
  requirement is ready for archival, pointing at the CLI. Keeps the
  requirements directory small so AI agents process less context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 15:42:53 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e4e861c261 refactor(cli): move ports/networks suggesters under cli/contributing/network and add diagnose subcommand
Relocates `cli/meta/{ports,networks/suggest}` to
`cli/contributing/network/{ports,address/suggest}` so port and subnet
allocators live alongside the rest of the contributor-side tooling
(mirror, update, changelog) instead of `meta`. Mirrors the tests
into `tests/unit/cli/contributing/network/` and renames the contract
docs to `docs/contributing/tools/network/{port,address}.md`. Prog
names update from `cli meta ports suggest` / `cli meta networks
suggest` to `cli contributing network ports suggest` / `cli
contributing network address suggest`; the corresponding lint
references and `utils.meta.scan` docstring follow.

Adds a third subcommand: `cli contributing network diagnose`
(stdlib-only DNS / TCP / TLS / PMTU probe against the standard
infrastructure hosts plus interface MTU, `/etc/resolv.conf`,
`/etc/hosts`, proxy env, CA-bundle summaries). Split across `cli.py`
(orchestration), `config.py` (SPOT for hosts / per-distro install
commands), `format.py` (output + subprocess helpers), `tools.py`
(distro-aware auto-install of `iputils-ping` + `iproute2`),
`report.py` (system-info dumps + IPv6-default-route detection), and
`probes.py` (per-host probes). Exposed as `make debug-network`.

Companion `docs/contributing/actions/debugging/network.md` absorbs
the diagnose-tool walkthrough (incl. how to read MTU-induced TLS
handshake timeouts) so the merged page covers DNS / MTU / IPv6 /
connectivity end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 98457f98fd refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.

utils/
  update/
    base.py        # semver primitives shared by both backends
    docker.py      # OCI registry lookups + version: line rewriter
    repository.py  # git ls-remote walker + ref: line rewriter

cli/contributing/update/
  docker/           # python -m cli.contributing.update.docker
  repository/       # python -m cli.contributing.update.repository

scripts/github/update/
  image_versions.sh
  repository_refs.sh

Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.

Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 525699a9ab test(lint/cache-read): extend scope to utils, cli, plugins, library
Modules imported by tests run inside the same pytest process as the
tests themselves. Raw ``path.read_text(...)`` calls in those modules
bypass the cache the tests just populated. ``roles/`` stays excluded
(deployed scripts have no ``utils/`` on PYTHONPATH); ``utils/cache/``
stays excluded (cache implementation itself).

Five conversions to ``utils.cache.files.read_text``:
- ``utils/handler/yaml.py`` (YAML loader, read-only)
- ``utils/annotations/summarize.py`` (CI log scanner, with
  ``UnicodeDecodeError`` fallback replacing ``errors="replace"``)
- ``cli/meta/roles/applications/resolution/combined/__main__.py``
  (single-shot CLI JSON input)
- ``cli/core/help.py`` (read_folder_title + read_folder_description;
  same README is read many times across CLI help iteration)

Eight ``# nocheck: cache-read`` markers, each annotated inline:
- ``utils/claude/settings_sort.py`` (read then rewrite sorted)
- ``utils/docker/version_updater.py`` (read then rewrite config)
- ``cli/administration/deploy/development/network.py`` (tests mock
  ``Path.read_text`` directly; caching would bypass the mock)
- ``cli/administration/deploy/development/log_utils.py`` (append flow)
- ``cli/contributing/changelog/archive/changelog_md.py``,
  ``__main__.py``, ``package_mirror.py`` (file mutated in same flow)

Two callers refactored to variable extraction so ruff's 88-char wrap
keeps the ``# nocheck`` marker on the ``read_text(...)`` AST span line
(``cli/.../network.py`` and ``package_mirror.py:105``).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 14:23:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 576fad84ff refactor(images): move role images from defaults/main.yml to meta/services.yml
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.

`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a26bfc4e94 refactor(cli): reorg admin/contributing tree + per-folder help system
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):

- cli/deploy             -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault              -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror             -> cli/contributing/mirror

The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".

Help system redesign in cli/core/{discovery,help,app}.py:

- Discovery now distinguishes runnable commands (directories with
  __main__.py) from category folders (directories that only group
  further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
  as the title and its first paragraph as the one-line description in
  the help output.
- "infinito" / "infinito --help" prints only the top-level entries
  (one level deep). "infinito <category>" navigates into a folder and
  shows its sub-folders and commands one level deep, with the README
  title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
  prefix recursively. N caps the depth (1-based); omit N for an
  unbounded walk. The output uses Unicode tree connectors, an emoji
  per entry (folder for categories, gear for commands), a globally
  aligned description column, and multi-line wrapped descriptions
  with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
  ("infinito meta callorder") rather than the bare leaf name, so the
  output is copy-pasteable.

Convention doc + 17 CLI READMEs:

- New docs/contributing/artefact/files/cli/readme_md.md describes the
  H1+paragraph layout the help system reads, and how it differs from
  role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
  and their immediate sub-folders, plus cli/meta/roles/applications
  and its resolution sub-folder. Each follows the "title with trailing
  emoji + single descriptive paragraph" convention.

Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.

Fixes uncovered by the move:

- import unittest.mock added to four dedicated-deploy tests so
  unittest.mock.patch resolves under stdlib unittest discovery
  (Python 3.13 no longer surfaces unittest.mock through "import
  unittest" alone).
- parents[N] index in the new
  cli/administration/inventory/validate/__init__.py corrected from
  3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
  substring assertions updated to match the new validator path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:17:09 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7dc6d6e491 chore(ruff): enable TC + N rules
TC (flake8-type-checking): typing-only imports moved into
``if TYPE_CHECKING:`` blocks across 8 source files via auto-fix
(117 hits). Reduces runtime import surface and rules out circular
imports for type-only deps. Project-wide ``from __future__ import
annotations`` (FA, already enabled) keeps annotations as strings,
so the moved imports never need to load at runtime.

N (pep8-naming): 32 hits handled.

Renames (semantic + lint):
- ``isSmallerThenMaximumBackupSize`` -> ``is_larger_than_maximum_backup_size``
  (snake_case + fixes both the typo "Then -> Than" and the
  semantic mismatch: the function returns ``current > maximum``)
- ``isDirectoryDeletable`` / ``deleteVersion`` /
  ``getAmountOfIteration`` / ``deleteIteration`` -> snake_case
- 8 test functions with ALL-CAPS env-var fragments
  (``test_..._DOMAIN_PRIMARY_...`` etc.) -> snake_case
- Mock alias ``VH`` -> ``vh`` in test_host_vars.py

Exception class renames (``N818`` Error suffix):
- ``TimeLimitExceededException`` -> ``TimeLimitExceededError``
- ``AttemptException`` -> ``AttemptError``
- ``PathNotFound`` -> ``PathNotFoundError`` (propagated across
  4 callers under tests/integration/lookups/config/)
- ``_ExitJson`` / ``_FailJson`` -> ``*_Error``

Refactor:
- ``METADATA = {...}`` local constant in two functions of
  ``plugins/filter/invokable_paths.py`` lifted to module-level
  ``_METADATA_KEYS``.
- ``from typing import TYPE_CHECKING`` in
  ``tasks/utils/migrate_meta_layout/__main__.py`` moved above
  the ``sys.path.insert`` hack to satisfy E402 (collateral from
  the TC auto-fix that landed it after the path mutation).

Per-file ignores in pyproject.toml for the legitimate cases the
rule cannot accept:
- ``pull-specific-host.py`` -> ``N999`` (Ansible-invoked role-level
  script, hyphenated filename pinned by the surrounding stack).
- 4 matrix-deploy files -> ``N806`` for the ``include_R`` /
  ``raw_include_R`` round-index notation; the uppercase ``R``
  here mirrors the formula in the docstring rather than naming
  a constant.
- ``plugins/filter/value_generator.py`` -> ``N806`` for the
  lazy-loaded class binding ``ValueGenerator =
  _value_generator_cls()``, which retains the imported class
  identifier per PEP 8.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 18:57:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 efacb56985 refactor(cli): retire one-shot fix tools, move docker version updater
Drop four one-shot or now-redundant tools from cli/fix/:

* vars_main_files — bulk-created vars/main.yml with application_id
  per role-prefix during the SPOT migration. The schema is now
  enforced declaratively by tests/integration/roles/applications/id/
  test_matches_role_name.py and the role-mapping SPOT lint.
* move_unnecessary_dependencies — moved meta dependencies to guarded
  include_role on first sweep. Permanent enforcement lives in
  tests/integration/roles/meta/dependencies/test_unnecessary.py.
* tabs — replaced literal tab characters with two-space indentation
  in YAML files. Autoformat covers new occurrences.
* ini_py — generic helper that filled missing __init__.py files; new
  packages now ship them from the start.

Move the remaining (still-actively-used) docker version updater out
from under cli/fix/ and into cli/contributing/, where the recurring
maintenance tooling lives:

* cli/fix/docker_image_versions → cli/contributing/docker/update_versions

Update the corresponding wiring:

* scripts/github/update_docker_image_versions.sh now calls
  python -m cli.contributing.docker.update_versions
  (the entry point invoked daily by .github/workflows/update.yml).
* docs/requirements/008-role-meta-layout.md references the new path.
* PROJECT_ROOT in __init__.py drops one parents[] level (the package
  sits one folder deeper now).

cli/fix/ is fully removed; new contributor-facing tooling belongs
under cli/contributing/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:19:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 66badf258e refactor(changelog): split archive CLI into per-responsibility modules
Break cli/contributing/changelog/archive/__main__.py into four
single-responsibility modules under the same package:

* versioning.py — pad_version / unpad_version / archive_filename
  (the canonical <padded-semver>-<date>.md filename schema).
* archive_dir.py — existing_archives, archived_releases_from_directory,
  build_index_section (filesystem operations on docs/changelog/).
* changelog_md.py — split_into_entries, md_body_after_header,
  trim_and_archive (CHANGELOG.md parsing and trimming, plus the
  always-rebuild-index logic).
* package_mirror.py — mirror_to_debian_changelog,
  mirror_to_rpm_spec_changelog (Debian and RPM regeneration from
  CHANGELOG.md kept entries, with the trailing notice that points at
  docs.infinito.nexus and lists archived versions as plain text).

__main__.py shrinks to argparse plus a thin orchestration layer that
parses CHANGELOG.md once, runs trim_and_archive, reads the populated
archive directory, and mirrors into the package files.

Add `mirror_to_debian_changelog` and `mirror_to_rpm_spec_changelog`
content-idempotency: skip the write when the regenerated content
already matches what is on disk so re-runs leave file mtimes
unchanged. Verified end-to-end: a follow-up run on a fully-processed
tree leaves all three files (CHANGELOG.md, Debian changelog, RPM
spec) byte-identical and mtime-stable.

Replace `datetime.strptime` calls with `date.fromisoformat` so the
DTZ007 lint stays clean: we only need calendar-date parsing for
display formatting, not timezone-aware moments.

Tests follow the same split: one test_<module>.py per source module,
40 tests in total. Shared fixtures live in _helpers.py (underscore
prefix exempts it from the test-file-naming lint), and each test
class explicitly mixes TempRepoMixin with unittest.TestCase so the
test-files-have-tests AST lint sees the direct unittest base.

Update docs/contributing/actions/release.md so the procedure
description matches the current behaviour: package changelogs are
"mirrored" (not "trimmed") from CHANGELOG.md, and the idempotence
section calls out byte-idempotency on each of the three files.

Run the CLI once against the live tree to bring packaging/debian/
changelog and packaging/fedora/infinito-nexus.spec back into sync
with CHANGELOG.md and the docs/changelog/ archive directory.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:19:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7d7301614d docs(changelog): per-release archive CLI + release docs + SPOT cleanup
Add cli/contributing/changelog/archive: a CLI that trims CHANGELOG.md
to the last N entries (default 7) and writes every older release to
its own file under docs/changelog/ named <padded-semver>-<date>.md
(7.0.0 → 007.000.000-2026-05-08.md). The padded form makes a directory
listing already chronological, and the index in CHANGELOG.md is
rebuilt from the archive directory listing on every run, so it never
drifts away from what is on disk. The tool is idempotent and never
overwrites an existing archive file.

Run the migration once to split the 44-entry CHANGELOG.md into 7
inline entries plus 37 archived per-release files, with the index at
the bottom of CHANGELOG.md linking each archive in descending order.

Add docs/contributing/actions/release.md as the contributor-facing
release procedure (versioning, active window vs. archive, cutting a
release end-to-end, CI hand-off on tag push) and add a docs README
under docs/changelog/ that explains the archive directory layout and
filename schema. Cross-link release.md from pipeline.md and add
"releases" to the actions/ scope description.

Documentation cleanup pass on every page touched in the previous
mapping refactor:

- Remove "is the SPOT for X" / "Single Point of Truth" self-labels
  across docs/contributing/, docs/agents/, and docs/changelog/. The
  rule in documentation.md forbids self-declared SPOT status; status
  is conferred by readers cross-linking, not by the page asserting
  it. Rephrase to descriptive phrasing ("This page covers X",
  "describes X", "lists X") throughout.
- Drop the orphaned docs/contributing/tools/actions.md duplicate
  (101 lines paralleling tools/github/actions/workflows.md, with no
  inbound links).
- Fix em-dashes used as thought breaks in dockerfile.md, origin.md,
  agents/style.css.md, and suppression.md per documentation.md.
- Unwrap soft-wrapped sentences in iam/common.md, iam/ldap.md,
  iam/oidc.md, image/origin.md, requirements.md, and dockerfile.md
  so each logical line carries one full sentence.

Add a unit test for the archive CLI under
tests/unit/cli/contributing/changelog/archive/test_main.py covering
_pad_version, _split_into_entries, idempotency, dry-run, no-overwrite
of existing archives, and the multi-run index rebuild.

Quote `# noqa:` in CHANGELOG.md prose with backticks so the
noqa-only-ruff-codes lint does not parse "for" as a non-ruff code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:19:42 +02:00