mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-10 12:47:04 +00:00
8c7cdff6b68e37e71fa2108be33db750e084e5ed
32
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  kevinveenbirkenbachandClaude Opus 4.8
|
6538905bd7 |
refactor(lint): fold the Dockerfile hadolint check into make lint
The separate CI job is gone; hadolint now runs as lint-dockerfile inside the parallel make lint fan-out through the standard wrapper, with the binary installed by the lint toolchain installer like every other tool. The SARIF gate script had no consumer left and is removed. Scope stays the root Dockerfile at the warning threshold, matching the old job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Fable 5
|
40e553069d |
refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations. Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
b533863559 |
fix(env): keep the .env generation stdlib-only for fresh-host bootstrap
The nfs SPOT loaders pulled PyYAML into the .env generation, which bootstraps fresh hosts BEFORE python dependencies exist: the import crashed cli.meta.env, load.sh produced an empty environment and make install-system-python died on the required-var guard (environment/manjaro and pkgmgr/centos CI jobs). The loaders now line-parse the services.yml SPOTs via the stdlib-backed cached reader and the mapping path constant. The client's protocol version field becomes nfs_version, since services.yml reserves version for image tags (schema lint), and the image discovery casts versions defensively. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Fable 5
|
e35acc09bc |
feat(updater): support registry.gitlab.com images
Add registry.gitlab.com to the mirrorable registries so discovery yields the GitLab CNG images to the version updater and mirror sync. Seed the generic v2 tag scan with the OCI last cursor for v-prefixed pins and raise the page size to 1000: CNG repos carry tens of thousands of commit-sha tags that sort before the release tags, so an unseeded scan exhausted the page cap without reaching any semver tag. Pin every gitlab service version in meta/services.yml and drop the GITLAB_VERSION template override so services.yml is the single source for per-service versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
1b774e410c |
feat(update): support every registry via a generic v2 client
Add a generic Docker Registry v2 client (WWW-Authenticate bearer-token challenge) so tag listing and reachability work for any registry (Docker Hub, GHCR, MCR, Quay, GitLab/opencode), and route the version updater through it so quay.io and opencode.de images auto-update like the rest. Add an external test that hard-fails when a pinned image:version tag is unreachable (404) and warns on incomplete image/version pairs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
e9b75f4ddf |
fix(mirror): enroll registry.opencode.de in the GHCR mirror (web-app-opentalk)
OpenTalk's images (web-frontend, controller, recorder) live on registry.opencode.de, which was absent from both _MIRRORABLE_REGISTRIES and _ALL_REGISTRY_PREFIXES in utils/docker/image/discovery.py. So is_mirrorable_image returned False, iter_role_images skipped them, the GHCR mirror never copied them, and the deploy pulled direct from registry.opencode.de -> TLS handshake timeout (deploy failed). Add the registry to both sets so the single iter_role_images gate enrolls the 3 images: the mirror-sync copies them to GHCR and the resolver rewrites the deploy to pull from the GHCR mirror. The prefix entry strips the host so the mirror destination is well-formed (no doubled registry). Add a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
83466868e9 |
feat(update): support mcr.microsoft.com in docker version checker
- Add MCR_REGISTRY constant, is_mcr / mcr_repo / fetch_mcr_tags helpers mirroring the Docker Hub and GHCR pattern. Tags are fetched via the Docker Registry V2 API with RFC 5988 Link-header pagination (anonymous access; MCR is a public read mirror). Wire MCR into both the external test_image_versions warning surface and the find_outdated_updates orchestrator in utils.update.docker. - Bump mcr.microsoft.com/playwright in test-e2e-playwright from v1.59.1-noble to v1.60.0-noble — the update was surfaced by the new MCR detection path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
98457f98fd |
refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
525699a9ab |
test(lint/cache-read): extend scope to utils, cli, plugins, library
Modules imported by tests run inside the same pytest process as the tests themselves. Raw ``path.read_text(...)`` calls in those modules bypass the cache the tests just populated. ``roles/`` stays excluded (deployed scripts have no ``utils/`` on PYTHONPATH); ``utils/cache/`` stays excluded (cache implementation itself). Five conversions to ``utils.cache.files.read_text``: - ``utils/handler/yaml.py`` (YAML loader, read-only) - ``utils/annotations/summarize.py`` (CI log scanner, with ``UnicodeDecodeError`` fallback replacing ``errors="replace"``) - ``cli/meta/roles/applications/resolution/combined/__main__.py`` (single-shot CLI JSON input) - ``cli/core/help.py`` (read_folder_title + read_folder_description; same README is read many times across CLI help iteration) Eight ``# nocheck: cache-read`` markers, each annotated inline: - ``utils/claude/settings_sort.py`` (read then rewrite sorted) - ``utils/docker/version_updater.py`` (read then rewrite config) - ``cli/administration/deploy/development/network.py`` (tests mock ``Path.read_text`` directly; caching would bypass the mock) - ``cli/administration/deploy/development/log_utils.py`` (append flow) - ``cli/contributing/changelog/archive/changelog_md.py``, ``__main__.py``, ``package_mirror.py`` (file mutated in same flow) Two callers refactored to variable extraction so ruff's 88-char wrap keeps the ``# nocheck`` marker on the ``read_text(...)`` AST span line (``cli/.../network.py`` and ``package_mirror.py:105``). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
576fad84ff |
refactor(images): move role images from defaults/main.yml to meta/services.yml
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.
`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c55ad69c41 |
refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`, `requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from non-Markdown source files. Comments anchored to requirement numbers rot when requirements are renumbered or rewritten and conflate present-state behaviour (which the comment should describe) with the audit trail of how the code got there (which belongs in commit messages, PR descriptions, and the requirement file itself). * Document the rule in `docs/contributing/documentation.md` under "Comments". Code MUST describe the *why* in present-state language; contract enforcement lives in tests, not folklore. * Clean 254 source files across `roles/`, `tests/`, `plugins/`, `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root `eslint.config.js`. Pure comment / docstring / log-message edits; no behavioural change. * Add `tests/lint/repository/test_no_req_references_in_code.py` to enforce the rule going forward across `.js`, `.py`, `.yml`, `.yaml`, `.j2`, and `.sh` files. The legacy `test_no_stale_requirement_refs.py` is exempt as the only file whose purpose is to detect these very tokens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
7dc6d6e491 |
chore(ruff): enable TC + N rules
TC (flake8-type-checking): typing-only imports moved into
``if TYPE_CHECKING:`` blocks across 8 source files via auto-fix
(117 hits). Reduces runtime import surface and rules out circular
imports for type-only deps. Project-wide ``from __future__ import
annotations`` (FA, already enabled) keeps annotations as strings,
so the moved imports never need to load at runtime.
N (pep8-naming): 32 hits handled.
Renames (semantic + lint):
- ``isSmallerThenMaximumBackupSize`` -> ``is_larger_than_maximum_backup_size``
(snake_case + fixes both the typo "Then -> Than" and the
semantic mismatch: the function returns ``current > maximum``)
- ``isDirectoryDeletable`` / ``deleteVersion`` /
``getAmountOfIteration`` / ``deleteIteration`` -> snake_case
- 8 test functions with ALL-CAPS env-var fragments
(``test_..._DOMAIN_PRIMARY_...`` etc.) -> snake_case
- Mock alias ``VH`` -> ``vh`` in test_host_vars.py
Exception class renames (``N818`` Error suffix):
- ``TimeLimitExceededException`` -> ``TimeLimitExceededError``
- ``AttemptException`` -> ``AttemptError``
- ``PathNotFound`` -> ``PathNotFoundError`` (propagated across
4 callers under tests/integration/lookups/config/)
- ``_ExitJson`` / ``_FailJson`` -> ``*_Error``
Refactor:
- ``METADATA = {...}`` local constant in two functions of
``plugins/filter/invokable_paths.py`` lifted to module-level
``_METADATA_KEYS``.
- ``from typing import TYPE_CHECKING`` in
``tasks/utils/migrate_meta_layout/__main__.py`` moved above
the ``sys.path.insert`` hack to satisfy E402 (collateral from
the TC auto-fix that landed it after the path mutation).
Per-file ignores in pyproject.toml for the legitimate cases the
rule cannot accept:
- ``pull-specific-host.py`` -> ``N999`` (Ansible-invoked role-level
script, hyphenated filename pinned by the surrounding stack).
- 4 matrix-deploy files -> ``N806`` for the ``include_R`` /
``raw_include_R`` round-index notation; the uppercase ``R``
here mirrors the formula in the docstring rather than naming
a constant.
- ``plugins/filter/value_generator.py`` -> ``N806`` for the
lazy-loaded class binding ``ValueGenerator =
_value_generator_cls()``, which retains the imported class
identifier per PEP 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
f6c8c85b1b |
test(lint): role-file-spot guard + 139-file mapping migration
Add tests/lint/repository/test_role_file_spot.py: AST-walks every .py file under the production trees and tests/, flags any reference to a role file (standalone literal, f-string segment, pathlib `/` chain, `+` concat chain, os.path.join / Path.joinpath / posixpath.join / ntpath.join call) whose value or joined trailing operands match a ROLE_FILE_* constant in utils.roles.mapping. The flag set is auto-discovered from the module so adding a new constant extends coverage without further edits. ROLE_FILE_README is exempt as too generic. Per-line `# nocheck: role-file-spot` (same-or-above) is the documented opt-out, catalog row added in docs/contributing/actions/testing/suppression.md. Migrate 139 .py files (377 occurrences) to use the constants: - 135 files via direct constant import (`from utils.roles.mapping import ROLE_FILE_*`), with the symbol used in pathlib chains, in os.path.join / Path.joinpath calls, in f-string interpolation, and in glob patterns. - 4 files via `# nocheck: role-file-spot` where the literal lives in argparse help text or in an allowlist tuple meant to declare the literal path. Adjust utils/roles/dependency_resolver.py to use lazy-formatting positional args in the logger.exception call (G003-clean) instead of string concatenation against the constant. Drop a redundant `if cond: return True; return False` guard in the new test for `return cond` (SIM103-clean). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
d3afe0bc81 |
test(lint): forbid a file from referencing its own path as a string
Hard-coded self-path references rot the moment the file moves: a header
comment ``# utils/foo/bar.py`` at the top of a module, a docstring
mentioning ``utils/foo/bar.py``, or a docs entry linking
``utils/foo/bar.py`` from inside ``utils/foo/bar.py`` itself all silently
desync from reality after the next ``git mv``.
The new lint walks every project file, computes its POSIX-style relative
path from PROJECT_ROOT, and reports any line whose text contains that
path as a substring. Files at the repo root (no ``/`` in their relative
path) and setuptools-generated ``*.egg-info/`` artefacts are exempt.
Per-line opt-out via ``# nocheck: self-path-reference`` on the offending
line, registered in the suppression catalog.
Existing offenders cleaned up:
- 72 redundant ``# <self-path>`` header lines removed across roles,
cli, plugins, scripts, tests, and utils.
- 4 prometheus templates that told consumers to "update
<self-template-path>" rewritten to drop the redundant self-reference.
- 8 docstring/comment self-references in cli, plugins, tests, utils,
and the dashboard oidc.js.j2 header rewritten to describe the layout
without naming the file's own path.
- 17 legitimate usage examples (shell scripts, test data lists, one
template self-include doc-block) marked with ``nocheck:
self-path-reference``.
- ``container.sh`` heredoc usage strings switched to ``${0}`` so the
marker stays out of the user-visible help output.
- ``healthcheck/http.yml.j2`` doc-block rewritten to prose because
``{# ... #}`` Jinja comments cannot host an inner ``{# nocheck #}``
(the inner ``#}`` closes the outer comment).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
85f1caccc7 |
refactor(utils): consolidate role-scoped helpers under utils/roles and split templating
Tighten the utils namespace so each subpackage owns one concern:
- utils/applications/* moves under utils/roles/applications/* (helpers
that operate on the runtime applications dict and meta/services.yml).
- utils/service_registry.py and utils/database_service.py move under
utils/roles/applications/services/ as registry.py and database.py.
- utils/entity_name_utils.py becomes utils/roles/entity_name.py.
- utils/invokable.py, utils/valid_deploy_id.py, and the existing
utils/roles/resource_validation.py group under utils/roles/validation/
as invokable.py, deploy_id.py, and resources.py.
- utils/templating.py and utils/jinja_strict.py split into
utils/templating/ansible.py (Templar + TrustedAsTemplate, lookup-env
shortcut) and utils/templating/jinja.py (vanilla Jinja2 Environment,
multi-pass strict render).
Every caller in utils, cli, plugins, roles/<role>/{filter,lookup}_plugins,
tests, and the docs that link to the moved files is updated to the new
import paths. Test files mirror the new source layout
(tests/unit/utils/{roles,templating}/...).
New READMEs at utils/cleanup/, utils/roles/, utils/roles/applications/services/,
utils/roles/validation/, and utils/templating/ follow the
documentation.md and readme_md.md guidelines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
191bc1229a |
chore(ruff): enable S (bandit) rule family — 269 → 0
Adds ``S`` (flake8-bandit) to the strict ruleset. ``PTH`` was scoped
out: it surfaced 1000+ violations with no auto-fix support (most PTH
sub-rules are warning-only because each ``str`` ↔ ``Path`` conversion
is context-sensitive). Reserved for its own dedicated commit.
Config tuning
=============
Global ``ignore`` additions with rationale:
- ``S603`` (subprocess call without ``shell=False``) and ``S607`` (start
process with partial path) are over-eager for a project whose CLI
invokes ``make`` / ``git`` / ``docker`` / ``ansible-playbook`` from
PATH using list-form args. The list-form already prevents shell
injection (Python's ``subprocess`` does not parse argv[0] as a shell
command), and resolving every binary's absolute path in the source
would couple the code to the host filesystem layout. The genuinely
dangerous shell-injection patterns (``shell=True``) are still caught
by S602 / S605.
New ``[tool.ruff.lint.per-file-ignores]`` for ``tests/**``:
- ``S101`` — ``assert`` is the unittest idiom.
- ``S102`` — ``exec`` is used to load script-under-test by file path.
- ``S105`` / ``S106`` — hardcoded credentials in fixtures / dummy creds.
- ``S108`` — ``/tmp`` paths in tempdir fixtures.
- ``S110`` / ``S112`` — ``try``/``except``/``pass``/``continue`` in cleanup helpers.
Hand-fixes (38 across 23 files)
================================
S101 × 3
Type-narrowing asserts (``cli/deploy/development/proc.py`` —
``p.stdout is not None`` after ``stdout=PIPE``) and an upstream-Odoo
invariant assert in ``roles/web-app-odoo/.../res_users.py``.
S105 × 1
``cli/core/app.py`` argv parser comparing ``token == "--log"`` —
``token`` is a CLI argv element, not a credential. Inline noqa.
S106 × 1
``utils/cache/users.py`` Jinja template string ``"{{ 42 |
strong_password }}"`` rendered to a real password at runtime; not a
literal credential. Inline noqa.
S108 × 6
Container-internal paths bind-mounted from the host:
- ``/tmp/infinito/ca/root-ca.crt`` and ``/tmp/infinito/bin/with-ca-trust.sh``
in ``roles/sys-svc-compose-ca/files/compose_ca.py`` and
``roles/sys-svc-container/files/container.py`` (CA-injection
contract between role and wrapper).
- ``/tmp/cache_nginx_general/`` and ``/tmp/cache_nginx_image/`` in
``plugins/lookup/nginx.py`` (well-known nginx cache dirs).
S110 × 1
``roles/web-app-bigbluebutton/filter_plugins/compose_mods.py``: filter
intentionally swallows YAML parse errors and falls back to the
unmodified text. Inline noqa.
S112 × 5
Filter plugins / inventory builders iterating over role config files
with skip-on-malformed semantics. The pattern is intentional
throughout (CLI tools / Ansible plugins shouldn't crash on a single
bad role file). Per-line noqa with rationale.
S324 × 2
``utils/cache/base.py`` uses md5 as a fast non-cryptographic
fingerprint for cache keying; ``utils/manager/value_generator.py``
uses sha1 when the caller selects ``algorithm == "sha1"`` for legacy
app compatibility. Both gained ``usedforsecurity=False`` (Python 3.9+
hashlib API) which silences the rule structurally; comments document
the non-security purpose.
S310 × 8
``urlopen`` calls to fixed HTTPS registries (``api.github.com``,
``hub.docker.com``, ``ghcr.io``). URL prefix is a literal; only the
``namespace`` / ``image`` segment varies and comes from the project's
own ``meta/services.yml``, never user input. No ``file://`` scheme
exposure. Inline noqa per call site.
S602 / S605 / S606 × 8
Role-deployed scripts that legitimately use ``shell=True`` /
``os.system`` for pipes/``&&``, or ``os.execvp`` in list-form (safe by
construction — no shell parses argv). Each call site gained an inline
noqa with a per-call rationale identifying where the command string
is sourced (Ansible-templated config, host-trusted, never untrusted
external input).
Verification
============
- ``ruff check .`` → All checks passed
- ``ruff format --check .`` → 894 files formatted
- 1621 unit tests → all pass
- Integration suite: 44 baseline failures (unchanged)
23 files changed, +97 / −38. Combined with the prior commits the
branch satisfies 21 ruff rule families end-to-end. ``PTH``
(modernise ``os.path`` to ``pathlib``) is the remaining big-impact
candidate, deferred to its own session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
dcd84754f0 |
chore(ruff): enable T10/PGH/EXE/RSE/ICN/DTZ rule families — 99 → 0
Adds six more rule families to the strict ruleset: - ``T10`` — committed ``breakpoint()`` / ``pdb`` calls - ``PGH`` — pygrep-hooks (blanket ``# type: ignore`` without code) - ``EXE`` — shebang/executable-bit consistency - ``RSE`` — ``raise X()`` style cleanups - ``ICN`` — standard import aliases - ``DTZ`` — tz-naive ``datetime`` bugs Fixes (99 violations cleared) ============================= EXE001 (68): every file with a Python shebang now has the executable bit set. Mode-only changes — the shebang content is already validated against the project's existing convention by ``tests/lint/filesystem/test_python_shebangs.py`` (``#\!/usr/bin/env python3`` for portable scripts, ``#\!/usr/bin/python`` for Ansible custom modules in ``library/``). PGH003 (17): removed blanket ``# type: ignore`` markers. The project doesn't run mypy/pyright in CI, so these were decoration only. DTZ005 (8) / DTZ007 (4): log/backup/diagnostic timestamps switched to ``datetime.now(tz=timezone.utc)`` so cross-host log correlation doesn't depend on each host's local TZ. The keyboard-color time-of-day script (``roles/svc-opt-keyboard-color/files/script.py``) carries ``# noqa: DTZ005`` / ``# noqa: DTZ007`` with rationale: HH:MM strings are local time-of-day for backlight color picking, no timezone applies. The OpenSSL date parser in ``utils/cert_utils.py`` anchors the parsed naive datetime to UTC explicitly via ``.replace(tzinfo=UTC)`` (openssl always emits the literal "GMT"). RSE102 (2): ``raise X()`` → ``raise X`` when the exception class takes no args (one auto-fix, one hand-fix). Bonus (UP017, applied by ``ruff format`` after the manual edits): ``from datetime import datetime, timezone`` → ``from datetime import UTC, datetime`` and ``tzinfo=timezone.utc`` → ``tzinfo=UTC`` (Python 3.11+ shorthand). Verification ============ - ``ruff check .`` → All checks passed - ``ruff format --check .`` → 894 files formatted - 1621 unit tests → all pass - Integration suite: 44 baseline failures, 1 skipped (unchanged) 86 files changed, +55 / −37 content lines, plus 68 mode-only changes (``chmod +x``). Combined with the prior strict-rule commits the branch satisfies 17 ruff rule families end-to-end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
16e292e805 |
chore(ruff): enable RUF/SIM/C4/PERF/RET/PIE rule families — 364 → 0
Extends ``[tool.ruff.lint].select`` with six additional rule families:
``RUF`` (ruff-specific), ``SIM`` (flake8-simplify), ``C4``
(flake8-comprehensions), ``PERF`` (perflint), ``RET`` (flake8-return),
``PIE`` (flake8-pie). Adds ``RUF001/002/003`` to ``ignore`` because
those exist for homoglyph-attack detection, not English typography —
the project uses em/en-dashes and ``ℹ`` as a UI-hint glyph in CLI help
output deliberately.
Auto-fixed (~225, mechanical & behavior-preserving)
====================================================
- RUF005: ``a + [b]`` → ``[*a, b]``
- C408: ``dict(k=v)`` → ``{"k": v}``
- C420: ``dict.fromkeys(seq)`` shape simplifications
- SIM103/SIM110/SIM118: truthy-return / dict-key / dict-method
idiom simplifications
- RET504/RET505: drop unneeded final assigns / ``else`` after
``return``
- PIE810: combine ``str.startswith(a) or .startswith(b)``
into one call with a tuple
- RUF010: ``f"{str(x)}"`` → ``f"{x\!s}"``
- RUF100: removed unused ``# noqa`` directives that
became stale (mostly old ``# noqa: E402``
markers that ruff's smart E402 implementation
no longer flags after ``sys.path`` shims)
- RUF015 / RUF013 / RUF022 / RUF046 / RUF059: misc small wins
- I001: re-sorted import blocks where the new
imports above shifted them
Hand-fixed (~140, scripted where mechanical)
============================================
PERF401 (42): scripted AST rewrite of
``for x in iter: target.append(expr)`` (with optional ``if`` guard)
→ ``target.extend(expr for x in iter [if cond])``.
``list.extend(genexp)`` consumes the generator immediately, so the
rewrite is semantically equivalent to the original eager loop.
SIM117 (41): scripted AST combiner for nested ``with`` blocks where
the inner is the only statement of the outer (38 sites). The remaining
3 multi-line ``patch.object(...)`` chains were combined by hand using
the parenthesised ``with (\\n A,\\n B,\\n):`` form.
SIM102 (14): scripted AST combiner for nested ``if`` (10 sites). The
remaining 4 carry an inline ``# noqa: SIM102`` with rationale —
combining would either change semantics (the outer body has siblings
beyond the inner ``if``, e.g. ``cli/encrypt/inventory/__main__.py``
where ``encrypted_value = …`` runs regardless of ``ask_confirmation``)
or hurt readability (long literal whitelists).
RUF012 (9): annotated mutable class attributes with ``ClassVar[…]``
and added ``from typing import ClassVar``.
SIM105 (6): ``try/except: pass`` → ``with contextlib.suppress(…):``
with the necessary ``import contextlib`` added. Verified on
``utils/templating.py`` (3 occurrences in a ``finally`` block) —
cleanup behaviour is identical.
SIM115 (4): wrapped ``tempfile.NamedTemporaryFile(..., delete=False)``
call patterns in a ``with`` block; ``delete=False`` keeps the file
on disk after the block, so callers' ``return f.name`` still works.
PERF402, PERF403, B007, SIM108, SIM103: 5 misc fixes (loop→list-copy,
loop→dict-comprehension, unused-loop-var rename, ternary, return
expression).
Verification
============
- ``ruff check .`` → All checks passed
- ``ruff format --check .`` → 894 files already formatted
- 1621 unit tests → all pass
- Integration suite: 44 baseline failures, 1 skipped (unchanged)
- Sample-audited PERF401 rewrites in ``resolver.py``, ``inventory.py``,
``cache/files.py``, ``ressources/__main__.py``; SIM117 in
``test_main.py``; SIM105 in ``utils/templating.py``; SIM102 noqa
rationales in ``cli/encrypt/inventory/__main__.py``.
182 files changed, +993 / −960. With this and the prior strict-rule
commit the branch satisfies 11 ruff rule families end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
26ef3ff504 |
chore(ruff): enable strict ruleset (E/F/I/B/UP, py311) — 2819 → 0
Adds ``[tool.ruff.lint]`` with ``select = ["E", "F", "I", "B", "UP"]``
and ``target-version = "py311"``. ``E501`` is ignored because
``ruff format`` already reflows what it can; the remaining long lines
are unsplittable string literals / URLs.
Auto-fixed (2753 issues, behavior-preserving):
- UP006/UP035: ``List[X]`` → ``list[X]``, ``Dict[X]`` → ``dict[X]``,
``from typing import …`` redirected where superseded by builtins.
- UP045: ``Optional[X]`` → ``X | None``.
- UP010/UP009/UP015: dropped Python-2-era pragmas (``from __future__
import absolute_import, division, print_function``,
``# -*- coding: utf-8 -*-``), redundant ``open(..., "r")`` mode.
- UP004: dropped implicit ``class X(object):``.
- UP022/UP008/UP032: stdlib idiom modernization (``capture_output``,
super-call simplification, etc.).
- B905: ``zip(...)`` → ``zip(..., strict=False)`` (preserves prior
behavior; switching to ``strict=True`` is a separate semantic
choice not made here).
- I001: import-block ordering.
Hand-fixed (66 issues):
- B904 × 54: appended ``from <bound>`` (when the enclosing ``except``
binds a name) or ``from None`` (when it doesn't) so chained
exceptions stay distinguishable from errors-during-exception-
handling. Applied via a small AST-aware script that walked back to
the enclosing ``except`` clause and matched the binding.
- B017 × 4: narrowed broad ``assertRaises(Exception)`` to the actual
``AnsibleError`` / ``AnsibleFilterError`` raised by the code under
test.
- B023 × 2: bound captured loop variables via default args
(``def _record(name, *, _path_real=path_real, ...)``) so each
closure gets its own snapshot.
- B007 × 2: unused loop control variables renamed to ``_`` /
rewritten to ``.values()``.
- UP007 × 2: ``Union[X, Y]`` → ``X | Y`` (plus dropping the now-unused
``from typing import Union``).
- B018 × 1: Odoo addon manifests are bare dict expressions at module
level by design — suppressed with an inline ``# noqa: B018`` and
rationale comment.
- UP031 × 1: printf-style format → f-string.
The 65 ``# noqa: …`` markers introduced by the auto-fixer all carry
real ruff codes (``B018``, ``E402``, ``F401``, etc.), so
``test_noqa_only_ruff_codes`` stays green.
Verification
============
- ``ruff check .`` → All checks passed
- ``ruff format --check .`` → 894 files already formatted
- ``tests.lint.repository.{test_project_root_import,
test_no_direct_yaml_calls, test_noqa_only_ruff_codes}`` and
``tests.unit.utils.annotations.test_suppress`` → 20 tests OK
- 1621 unit tests → all pass
- Integration suite: 44 failures (pre-existing baseline, none from
this change), 1 skipped.
Going forward, the new rules block regressions on E (style), F
(pyflakes), I (import order), B (likely-bug patterns), and UP
(modernization to py311) for any new code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
4d0ebc2b91 |
refactor(suppress): migrate # noqa: <rule> to # nocheck: <rule> repo-wide
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive and warns on every non-flake8 rule key. Every catalog rule in `docs/contributing/actions/testing/suppression.md` is project-specific (kebab-case), so `# noqa:` markers fired the warning on every parse. `# nocheck:` is the project synonym ruff ignores. Scope: - 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure noqa→nocheck swap for catalog rules. Mixed flake8 markers (`# noqa: E402, X`) preserved verbatim. - 5 production .py + 2 production scripts: same swap. - All catalog-rule lints' docstrings updated; redundant "or # nocheck:" parentheticals collapsed to a single nocheck form. - `utils/annotations/suppress.py`: docstring rewritten to spell out the new convention. Parser still accepts both keywords as synonyms. - `docs/contributing/actions/testing/suppression.md`: convention rewritten to mandate `nocheck:`; missing `domain-spot` and `literal-protocol-lookup` rows added; "enforced by" pointer added. New guard: - `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+` (real ruff/flake8 shape). Allowlists the impl, its unit test, the catalog page, the direct-yaml lint, and itself. - `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to reject the `noqa:` form via inlined regex (no longer routed through `utils.annotations.suppress`). Tangential cleanups carried in this same commit: - `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml` markers swapped. - `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved up to the import block (E402). - `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined before `__main__` is imported (also fixed a latent circular import). - PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test files swapped from `def repo_root()` to `from . import PROJECT_ROOT`. - ~20 ruff auto-fix removals of unused `from pathlib import Path`. Verification: - `ruff check .` clean. - 23 suppression-related tests pass. - The new lint detects all three regression shapes (project rule alone, mixed with real ruff code, lowercase-only). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbach
|
837d0c5335 |
test(suppression): unify # noqa / # nocheck marker grammar
Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.
Helper
utils/annotations/suppress.py exposes is_suppressed_at,
is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
(case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
`<\!-- ... -->` comments, and combine multiple kebab-case rules
comma-separated. 17 unit tests cover the grammar in isolation.
Refactored consumers
- tests/external/repository/test_urls_reachable.py (rule: url)
- tests/external/docker/test_image_versions.py via
utils/docker/version_updater.py (rule: docker-version)
- tests/lint/repository/test_no_direct_yaml_calls.py (rule:
direct-yaml)
- tests/lint/ansible/test_service_shared_consistency.py (rule:
shared)
- tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
email)
- tests/lint/repository/test_python_file_size.py (rule: file-size)
- tests/lint/ansible/test_run_once_tags.py +
tests/integration/roles/run_once/test_schema.py (rules: run-once,
run-once-suffix)
Marker migration
12 role tasks/main.yml files: `# run_once_<key>: deactivated`
-> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
drops a malformed inline `noqa: intentional legacy pin` from a TODO
comment so the new parser does not pick it up as a stray rule.
Caching wins
Test files that previously loaded YAML / read files via raw
yaml.safe_load + Path.read_text now route through utils.cache.yaml
(load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
- test_service_shared_consistency.py
- test_web_app_email_integration.py
- tests/integration/roles/run_once/test_schema.py
- utils/docker/version_updater.py
This both speeds up repeated reads and stays consistent with the
direct-yaml lint policy.
Documentation
New SPOT page at docs/contributing/actions/testing/suppression.md
with grammar, position semantics, the catalog of rules, examples,
and an "adding a new rule" checklist. lint.md and external.md drop
their per-rule tables and link to the new page. All touched docs
conform to docs/contributing/documentation.md (no em-dash thought
breaks, no SPOT self-labels, RFC-2119 keywords used consistently).
Settings sort tooling
scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
and is wired into scripts/lint/autoformat.sh. The tooling fixes the
ASCII-ascending requirement enforced by
tests/lint/repository/test_claude_settings_sorted.py; the test's
failure message now points operators at the script. .claude/settings.json
is committed in sorted form.
|
||
|
kevinveenbirkenbach
|
7e10e94d7b |
Merge branch 'main' into feature/role-meta-refactor
# Conflicts: # roles/web-app-baserow/config/main.yml # roles/web-app-funkwhale/config/main.yml # roles/web-app-prometheus/config/main.yml |
||
 kevinveenbirkenbachandCopilot Autofix powered by AI
|
bae9d4a940 |
Potential fix for code scanning alert no. 280: Unused global variable
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
a8a117c86e |
refactor(yaml): route every YAML touchpoint through utils.cache.yaml + lint
New lint guard
==============
tests/lint/repository/test_no_direct_yaml_calls.py AST-walks the
production tree (utils/, cli/, plugins/, filter_plugins/,
lookup_plugins/, roles/, scripts/, tasks/) and flags any direct
`yaml.safe_load` / `yaml.safe_dump` / `yaml.load` / `yaml.dump` /
`yaml.safe_load_all` / `yaml.dump_all` call. Aliased imports are
tracked: `import yaml as Y` ⇒ `Y.safe_load(...)` is flagged;
`from yaml import safe_load` ⇒ bare `safe_load(...)` is flagged.
Per-line opt-out: `# noqa: direct-yaml` on the call's physical line
(or any line spanned by a multi-line call). No central exemption
list — every legitimate exception lives next to its call with a
human-readable rationale in the surrounding comment.
Cache module additions
======================
* `dump_yaml_str(data)` — symmetric to `safe_dump(data)` with no file
path. Used for stdout output and filter return values where the
cache cannot help but the API needs to stay consistent.
* `load_yaml_str(text)` — symmetric to `safe_load(text)`. Used when
YAML comes from an HTTP body, a mutated buffer, or a mock-patched
source whose content does not correspond to a stable on-disk file.
Migrated production call sites (33 → 0 direct-yaml violations)
==============================================================
* utils/{entity_name_utils, valid_deploy_id, invokable, service_registry,
domains/list, applications/{config, in_group_deps},
docker/image/discovery, meta/{port_bands, scan},
roles/{dependency_resolver, meta_lookup, resource_validation}}
* cli/{build/{graph, inventory/full/command}, create/inventory/{host_vars,
mirror_overrides}, fix/vars_main_files, meta/{applications/in_group_deps,
callorder}, mirror/resolver, validate/inventory}
* plugins/{filter/{compose_volumes, get_all_application_ids, get_role,
invokable_paths, role_path_by_app_id}, lookup/applications_current_play}
* roles/{svc-db-postgres/filter_plugins/split_postgres_connections,
web-app-bigbluebutton/filter_plugins/compose_mods,
web-app-dashboard/lookup_plugins/docker_cards}
Behaviour adjustments
=====================
* utils/roles/dependency_resolver — `safe_load_all` (multi-doc) →
single-doc `load_yaml_any` wrapped in a list. Ansible task files are
single-doc by convention; the downstream multi-doc loop stays intact
for forward compatibility.
* roles/sys-svc-compose-ca/files/compose_ca.py — KEEPS direct
yaml.safe_load / yaml.safe_dump (annotated with noqa). The script is
copied to the target host via `ansible.builtin.copy` and the target
has no `utils/` package on PYTHONPATH.
noqa-annotated direct-yaml call sites (with rationale)
======================================================
* utils/cache/yaml.py — IS the cache; would recurse.
* utils/handler/yaml.py — vault-aware Loader subclass that wraps
\!vault scalars in VaultScalar; cache-by-path collapses the wrapping.
* cli/encrypt/inventory/__main__.py — vault-aware Dumper subclass that
emits \!vault literal blocks; the cached dump_yaml writes plain YAML
which is the wrong output here.
* roles/sys-svc-compose-ca/files/compose_ca.py — runtime-deployed.
* tasks/utils/migrate_meta_layout.py — once-off migration script.
Validation
==========
* Unit tests: 1573 passed.
* Lint: 56 passed (incl. new test_no_direct_yaml_calls).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbach
|
07247b3552 |
Merge branch 'main' into feature/role-meta-refactor
# Conflicts: # roles/web-app-bridgy-fed/config/main.yml # roles/web-app-decidim/config/main.yml # roles/web-app-joomla/config/main.yml # roles/web-app-magento/config/main.yml # roles/web-app-shopware/config/main.yml # roles/web-app-suitecrm/config/main.yml |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
ed04dad5d5 |
refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbach
|
fab099c1c1 |
feat(version-updater): support flavored semver tags and centralise logic
Docker Official Image tags of the form `<semver>-<flavor>` (e.g. the
joomla:5.4.5-php8.3-apache pin) were previously skipped by the
`web-app` image version checker because its strict regex only matched
pure `x.y.z` shapes. As a result the auto-update workflow and the
external warning test never proposed a newer 5.4.x-php8.3-apache tag,
and flavored pins silently drifted out of date.
Widen the regex in `utils/docker/version_updater.py` to accept the
`<semver>-<flavor>` shape, with the flavor suffix treated as an opaque
discriminator: a candidate tag is only considered an upgrade when it
shares the *same* flavor. That guarantees a 5.4.5-php8.3-apache pin
never gets silently switched to 5.4.6-php8.4-apache or 5.4.6-fpm.
Refactor `tests/external/docker/test_image_versions.py` to import
directly from the util: ~130 lines of duplicated semver / Docker Hub /
GHCR helper code deleted, and the file now has exactly one code path
to keep in sync with the one used by the auto-update CLI
(`cli/fix/docker_image_versions`).
Chain impact:
* `.github/workflows/update.yml` -> `scripts/github/update_docker_image_versions.sh`
-> `python3 -m cli.fix.docker_image_versions` -> same util. No
workflow change needed; the next scheduled update run will pick
up flavored upgrades automatically.
* Live external run now reports 5 previously-silent upgrades:
web-app-joomla/joomla, web-app-magento/php, web-app-bridgy-fed,
web-app-suitecrm/php, web-app-shopware/redis.
Tests: existing `test_latest_semver_respects_depth` still covers the
pure-semver case; three new unit tests pin the flavor behaviour
(`is_semver` accepts flavored tags, `version_key/depth/flavor` strip
the flavor, `latest_semver` refuses cross-flavor upgrades).
Also drop an unused `autocreate_users: true` marker at the top of
`roles/web-app-joomla/config/main.yml` that was left over from an
earlier LDAP-autocreate experiment.
|
||
 Environment TestandClaude Sonnet 4.6
|
eebb3b9788 |
refactor(utils): move utils/docker/image_*.py into utils/docker/image/ subpackage
- Move image_discovery.py → utils/docker/image/discovery.py - Move image_ref.py → utils/docker/image/ref.py - Add utils/docker/image/__init__.py - Update all imports and references across cli/, tests/, utils/, and docs/ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Environment TestandClaude Sonnet 4.6
|
bf8c535988 |
feat(update): add docker image version updater CLI, workflow, and lint tests
- Add utils/docker/version_updater.py with CLI for updating Docker image versions - Add cli/fix/docker_image_versions module wrapping the updater - Add .github/workflows/update.yml replacing update-skills.yml - Add scripts/github/ helpers: check_git_changes, install_update_python, open_update_pr, update_docker_image_versions - Add tests/lint/filesystem/ for scripts/ and utils/ folder file-type enforcement - Add tests/lint/ansible/ for legacy module usage and vars usage checks - Reorganize tests/lint/repository/ and move test_vars_usage_in_yaml.py to ansible/ - Add artefact docs: files/github/workflow.md, scripst.md, utils.md - Apply documentation.md standards to all modified markdown files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Environment TestandClaude Sonnet 4.6
|
aa33256cde |
feat(mirror): split resolver output, move images to defaults, add lookup plugin
- resolver: output both applications: (config/main.yml compose images) and images: (defaults/main.yml images) keyed by source_file - image_discovery: scan defaults/main.yml instead of vars/main.yml for images: - mirror_overrides: write mirror URLs to applications.*.compose.services.* (mirror_policy-aware) and images_overrides.*.* (always merged, if_missing) - roles/sys-ctl-hlth-csp, roles/test-e2e-playwright: move images: from vars/main.yml to defaults/main.yml for correct Ansible variable precedence - plugins/lookup/image.py: new Ansible lookup plugin that prefers images_overrides over role-local images defaults - docs: add artefact/image.md and artefact/mirror.md as SPOTs; add code/role/image.md contributor guide; link from ci.md - tests: update test_util, test_mirror_overrides; add test_resolver, test_publish (404/401 coverage), test_image (lookup plugin) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Environment TestandClaude Sonnet 4.6
|
317287983a |
fix(mirror): resolve merge conflicts with fix/playwright-pull
Keep new multi-registry logic (feature/pre-commit):
- Merge is_mirrorable_image(), _detect_registry(), image_source() and
canonical_image_name() into utils/docker/image_discovery.py
- iter_role_images() now scans both config/main.yml and vars/main.yml
and uses image_source() for full registry-prefixed URLs
- Delete cli/mirror/util.py (superseded by utils/docker/image_discovery)
- model.py re-exports ImageRef from utils.docker.image_discovery
- sync/__main__.py: revert fix/playwright-pull's if/else src URL to
simple `src = f"docker://{img.source}"` (source already contains
the full registry prefix)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Environment TestandClaude Sonnet 4.6
|
9ea2ad6ff1 |
refactor(docker): introduce utils/docker/image_discovery as single SPOT for image enumeration
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
(test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|