Commit Graph
15 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5 c693f4f619 style(quality): apply what the gate wrote back into the tree
Nothing here was typed by hand. The docs generator rebuilt the MCP audit
table against pins that had already landed, so fider reads v0.36.0 instead
of stable, jenkins 2.568.3-lts instead of lts, pretix 2026.7.0 instead of
stable, and flowise 24-slim instead of 24-bookworm-slim. That is the table
catching up with da665ab69 and 10c5b35e8, reading the same declaration the
updater reads.

Autoformat wrapped three test files from the same series: one condition
pulled onto a single line, three assertEqual calls broken across lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:35:05 +02:00
kevinveenbirkenbachandClaude Opus 5 0c018fa318 test(update): require every image pin to be orderable or declared
The ranking the last commits built toward is now enforced: a semver tag,
else a digest, and nothing else without a written reason. Two states fail.
A pin that is neither names something the registry can move underneath us,
and nothing here can tell whether it still points where it did when it was
written. A digest on an upstream that does publish semver takes the
fallback for no gain and hides the entry from the update job.

Thirteen entries stood in the way when the check was first run. Four were
resolved by the channel grammar, which made the Jitsi components orderable
without touching their pins. Three needed nothing once a moving tag could
declare itself with 'docker-version'. Two are fixed here and four are
declared.

LiteLLM leaves 'main-v1.77.3.dynamic_rates'. That tag exists exactly once
among 2609, so it is a dead end no updater can follow, and it misreports
itself: the image inside answers 1.77.2. The pin moves to
'main-v1.77.7-stable', five patches along the same minor and onto a channel
with 134 builds. Started against this repository's config shape, 1.77.7
registers both model entry forms and accepts drop_params and the
os.environ master key.

Pixelfed leaves 'latest' for '20260917'. The upstream stamps its dates as
one eight-digit number, which is depth 1 and orderable with no tooling
change at all, and the two tags resolve to the same digest, so nothing
about the deployed image moves.

The four declarations carry the count that justifies them rather than a
claim: LM Studio publishes two tags and neither is a version, the MIG
registry carries one, and the two Nextcloud Talk images publish 174 and 104
AIO build stamps between them.

Both branches of the gate were negative-controlled. The digest branch
skips in this repository because no digest pin exists at all, so it was
exercised by introducing one and watching it fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:50:27 +02:00
kevinveenbirkenbachandClaude Opus 5 be6a25517c feat(docker): classify image pins and build pull references from the class
A digest pin was emitted as `name:sha256:...`, which no registry accepts:
image_source('ghcr.io/.../csp-checker', 'sha256:d559...') returned
'ghcr.io/.../csp-checker:3.0.0'. The separator depends on the pin class, so
the class has to be decided before the reference is built.

utils/docker/image/pin.py names the three classes in preference order
(semver, then digest, then a general ref that no registry can order) and
maps each to the nocheck marker that declares it. Both discovery entry
points now end in pull_reference(), so '@' versus ':' is decided in one
place instead of at every call site.

The semver pin lint points a digest at 'docker-digest' instead of
'docker-version', so the two fallbacks stay distinguishable in the
suppression catalog. It stays warn-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 16:39:16 +02:00
kevinveenbirkenbachandClaude Opus 5 2608cb08d6 test(lint): fail closed on five properties that used to pass silently
Each of these is invisible because every individual setting is valid on its own. A sandboxed workload and the host container socket are both legitimate, only their combination is not: the socket hands over the host daemon, which can start a privileged container on the host kernel, and the isolating runtime becomes decoration while the deploy stays green. An embodiable agent that forgets the robot admission include deploys privileged onto a shared node and nothing says a word, and the include cannot live in svc-ai-robot because the service loader preloads it a stage earlier.

A provider bearer handed to a browser through playwright.env.j2 lands in the retained trace, its headers, its environment dump and its network log, and outlives the deploy that minted it inside a downloadable CI artifact. A served surface without its own mcp RBAC role leaves a client nothing to scope a grant to, and the fallback is not nobody: an empty grant reads as every administrator. An image bump to an amd64-only tag turns an arm64 promise into a scheduling failure on the Pi rather than a red test here, because platform_arch exists to pin a role to one architecture and a role that must run everywhere declares nothing.

Four of the five derive their scan from declarations rather than a list, and each carries a vacuum guard that fails when its scan comes back empty, since a rule whose derivation broke is otherwise a rule that passes forever.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:56:52 +02:00
kevinveenbirkenbachandClaude Opus 5 700d537855 refactor(mcp): pin the adapter base by tag where the mirror looks
The adapter was the only image in the tree pinned by a hand-maintained digest,
and the pin was never recomputed by anything. It drifted twice: run 31374959721
lost seven of eight jobs to mirror/docker.io/python@sha256:6771159c... not
found, and the renewal to the newest upstream digest earlier today would have
reproduced it, because builds resolve FROM against the GHCR mirror while the
mirror is copied by tag and lags upstream by design.

The base is now FROM <image>:<version>, taken from the sidecar service in
meta/services.yml - the only file iter_role_images scans, so the tag the build
asks for is by construction the tag the mirror was told to copy. The version
moves from the moving 3.13-slim to 3.13.15-slim, which upstream stops rewriting
once published, and which the existing version drift job renews like every other
image.

With the build reading services.yml, the adapter block's digest, image and
version had no consumer left. They are removed together with the schema check
for the digest form, the warn-only drift test, and the manifest_digest helper
that existed only for it. What remains in the block describes the adapter rather
than the image it is built from.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 23:38:53 +02:00
kevinveenbirkenbachandClaude Opus 5 55f10c459e fix(mcp): pin the digest the mirror serves, not the newest upstream one
Reverts the twelve pins to 9662417a, the digest the GHCR mirror currently
carries for python:3.13-slim. The previous commit renewed them to the newest
Docker Hub digest, which is the wrong registry: builds resolve FROM against the
mirror whenever a mirror prefix is configured, and the mirror is copied by tag,
so it holds whatever the tag pointed at when the sync last ran. A pin ahead of
the mirror resolves to nothing there while a local build keeps working - run
31374959721 lost seven of its eight jobs to exactly that, as
mirror/docker.io/python@sha256:6771159c...: not found.

The external check measured Docker Hub and would therefore have kept advising
the change that breaks the build. It now resolves the source the build uses via
mirror_image, asks whether the pin exists there at all, and only falls back to
upstream when no mirror is configured.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:48:38 +02:00
kevinveenbirkenbachandClaude Opus 5 214e7624bc fix(mcp): pin what the adapters actually build and serve
Three fields pinned artefacts and nothing recomputed any of them.

The adapter base digest was stale for all twelve provider roles: the tag
python:3.13-slim moved to ffb752e1 while every role still pinned 9662417a.
The GHCR mirror copies by tag, so the pinned manifest is one it was never
asked to copy - a mirrored build resolves FROM to nothing while a direct
build keeps working.

adapter.specification_sha256 matched no revision of the file it pins in ten
of twelve roles; three of the old values were byte-identical to the same
role's tools.schema_sha256, so they had been copied rather than computed.
The field is enforced nowhere at runtime, which is exactly why it needs a
check.

Add a hard lint for both offline pins, computing the tool digest with the
adapter's own policy.schema_digest so it cannot drift from the code that
enforces it in production, and a warn-only external check that reports a
base digest the upstream tag no longer resolves to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 19:14:47 +02:00
kevinveenbirkenbachandClaude Opus 5 f3963dcbdd test(storage): gate min_storage against the image size it is declared for
An over-declared min_storage makes a host look too small: it feeds the storage guard, the runner packer and the sufficient-storage filter, so a deploy that would have fit prunes a cache it did not need to lose. Nothing checked the declared number against reality.

The new external test measures what a pull of the service's own image:version actually transfers - config blob plus every layer blob, resolved for linux/amd64 - and fails when min_storage exceeds it by more than 1GB. A service whose data volume genuinely dominates keeps its value by carrying '# nocheck: min-storage-headroom' together with a 'Reason:', the same two-marker shape meta/volumes.yml already uses for nfs: false.

Sizes come from the repo's own registry client rather than 'docker buildx imagetools': make test-external runs inside the compose container, which has no docker socket. manifest_transfer_size reuses the existing bearer-challenge flow, keeps the tri-state contract of manifest_exists (an indeterminate registry warns instead of failing), and resolves a multi-platform index to its amd64 manifest - the case a raw jq over '.layers' would miss entirely. Distinct image refs are measured concurrently through resolve_max_fetch_workers().

A min_storage that parses out of the YAML but cannot be located in the file text fails the test, so a declaration can never silently lose its marker resolution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1b774e410c feat(update): support every registry via a generic v2 client
Add a generic Docker Registry v2 client (WWW-Authenticate bearer-token
challenge) so tag listing and reachability work for any registry (Docker
Hub, GHCR, MCR, Quay, GitLab/opencode), and route the version updater
through it so quay.io and opencode.de images auto-update like the rest.
Add an external test that hard-fails when a pinned image:version tag is
unreachable (404) and warns on incomplete image/version pairs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbach 9ed04c22ab Revert "feat(test): add live registry-reachability probe for image pins"
This reverts commit 63c6912fae.
2026-05-23 07:00:51 +02:00
kevinveenbirkenbachandClaude Opus 4.7 63c6912fae feat(test): add live registry-reachability probe for image pins
Closes the gap left by test_image_versions, which silently skips any
non-semver pin (latest, stable, branch tags). The new test_image_reachable
walks every entry from iter_role_images and probes the Docker Registry v2
manifest endpoint via HTTPS with anonymous bearer-token negotiation
(docker.io, ghcr.io, quay.io, mcr.microsoft.com). HTTP 404 hard-fails as
a broken pin; auth/rate-limit/5xx/network errors warn. Bounded by per-
request, per-probe, and global deadlines and parallelized via
INFINITO_WORKER_FETCH.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 05:31:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19f9735909 feat(test/external): bound runtime + share parallel update pipeline
The URL-reachability probe now bounds its wall-clock end-to-end:
allow_redirects=False (a redirect chain previously stacked
timeouts), a 5h30m global deadline that marks in-flight probes
as warnings on expiry, and live `[done/total]` progress every 50
probes. Submission order is shuffled so same-host URLs no longer
batch at the front and self-DoS one origin.

The Docker image-version test drops its bespoke sequential
fetch loop and calls find_outdated_updates() from utils.update.docker,
so it now shares the INFINITO_WORKER_FETCH-sized worker pool with
the repository-refs check.

`scripts/tests/code/wrapper.sh` propagates BASH_ENV into the
container exec — without it INFINITO_WORKER_FETCH was unset
inside the container and the URL probe collapsed to 1 worker.
`make test` gains test-external; `make dotenv-force` regenerates
.env with a stripped environment so stale BASH_ENV-preloaded
INFINITO_* values cannot pin old numbers via setdefault.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 83466868e9 feat(update): support mcr.microsoft.com in docker version checker
- Add MCR_REGISTRY constant, is_mcr / mcr_repo / fetch_mcr_tags helpers
  mirroring the Docker Hub and GHCR pattern. Tags are fetched via the
  Docker Registry V2 API with RFC 5988 Link-header pagination (anonymous
  access; MCR is a public read mirror). Wire MCR into both the external
  test_image_versions warning surface and the find_outdated_updates
  orchestrator in utils.update.docker.
- Bump mcr.microsoft.com/playwright in test-e2e-playwright from
  v1.59.1-noble to v1.60.0-noble — the update was surfaced by the new
  MCR detection path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 14:02:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 98457f98fd refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.

utils/
  update/
    base.py        # semver primitives shared by both backends
    docker.py      # OCI registry lookups + version: line rewriter
    repository.py  # git ls-remote walker + ref: line rewriter

cli/contributing/update/
  docker/           # python -m cli.contributing.update.docker
  repository/       # python -m cli.contributing.update.repository

scripts/github/update/
  image_versions.sh
  repository_refs.sh

Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.

Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00