The daily mirror waited about 70 s before each of its 195 copies (50 images
per hour), roughly 3.8 h of pauses, and logged no single rate-limit error.
37 of the copies come from ghcr.io, registry.gitlab.com, quay.io,
registry.opencode.de and mcr.microsoft.com, which the Docker Hub pull limit
does not cover, and --only-missing also waited before images it then skipped.
The throttle now waits only before an actual copy from docker.io, and the
schedule allows 150 Docker Hub copies per hour (24 s apart).
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scan threshold becomes a severity-to-verdict mapping in
roles/sys-ctl-hlth-runtime/meta/services.yml, CRITICAL: fail and HIGH: warn,
for production and CI alike. A finding mapped to fail fails the service and
the deploy, a warn prints a warning line and passes, a suppressed one prints
an info line. Engine errors, missing databases and PHP hardening violations
still fail. The old list form is rejected with a config error. The daily
mirror scan stays strict: every unsuppressed fail or warn finding is a row,
so it fails while an image is red.
Every run publishes one list of judged findings four ways:
- report.json of the latest run (summary + records) under
DIR_HEALTH_RUNTIME_REPORT, written atomically;
- findings.jsonl under DIR_HEALTH_RUNTIME_LOG, one record per finding,
appended per run and rotated weekly by a logrotate drop-in;
- one JSON line per target plus a summary line on stdout for the journal;
- infinito_hlth_runtime.prom in DIR_METRICS_TEXTFILE when web-app-prometheus
is deployed: findings per target, kind, image, severity, scanner and
verdict (no CVE label), last run time and success, scanner errors.
Each record carries run_id, timestamp, host, environment, target, kind,
image, scanner, id, severity, package, installed_version, fixed_version,
suppressed, suppressed_until and verdict. The paths are SPOT variables in
group_vars/all/05_paths.yml, the script takes them as arguments.
Requirement 070 and the role README describe the mapping, the outputs and
the per-image vulnerability issue of the previous commit.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dry-run mirror scan (run 41286420589) reported 10,179 CVE/image pairs
across 135 red digests; one issue per pair would flood the tracker. The
sync now keeps exactly one open issue per pinned image:
- The body lists every CVE row (id, severity, engine, role, service,
package, installed, fixed), CRITICAL first, and stops below 60,000
characters with the number of rows left out and a pointer to the
vulnerability-findings artifact. On the dry-run data: 135 issues, 8
capped, the largest body 59,872 characters.
- A further open issue with the same title closes as a duplicate of the
oldest one.
- When an image's CVE table changed, the issue gets a "Still unresolved"
comment with the table and an updated body; an unchanged table leaves the
issue alone, so a daily run does not add 135 identical comments.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL (py/call-to-non-callable, alerts 1073 and 1074 on PR #1326) could not
see the __call__ of the HangingEdit class defined inside the test method and
flagged both gh() calls in sync(). FakeGh now takes the verbs to hang as a
parameter, so the test double stays one module-level class.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeRabbit review of PR #1326:
- fetch_databases downloads each OSV archive to all.zip.part and replaces
all.zip only after the transfer completes. An OSError removes the partial
file and comes back as an `osv <ecosystem>: <error>` line instead of a
traceback.
- run_gh bounds every gh call with a 120 s timeout. A failed or hung create,
edit or close becomes a `failed <action>: <error>` line while the other
issues and the close pass still run, and the command exits 1.
- The suppression lint checks every list item of a suppressions block, so an
entry that starts with `until:` still needs its `# nocheck: cve <reason>`
marker.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The daily mirror workflow gains a scan job after the sync. It runs
make test-security against the GHCR mirror, writes the failing findings per
image and per CVE into the job summary and uploads the report as the
vulnerability-findings artifact.
- One issue per CVE and image, labelled vulnerability, is opened, updated
while the finding stays and closed once no engine reports it. Issues are
only closed when the scan reported no error, so an image that failed to
scan keeps its issues.
- workflow_dispatch and workflow_call take an issues switch; false is a dry
run that only produces the summary and the artifact. Scheduled runs always
sync the issues.
- Only the scan job holds issues: write.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each CI deploy row now draws one vulnerability engine (trivy, grype or osv)
as a fifth rotation axis, shown in the job name next to distro and
filesystem (🔬 axis, 🔺 trivy, 🍇 grype, 🐙 osv). The deploy-time scan of
sys-ctl-hlth-runtime runs that engine behind Syft and the Grype runtime pass.
- A vulnerability-db job keeps one Actions cache per database and day and
downloads only the missing ones. Every deploy job restores Grype plus its
drawn engine, fails on a cache miss and never downloads a database itself.
- Local stacks share one host directory, INFINITO_VULNERABILITY_DB_HOST_PATH,
mounted read-only at INFINITO_VULNERABILITY_DB_DIR in compose.yml and in
every swarm node. A local deploy without a pinned engine draws one and
refreshes only the databases it reads, at most once a day, under a lock.
- A pinned engine whose database is missing stops the deploy before it
starts.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runtime check judged PHP by its upstream version and failed every
container on an end-of-life line. Distribution builds backport fixes without
raising that number, so main run 41154895895 failed Checkmk#0 on
`PHP 8.1.2-1ubuntu2.28 runs the end-of-life line 8.1`, a build Ubuntu lists
as fixed for CVE-2026-109618 and CVE-2026-109619.
- The role now runs Trivy, Grype and OSV-Scanner, with a Syft SBOM per
target, against the host and the images of the running containers, in
bounded parallel lanes. Targets, engines and the failing severities are
lists in meta/services.yml; the default is every target and engine at
CRITICAL and HIGH with a fix available.
- Grype always scans the binary-classified packages of the SBOM, so a PHP
built from source in an official php image (8.4.26) still fails on the
CVEs fixed in 8.4.27, which Trivy and OSV do not see.
- OSV reads an SBOM copy whose distro purls carry the source package name.
- A CVE is suppressed only in the role owning the image, under
services.<service>.security.suppressions with id, until and a
`# nocheck: cve <reason>` marker; a lint rejects bare, malformed and
expired entries.
- The deploy-time run fails on its first finding: sys-service gains
system_service_run_once and sys-service-terminator skips the retries
only for those units. The timer runs daily at 00:00 UTC.
- The PHP version floors are removed; the php_hardening check stays.
- make test-security replaces make security-runtimes: Syft builds an SBOM
of every pinned image straight from the GHCR mirror, cached by digest,
and the three engines scan it with the role's threshold and suppressions.
Part of #1324
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three CodeRabbit findings on PR #1265:
- script.py listed the running containers with `container ps` and
`container inspect` without a timeout, so a hung engine kept the daily
unit active forever and blocked the next timer run. Both calls now use
EXEC_TIMEOUT_SECONDS; main turns a timeout or engine error into a
one-line failure and exit 1 instead of a traceback.
- runtime.py's PHP_PACKAGE_PATTERN requires a digit after "php", so an
image installing an unversioned interpreter package (`php-cli`) and
setting no PHP_VERSION was reported as PHP-free. The interpreter
packages php, php-cli, php-fpm, php-cgi, php-phpdbg and
libapache2-mod-php are now recorded as PhpRuntime.packages and reported
with an unknown patch level; extension and library packages
(php-mysql, php-pear, php-common, phpmyadmin) and paths do not count.
- php_finding returned early for a patched PHP_VERSION, so an end-of-life
package line installed next to it (e.g. php7) was never reported. The
version only returns a finding now; otherwise the lines are judged too.
Every finding has unit tests that fail on the previous code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Answering "which deployed containers run a vulnerable PHP?" after
CVE-2026-109618/109619 took a manual inventory of registry metadata, and
nothing noticed a hardening ini that an image never loads.
- New role sys-ctl-hlth-runtime, routed through sys-svc-compose next to
sys-ctl-hlth-container. A daily timer runs `php` in every running
container and fails, alerting through sys-ctl-alm-compose, when PHP is
below the floor of its line or on a line older than every floor (7.4,
8.1). Every service that mounts php_hardening in meta/volumes.yml fails
while expose_php or session.upload_progress.enabled is on; other PHP
containers of such a stack are listed as info. Compose and swarm
containers are matched by their project/stack and service labels. The
check is not run at deploy time, so a finding never fails a deploy.
- The floors (8.2.35, 8.3.36, 8.4.27, 8.5.12) live once, in
HEALTH_RUNTIME_FLOORS of the role's vars/main.yml.
- `make security-runtimes` lists the PHP of every pinned role image
without pulling: Docker Hub tags through the Hub API, which no pull rate
limit meters, other registries through the OCI config blob (new
registry.config_blob, which the healthcheck probe now reuses).
PHP_VERSION, distro PHP packages and a PHP_BASE build argument count. It
exits 1 on an image below its floor or one the registry did not answer.
Today it flags nfrastack/fusiondirectory:1.5 (line 7.4) and
shopware/docker-base:8.3 (8.3.25).
The role TODO.md keeps the follow-ups: further runtimes, the split web
SAPI of web-app-lam, a scheduled CI run of the report and the CVE scan.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.
iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.
Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5.5