computer-playbook/docs/contributing/artefact/image/mirror.md

Image Mirroring 🪞

This document explains why image mirroring exists, how it works, and what each component does. For how images are declared in roles, see origin.md.

Why Mirrors Exist 🎯

Upstream registries (Docker Hub, quay.io, mcr.microsoft.com, ghcr.io) impose rate limits and occasionally have availability issues. CI pipelines pulling images directly from upstream are fragile: a rate limit hit or a temporary outage fails an otherwise healthy build.

To solve this, all upstream images used in the project are mirrored to GHCR (ghcr.io) under the project namespace before CI deploy tests run. Test jobs pull from the mirror and never from the upstream registry directly.

This also enables fork PRs to pull images without needing upstream credentials, because the mirror is public.

Naming Convention 📛

Mirrored images follow this scheme:

ghcr.io/{namespace}/{repository}/mirror/{registry}/{name}:{version}

Segment	Example
namespace	kevinveenbirkenbach
repository	infinito-nexus-core
registry	docker.io, quay.io, mcr.microsoft.com, ghcr.io
name	nextcloud, keycloak/keycloak, prom/prometheus
version	latest, 31.0.0

Example: ghcr.io/kevinveenbirkenbach/infinito-nexus-core/mirror/docker.io/nextcloud:production-fpm-alpine

Components 🧩

Image Discovery 🔍

utils/docker/image/discovery.py discovers all role images via iter_role_images(). See origin.md for the declaration format and supported registries.

GHCRProvider 🐳

cli/mirror/providers.py contains the GHCRProvider class, which computes the destination image name via image_base(img) and provides add_args / from_args so all CLI tools share a single argument definition (SPOT).

Resolver 🗺️

cli/mirror/resolver/__main__.py reads all role images and outputs a mirrors.yml file with separate sections for compose images and role-local flat images:

applications:
  <role>:
    compose:
      services:
        <service>:
          image: ghcr.io/{namespace}/{repository}/mirror/{registry}/{name}
          version: <tag>
images:
  <role>:
    <service>:
      image: ghcr.io/{namespace}/{repository}/mirror/{registry}/{name}
      version: <tag>

The split matches the source declaration:

• config/main.yml image refs stay on the existing applications.*.compose.services.* path
• defaults/main.yml image refs are emitted under images.*.*

This file is consumed by the inventory creator to substitute mirror URLs into host variables.

Sync 🔄

cli/mirror/sync/__main__.py copies each image from the upstream source to the GHCR mirror destination using skopeo copy. Supports --only-missing to skip already-mirrored images and --images-per-hour for throttling.

Wait Script ⏳

scripts/meta/wait/mirrors.sh waits for fork PRs until all required mirror images are available in GHCR before letting the deploy tests proceed. This is necessary because fork PRs cannot push images themselves; the pull_request_target event handles that.

CI Integration 🔄

The mirror workflow runs as stage 8 of the CI pipeline. See ci.md. It runs in parallel with the DNS tests and MUST complete before deploy tests start.

The mirrors file is generated into the inventory directory. The inventory creator applies mirror image overrides to host variables via cli/create/inventory/mirror_overrides.py, which reads both top-level keys from mirrors.yml and writes into host vars as follows:

• applications.{role}.compose.services.{service}.image: Populated from mirrors.yml.applications and still subject to mirror_policy.
• images_overrides.{role}.{service}.image: Populated from mirrors.yml.images and consumed by lookup('image', ...).

Mirror Policy 📋

Each service in host vars MAY carry a mirror_policy field that controls how the override is applied:

Policy	Behavior
if_missing (default)	Fill image/version only if they are blank or absent
force	Always overwrite image/version from the mirror
skip	Never touch this service

Adding a New Mirrored Image 🆕

No manual registration is needed. Images declared in a role are automatically discovered and included in the next mirror run. See origin.md for the correct declaration format.

Cleanup 🗑️

To delete stale private GHCR mirror packages (e.g. packages pushed before GITHUB_TOKEN authentication was adopted), see cleanup.md.