Adds cli.mirror.cleanup for deleting GHCR container packages by prefix
and visibility via the GitHub Packages REST API. Supports dry-run mode.
Requires a classic PAT with delete:packages scope (GITHUB_TOKEN from
gh auth or GitHub Actions does not include this scope).
Also adds unit tests and a usage guide at
docs/contributing/flow/security/mirror-cleanup.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces GHCR_PAT-based authentication with GITHUB_TOKEN across all
mirror and build workflows. Packages pushed from a public repository
with GITHUB_TOKEN are automatically linked to the repository and
inherit its public visibility — no publish step required.
Removes cli.mirror.publish (used a non-existent GitHub REST API
endpoint for changing package visibility), the associated unit tests,
and the GHCR_PAT debug helper. Cleans up all GHCR_PAT/GHCR_USERNAME
references and rewrites the GHCR authentication documentation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace safe="/" with safe="" so package names like
mirror/docker.io/openresty/openresty are encoded as a single URL path
segment (mirror%2Fdocker.io%2Fopenresty%2Fopenresty). With literal
slashes GitHub's router matched only "mirror" as the package name → 404.
- _set_public now returns bool; main() only prints "✓ set to public" and
increments updated when True. Previously a 404-skip returned None which
still triggered the success path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old packages (pre-rename) have no versions left in GHCR and return 404
on PATCH. This caused an early exit that skipped all remaining packages.
404 means the package is gone; actual token scope failures return 401/403.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Image names now follow {owner}/{repository}/mirror/{registry}/{name}
instead of {owner}/mirror/{registry}/{name}.
- GHCRProvider: add `repository` param + `add_args()`/`from_args()` classmethods
as SPOT for CLI argument registration and provider construction
- sync, resolver: use GHCRProvider.add_args() / from_args() (no duplication)
- Workflows: resolve repository name via scripts/meta/resolve/repository/name.sh
and pass --ghcr-repository to sync and publish
- mirrors.sh: require and forward GHCR_REPOSITORY env var
- deploy/development/mirrors.py: derive repository from GITHUB_REPOSITORY
- Tests: update expected paths, argv and env fixtures
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep new multi-registry logic (feature/pre-commit):
- Merge is_mirrorable_image(), _detect_registry(), image_source() and
canonical_image_name() into utils/docker/image_discovery.py
- iter_role_images() now scans both config/main.yml and vars/main.yml
and uses image_source() for full registry-prefixed URLs
- Delete cli/mirror/util.py (superseded by utils/docker/image_discovery)
- model.py re-exports ImageRef from utils.docker.image_discovery
- sync/__main__.py: revert fix/playwright-pull's if/else src URL to
simple `src = f"docker://{img.source}"` (source already contains
the full registry prefix)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `registry` field to `ImageRef` (default `"docker.io"`)
- Populate `registry` from `_detect_registry()` in `iter_role_images()`
- `GHCRProvider.image_base()` and `GiteaProvider.image_base()` now produce
`{prefix}/{registry}/{name}` paths (e.g. `mirror/docker.io/nextcloud`,
`mirror/quay.io/keycloak/keycloak`) instead of flattening slashes to dashes
- Fix `mirror()` in both providers: use `docker://{image.source}` directly
instead of always prepending `docker://docker.io/`, which broke quay.io/ghcr.io images
- Update tests to reflect new naming convention and add quay.io test case
- Add missing __init__.py files in test subdirectories
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Keep HEAD's registry-aware src construction in sync/__main__.py
(docker://docker.io/{source} for Docker Hub, docker://{source} otherwise)
to stay compatible with image_discovery.py's ImageRef.registry field
- Accept deletion of cli/mirror/util.py (content moved to utils/docker/image_discovery.py)
- Revert test_sync.py source format to library/nextcloud:31.0.0 (HEAD format)
- Retain publish/__main__.py improvements from feature/pre-commit:
graceful 404 handling, no blind retries on client errors, pkg_url usage
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extend image mirroring to cover quay.io and ghcr.io in addition to Docker Hub,
and make the publish step non-fatal when the token lacks write:packages scope.
cli/mirror/util.py:
- Add _detect_registry(), _strip_registry_prefix(), is_mirrorable_image()
to support Docker Hub, quay.io, ghcr.io, and mcr.microsoft.com
- Update canonical_image_name() to strip any known registry prefix so that
quay.io/keycloak/keycloak -> keycloak/keycloak (mirror/keycloak-keycloak)
- Add image_source() returning full registry-prefixed pull reference
(docker.io/library/postgres:16, quay.io/keycloak/keycloak:latest, …)
- iter_role_images() now uses is_mirrorable_image + image_source, picking up
the 17 previously skipped quay.io/ghcr.io images
cli/mirror/sync/__main__.py:
- Use docker://{img.source} directly (no hardcoded docker.io/ prefix)
cli/mirror/publish/__main__.py:
- Detect HTTP 404 on PATCH as _InsufficientTokenError (no retry, graceful
GHA warning + exit 0) instead of retrying 5x and failing the workflow
- Use pkg.get("url") from listing response as canonical PATCH target URL
- Use safe="/" in urllib.parse.quote so mirror/name stays as literal path
- Do not retry on non-transient 4xx errors (400/401/403)
tests/unit/cli/mirror/test_sync.py:
- Update ImageRef.source to new docker.io/library/… format
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
(test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace templated Dockerfiles with plain Dockerfiles for web-app-littlejs,
web-app-openproject, web-app-roulette-wheel, and web-app-wordpress.
Variables are passed as Docker ARG without defaults, keeping config/main.yml
and vars/main.yml as the single source of truth. Build args are declared
in compose.yml.j2 directly after the build.yml.j2 lookup. Image and version
for roulette-wheel and littlejs are added to config/main.yml and exposed
via vars/main.yml.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move _set_public/ensure_public out of GHCRProvider into a new
cli.mirror.publish module that lists all non-public mirror packages
via the GitHub API and sets them public in bulk. The workflow runs
this as a separate step after mirroring. Mirror sync itself no longer
touches visibility.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move utils/gha/annotations.py → utils/annotations/message.py
- Move scripts/gha/summarize_annotations.py → utils/annotations/summarize.py
- Keep utils/gha/annotations.py as compatibility shim
- Update all imports across codebase to utils.annotations.message
- Wire tee + summary step into test-code-lint, unit, and integration workflows
- Add unit tests for utils/annotations/summarize (20 tests)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add utils/gha/annotations.py with warning/error/notice helpers and *_each
variants that emit one annotation per list item
- Use warning() in GHCRProvider._set_public for missing GHCR_PAT case
- Expose GHCR_PAT_SET env var in mirror step to distinguish missing-PAT from
real API failures
- Add unit tests for all annotation functions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/users/{username}/packages does not support PATCH — it is a read-only endpoint
for public data. The correct endpoint for updating visibility of the authenticated
user's packages is /user/packages (without username).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A freshly pushed package may not be immediately visible in the GitHub
Packages API. Add up to 5 retries with 10s delay before failing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause: GITHUB_TOKEN in Actions is repo-scoped and cannot change package
visibility in user/org namespaces — all 58 mirror operations failed with 404.
- Use GHCR_PAT (which has write:packages scope) as GITHUB_TOKEN in the mirror
step, falling back to GITHUB_TOKEN if GHCR_PAT is not set
- Remove ensure_public() call from the already-exists skip path (wrong place:
existing images were already made public when first mirrored)
- Update test to assert ensure_public is not called for skipped images
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove redundant `images-mirror-missing` job from entry-pull-request-change.yml
(mirroring already runs via ci-orchestrator for same-repo PRs)
- Set GHCR package visibility to public after each successful mirror via GitHub API
- Add `ensure_public` hook to base class (no-op for other providers)
- Also call `ensure_public` for already-existing images to fix any that are still private
- Pass GITHUB_TOKEN explicitly to the mirror step
- Add unit tests for _set_public (user/org endpoint fallback) and sync skip-path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Introduce reusable workflow `mirror-images-missing.yml` to mirror only missing
Docker Hub images to GHCR using `--only-missing`
- Scope mirror concurrency per branch using `github.ref_name`
- Rename full mirror workflow to `mirror-images-all.yml` for clarity
- Integrate missing-image mirror into CI orchestrator and gate server/universal
deploys on successful mirror completion
- Add tag existence checks via `skopeo inspect` to avoid redundant uploads
- Keep mirror runs best-effort to avoid blocking CI on transient registry issues
https://chatgpt.com/share/698141e9-0844-800f-9c4f-fecd468ede73
- Add required GHCR mirror environment variables to CI workflows
- Generate mirrors.yml during development inventory init on CI
- Introduce mirrors helper module with strict env validation
- Minor formatting cleanup in mirror providers
`
https://chatgpt.com/share/6980ff2e-8f18-800f-85e1-79d56a2027f5
- mirror sync now continues on individual skopeo failures
- collect and summarize failed images
- return non-zero exit code only after all images were processed
- make GitHub workflow respect mirror sync exit status
https://chatgpt.com/share/6980b558-5fdc-800f-911e-48bbc35a461f
