feat(sys-os): declare the workspace runtime images where the updater finds them

The runtime image pins lived in meta/distros.yml, which neither the docker
updater, the image mirror nor the vulnerability scan reads, so a pin would
only ever move by hand. The new role sys-os carries them in its
meta/services.yml under the distro id; utils/distros.py renders the
workspace matrix from there with the stdlib block reader, and
meta/distros.yml keeps the distro attributes only.

A new release now arrives as an updater pull request. follow: latest holds
debian, ubuntu and fedora to the release their latest tag points at, by
Docker Hub digest: Ubuntu moves between LTS releases only (26.10 shares the
digest of rolling, not latest) and Fedora skips rawhide (46 shares the
digest of rawhide, latest is 45). Arch stays rolling; CentOS stream tags
carry no number the updater can order.

dockerhub_repo and is_dockerhub move to utils/docker/image/ref.py beside
split_registry_and_name, so utils/update/follow.py can use them without an
import cycle and utils/update/docker.py stays under the size cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5.5 committed 2026-11-03 19:55:32 +01:00
1 parent f12658927f
commit 0ed50ae0a5
9 files changed
+245 -35

No files matched your search

-8
View File
@@ -20,9 +20,6 @@
# slug path segment inside the image reference
# family ansible_facts['os_family'] the distribution reports,
# consumed by the meta/packages.yml registry
# dev_runtime_image upstream base image the development-environment test
# matrix boots the distribution from. Not one of ours,
# so it carries its own registry and tag.
# galaxy_platform platform name Ansible Galaxy knows the distribution by,
# used to build the canonical galaxy_info.platforms list
# every role's meta/main.yml must carry. The versions
@@ -41,25 +38,20 @@ distros:
arch:
slug: arch
family: Archlinux
dev_runtime_image: manjarolinux/base
galaxy_platform: ArchLinux
debian:
slug: debian
family: Debian
dev_runtime_image: debian:13
galaxy_platform: Debian
ubuntu:
slug: ubuntu
family: Debian
dev_runtime_image: ubuntu:26.04
galaxy_platform: Ubuntu
fedora:
slug: fedora
family: RedHat
dev_runtime_image: fedora:44
galaxy_platform: Fedora
centos:
slug: centos
family: RedHat
dev_runtime_image: quay.io/centos/centos:stream10
galaxy_platform: EL
+40
View File
@@ -0,0 +1,40 @@
# Operating System Images
## Description
The operating system images are the upstream container images of [Arch Linux](https://archlinux.org/) (through [Manjaro](https://manjaro.org/)), [Debian](https://www.debian.org/), [Ubuntu](https://ubuntu.com/), [Fedora](https://fedoraproject.org/) and [CentOS Stream](https://www.centos.org/centos-stream/), each published per distribution release.
## Overview
This role deploys nothing. Its `meta/services.yml` pins the image and release each distribution's development environment boots from, keyed by the distribution id of [meta/distros.yml](../../meta/distros.yml), and `utils/distros.py` reads the workspace matrix from it. Because the pins live in a role's `meta/services.yml`, the docker updater proposes new releases as pull requests and the image mirror and the vulnerability scan cover them like every other role image. `follow: latest` holds a pin to the release the `latest` tag points at, so Ubuntu moves between LTS releases only and Fedora skips rawhide. Arch stays rolling, and CentOS Stream tags carry no number the updater can order, so its next stream is a manual change.
## Cosmos
The diagram places Operating System Images in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks).
```mermaid
flowchart LR
subgraph role [sys-os 💻]
svc_arch["arch"]
svc_debian["debian"]
svc_ubuntu["ubuntu"]
svc_fedora["fedora"]
svc_centos["centos"]
end
```
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
## Features
- One `image` and `version` per distribution id of `meta/distros.yml`
- Workspace test matrix rendered from these pins by `utils/distros.py`
- New releases proposed as updater pull requests
- `follow: latest` keeps Ubuntu on LTS releases and Fedora off rawhide
- Images mirrored and scanned like every other role image
## Credits
Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**.
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
+32
View File
@@ -0,0 +1,32 @@
---
galaxy_info:
author: Kevin Veen-Birkenbach
description: "Pins the upstream operating system images the development environment boots each distribution from"
license: Infinito.Nexus Community License (Non-Commercial)
company: |
Kevin Veen-Birkenbach
https://www.veen.world
min_ansible_version: "2.9"
platforms:
- name: ArchLinux
versions:
- all
- name: Debian
versions:
- all
- name: EL
versions:
- all
- name: Fedora
versions:
- all
- name: Ubuntu
versions:
- all
galaxy_tags:
- os
- distribution
- image
- infinito
- automation
issue_tracker_url: https://s.infinito.nexus/issues
+23
View File
@@ -0,0 +1,23 @@
---
# nocheck: required-by-coverage -- read by utils/distros.py and the docker updater; no role requires it
os:
lifecycle: beta
arch:
image: manjarolinux/base
# nocheck: docker-version -- Manjaro publishes rolling builds only, so Arch stays rolling
version: latest
debian:
image: debian
version: "13"
follow: latest
ubuntu:
image: ubuntu
version: "26.04"
follow: latest
fedora:
image: fedora
version: "44"
follow: latest
centos:
image: quay.io/centos/centos
version: stream10
@@ -0,0 +1,50 @@
from __future__ import annotations
import unittest
from unittest import mock
from utils.update.follow import followed_tags
UBUNTU = {
"latest": "sha256:lts",
"26.04": "sha256:lts",
"26.10": "sha256:interim",
"rolling": "sha256:interim",
}
FEDORA = {
"latest": "sha256:45",
"44": "sha256:44",
"45": "sha256:45",
"46": "sha256:rawhide",
"rawhide": "sha256:rawhide",
}
def _digests(table: dict[str, str]):
return lambda _image, tag: table[tag]
class TestFollowedTags(unittest.TestCase):
def test_ubuntu_on_the_lts_gets_no_interim_release(self) -> None:
with mock.patch("utils.update.follow.tag_digest", _digests(UBUNTU)):
found = followed_tags("ubuntu", list(UBUNTU), {"follow": "latest"}, "26.04")
self.assertEqual(found, [])
def test_fedora_moves_to_the_stable_release_not_rawhide(self) -> None:
with mock.patch("utils.update.follow.tag_digest", _digests(FEDORA)):
found = followed_tags("fedora", list(FEDORA), {"follow": "latest"}, "44")
self.assertEqual(found, ["45"])
def test_an_entity_without_follow_keeps_every_tag(self) -> None:
tags = list(FEDORA)
self.assertEqual(followed_tags("fedora", tags, {"version": "44"}, "44"), tags)
self.assertEqual(followed_tags("fedora", tags, None, "44"), tags)
def test_an_unreadable_registry_proposes_nothing(self) -> None:
with mock.patch("utils.update.follow.tag_digest", side_effect=OSError):
found = followed_tags("fedora", list(FEDORA), {"follow": "latest"}, "44")
self.assertEqual(found, [])
if __name__ == "__main__":
unittest.main()
+14 -7
View File
@@ -2,9 +2,11 @@
``meta/distros.yml`` declares every distribution the project builds, tests
and deploys on: the id used as ``INFINITO_DISTRO``, the ``os_family``
ansible reports for it, the Galaxy platform name, the upstream base image
the development-environment matrix boots it from, and the templates the
ansible reports for it, the Galaxy platform name, and the templates the
pkgmgr base image and the environment image references are rendered from.
The upstream image the development-environment matrix boots each distro
from is pinned in ``roles/sys-os/meta/services.yml`` under the distro id,
where the docker updater, the mirror and the vulnerability scan find it.
Consumers MUST read the distro set through this module. A local tuple of
distro names or an f-string spelling out ``ghcr.io/.../pkgmgr-...`` is a
@@ -26,9 +28,11 @@ from __future__ import annotations
from utils.cache import PROJECT_ROOT
from utils.cache.files import read_text
from utils.roles.mapping import ROLE_FILE_META_SERVICES
from utils.yaml_bootstrap import load_block
FILE_META_DISTROS: str = "meta/distros.yml"
RUNTIME_IMAGES_ROLE: str = "sys-os"
IMAGE_PKGMGR = "pkgmgr"
IMAGE_PKGMGR_VIRGIN = "pkgmgr_virgin"
@@ -82,21 +86,24 @@ def galaxy_platforms() -> list[dict[str, object]]:
def dev_runtime_images() -> tuple[str, ...]:
"""Upstream base images of the development-environment test matrix."""
return tuple(spec["dev_runtime_image"] for spec in _specs().values())
return tuple(dev_runtime_image(name) for name in distro_names())
def dev_runtime_image(distro: str) -> str:
"""Upstream base image the development environment boots a distro from."""
return _spec(distro)["dev_runtime_image"]
_spec(distro)
services = PROJECT_ROOT / "roles" / RUNTIME_IMAGES_ROLE / ROLE_FILE_META_SERVICES
entity = load_block(read_text(str(services)))[distro]
return f"{entity['image']}:{entity['version']}"
def distro_of_dev_runtime_image(image: str) -> str:
"""Distro id whose development environment boots from ``image``."""
for name, spec in _specs().items():
if spec["dev_runtime_image"] == image:
for name in distro_names():
if dev_runtime_image(name) == image:
return name
raise UnknownDistroError(
f"{image!r} is not a dev_runtime_image in {FILE_META_DISTROS}. "
f"{image!r} is not a runtime image of {RUNTIME_IMAGES_ROLE}. "
f"Declared: {', '.join(dev_runtime_images())}."
)
+18
View File
@@ -50,3 +50,21 @@ def split_registry_and_name(image: str) -> tuple[str | None, str] | None:
return first, rest
return None, image
def is_dockerhub(image: str) -> bool:
parsed = split_registry_and_name(image)
if parsed is None:
return False
registry, _name = parsed
return registry is None or registry in DOCKER_HUB_REGISTRIES
def dockerhub_repo(image: str) -> str:
parsed = split_registry_and_name(image)
if parsed is None:
raise ValueError(f"Invalid Docker image reference: {image!r}")
registry, name = parsed
if registry is not None and registry not in DOCKER_HUB_REGISTRIES:
raise ValueError(f"Image is not a Docker Hub reference: {image!r}")
return name if "/" in name else f"library/{name}"
+7 -20
View File
@@ -34,9 +34,10 @@ from utils.cache.files import read_text
from utils.cache.yaml import load_yaml
from utils.docker.image.discovery import iter_role_images
from utils.docker.image.ref import (
DOCKER_HUB_REGISTRIES,
GHCR_REGISTRY,
MCR_REGISTRY,
dockerhub_repo,
is_dockerhub,
split_registry_and_name,
)
from utils.docker.registry import fetch_registry_tags
@@ -51,6 +52,7 @@ from utils.update.base import (
version_flavor,
version_key,
)
from utils.update.follow import followed_tags
if TYPE_CHECKING:
from pathlib import Path
@@ -78,24 +80,6 @@ class DockerImageVersionUpdate:
latest: str
def is_dockerhub(image: str) -> bool:
parsed = split_registry_and_name(image)
if parsed is None:
return False
registry, _name = parsed
return registry is None or registry in DOCKER_HUB_REGISTRIES
def dockerhub_repo(image: str) -> str:
parsed = split_registry_and_name(image)
if parsed is None:
raise ValueError(f"Invalid Docker image reference: {image!r}")
registry, name = parsed
if registry is not None and registry not in DOCKER_HUB_REGISTRIES:
raise ValueError(f"Image is not a Docker Hub reference: {image!r}")
return name if "/" in name else f"library/{name}"
def fetch_dockerhub_tags(image: str, max_pages: int = 5) -> list[str]:
repo = dockerhub_repo(image)
tags: list[str] = []
@@ -356,7 +340,10 @@ def find_outdated_updates(repo_root: Path) -> list[DockerImageVersionUpdate]:
)
for entry in entries:
tags = image_tags.get(entry.image, [])
config = (load_yaml(str(entry.config_path)) or {}).get(entry.service)
tags = followed_tags(
entry.image, image_tags.get(entry.image, []), config, entry.version
)
if not tags:
continue
latest = latest_semver(
+61
View File
@@ -0,0 +1,61 @@
"""Hold a Docker Hub pin to the release a moving tag points at.
A ``meta/services.yml`` entity that sets ``follow: <tag>`` only moves to a
newer tag of the same shape whose digest equals the one ``<tag>`` points at.
Ubuntu's ``latest`` names the newest LTS release and Fedora's the newest
stable one, while the higher numbers (Ubuntu's interim releases, Fedora's
rawhide) point elsewhere.
"""
from __future__ import annotations
import json
from utils.docker.image.ref import dockerhub_repo
from utils.update.base import version_depth, version_flavor, version_key
from utils.update.fetch import get
def tag_digest(image: str, tag: str) -> str:
"""Return the digest Docker Hub reports for ``image:tag``.
Args:
image: Docker Hub reference.
tag: the tag to resolve.
"""
url = f"https://hub.docker.com/v2/repositories/{dockerhub_repo(image)}/tags/{tag}"
return str(json.loads(get(url))["digest"])
def followed_tags(
image: str, tags: list[str], config: object, current: str
) -> list[str]:
"""Narrow *tags* to the newer releases the entity's ``follow`` tag names.
Args:
image: the entity's Docker Hub image.
tags: the registry's tags for that image.
config: one entity of ``meta/services.yml``; without ``follow`` every
tag is kept.
current: the pinned version.
Returns:
The tags unchanged without ``follow``; otherwise the newer tags of the
pin's depth and flavor that share the followed tag's digest, and none
when Docker Hub cannot be read.
"""
follow = config.get("follow") if isinstance(config, dict) else None
if follow is None:
return tags
newer = [
tag
for tag in tags
if version_depth(tag) == version_depth(current)
and version_flavor(tag) == version_flavor(current)
and version_key(tag) > version_key(current)
]
try:
target = tag_digest(image, str(follow))
return [tag for tag in newer if tag_digest(image, tag) == target]
except (OSError, ValueError, KeyError):
return []