mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
feat(sys-os): declare the workspace runtime images where the updater finds them
The runtime image pins lived in meta/distros.yml, which neither the docker updater, the image mirror nor the vulnerability scan reads, so a pin would only ever move by hand. The new role sys-os carries them in its meta/services.yml under the distro id; utils/distros.py renders the workspace matrix from there with the stdlib block reader, and meta/distros.yml keeps the distro attributes only. A new release now arrives as an updater pull request. follow: latest holds debian, ubuntu and fedora to the release their latest tag points at, by Docker Hub digest: Ubuntu moves between LTS releases only (26.10 shares the digest of rolling, not latest) and Fedora skips rawhide (46 shares the digest of rawhide, latest is 45). Arch stays rolling; CentOS stream tags carry no number the updater can order. dockerhub_repo and is_dockerhub move to utils/docker/image/ref.py beside split_registry_and_name, so utils/update/follow.py can use them without an import cycle and utils/update/docker.py stays under the size cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
f12658927f
commit
0ed50ae0a5
9 files changed
+245
-35
No files matched your search
@@ -20,9 +20,6 @@
|
||||
# slug path segment inside the image reference
|
||||
# family ansible_facts['os_family'] the distribution reports,
|
||||
# consumed by the meta/packages.yml registry
|
||||
# dev_runtime_image upstream base image the development-environment test
|
||||
# matrix boots the distribution from. Not one of ours,
|
||||
# so it carries its own registry and tag.
|
||||
# galaxy_platform platform name Ansible Galaxy knows the distribution by,
|
||||
# used to build the canonical galaxy_info.platforms list
|
||||
# every role's meta/main.yml must carry. The versions
|
||||
@@ -41,25 +38,20 @@ distros:
|
||||
arch:
|
||||
slug: arch
|
||||
family: Archlinux
|
||||
dev_runtime_image: manjarolinux/base
|
||||
galaxy_platform: ArchLinux
|
||||
debian:
|
||||
slug: debian
|
||||
family: Debian
|
||||
dev_runtime_image: debian:13
|
||||
galaxy_platform: Debian
|
||||
ubuntu:
|
||||
slug: ubuntu
|
||||
family: Debian
|
||||
dev_runtime_image: ubuntu:26.04
|
||||
galaxy_platform: Ubuntu
|
||||
fedora:
|
||||
slug: fedora
|
||||
family: RedHat
|
||||
dev_runtime_image: fedora:44
|
||||
galaxy_platform: Fedora
|
||||
centos:
|
||||
slug: centos
|
||||
family: RedHat
|
||||
dev_runtime_image: quay.io/centos/centos:stream10
|
||||
galaxy_platform: EL
|
||||
@@ -0,0 +1,40 @@
|
||||
# Operating System Images
|
||||
|
||||
## Description
|
||||
|
||||
The operating system images are the upstream container images of [Arch Linux](https://archlinux.org/) (through [Manjaro](https://manjaro.org/)), [Debian](https://www.debian.org/), [Ubuntu](https://ubuntu.com/), [Fedora](https://fedoraproject.org/) and [CentOS Stream](https://www.centos.org/centos-stream/), each published per distribution release.
|
||||
|
||||
## Overview
|
||||
|
||||
This role deploys nothing. Its `meta/services.yml` pins the image and release each distribution's development environment boots from, keyed by the distribution id of [meta/distros.yml](../../meta/distros.yml), and `utils/distros.py` reads the workspace matrix from it. Because the pins live in a role's `meta/services.yml`, the docker updater proposes new releases as pull requests and the image mirror and the vulnerability scan cover them like every other role image. `follow: latest` holds a pin to the release the `latest` tag points at, so Ubuntu moves between LTS releases only and Fedora skips rawhide. Arch stays rolling, and CentOS Stream tags carry no number the updater can order, so its next stream is a manual change.
|
||||
|
||||
## Cosmos
|
||||
|
||||
The diagram places Operating System Images in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph role [sys-os 💻]
|
||||
svc_arch["arch"]
|
||||
svc_debian["debian"]
|
||||
svc_ubuntu["ubuntu"]
|
||||
svc_fedora["fedora"]
|
||||
svc_centos["centos"]
|
||||
end
|
||||
```
|
||||
|
||||
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
||||
|
||||
## Features
|
||||
|
||||
- One `image` and `version` per distribution id of `meta/distros.yml`
|
||||
- Workspace test matrix rendered from these pins by `utils/distros.py`
|
||||
- New releases proposed as updater pull requests
|
||||
- `follow: latest` keeps Ubuntu on LTS releases and Fedora off rawhide
|
||||
- Images mirrored and scanned like every other role image
|
||||
|
||||
## Credits
|
||||
|
||||
Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**.
|
||||
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
|
||||
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
galaxy_info:
|
||||
author: Kevin Veen-Birkenbach
|
||||
description: "Pins the upstream operating system images the development environment boots each distribution from"
|
||||
license: Infinito.Nexus Community License (Non-Commercial)
|
||||
company: |
|
||||
Kevin Veen-Birkenbach
|
||||
https://www.veen.world
|
||||
min_ansible_version: "2.9"
|
||||
platforms:
|
||||
- name: ArchLinux
|
||||
versions:
|
||||
- all
|
||||
- name: Debian
|
||||
versions:
|
||||
- all
|
||||
- name: EL
|
||||
versions:
|
||||
- all
|
||||
- name: Fedora
|
||||
versions:
|
||||
- all
|
||||
- name: Ubuntu
|
||||
versions:
|
||||
- all
|
||||
galaxy_tags:
|
||||
- os
|
||||
- distribution
|
||||
- image
|
||||
- infinito
|
||||
- automation
|
||||
issue_tracker_url: https://s.infinito.nexus/issues
|
||||
@@ -0,0 +1,23 @@
|
||||
---
|
||||
# nocheck: required-by-coverage -- read by utils/distros.py and the docker updater; no role requires it
|
||||
os:
|
||||
lifecycle: beta
|
||||
arch:
|
||||
image: manjarolinux/base
|
||||
# nocheck: docker-version -- Manjaro publishes rolling builds only, so Arch stays rolling
|
||||
version: latest
|
||||
debian:
|
||||
image: debian
|
||||
version: "13"
|
||||
follow: latest
|
||||
ubuntu:
|
||||
image: ubuntu
|
||||
version: "26.04"
|
||||
follow: latest
|
||||
fedora:
|
||||
image: fedora
|
||||
version: "44"
|
||||
follow: latest
|
||||
centos:
|
||||
image: quay.io/centos/centos
|
||||
version: stream10
|
||||
@@ -0,0 +1,50 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from utils.update.follow import followed_tags
|
||||
|
||||
UBUNTU = {
|
||||
"latest": "sha256:lts",
|
||||
"26.04": "sha256:lts",
|
||||
"26.10": "sha256:interim",
|
||||
"rolling": "sha256:interim",
|
||||
}
|
||||
FEDORA = {
|
||||
"latest": "sha256:45",
|
||||
"44": "sha256:44",
|
||||
"45": "sha256:45",
|
||||
"46": "sha256:rawhide",
|
||||
"rawhide": "sha256:rawhide",
|
||||
}
|
||||
|
||||
|
||||
def _digests(table: dict[str, str]):
|
||||
return lambda _image, tag: table[tag]
|
||||
|
||||
|
||||
class TestFollowedTags(unittest.TestCase):
|
||||
def test_ubuntu_on_the_lts_gets_no_interim_release(self) -> None:
|
||||
with mock.patch("utils.update.follow.tag_digest", _digests(UBUNTU)):
|
||||
found = followed_tags("ubuntu", list(UBUNTU), {"follow": "latest"}, "26.04")
|
||||
self.assertEqual(found, [])
|
||||
|
||||
def test_fedora_moves_to_the_stable_release_not_rawhide(self) -> None:
|
||||
with mock.patch("utils.update.follow.tag_digest", _digests(FEDORA)):
|
||||
found = followed_tags("fedora", list(FEDORA), {"follow": "latest"}, "44")
|
||||
self.assertEqual(found, ["45"])
|
||||
|
||||
def test_an_entity_without_follow_keeps_every_tag(self) -> None:
|
||||
tags = list(FEDORA)
|
||||
self.assertEqual(followed_tags("fedora", tags, {"version": "44"}, "44"), tags)
|
||||
self.assertEqual(followed_tags("fedora", tags, None, "44"), tags)
|
||||
|
||||
def test_an_unreadable_registry_proposes_nothing(self) -> None:
|
||||
with mock.patch("utils.update.follow.tag_digest", side_effect=OSError):
|
||||
found = followed_tags("fedora", list(FEDORA), {"follow": "latest"}, "44")
|
||||
self.assertEqual(found, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+14
-7
@@ -2,9 +2,11 @@
|
||||
|
||||
``meta/distros.yml`` declares every distribution the project builds, tests
|
||||
and deploys on: the id used as ``INFINITO_DISTRO``, the ``os_family``
|
||||
ansible reports for it, the Galaxy platform name, the upstream base image
|
||||
the development-environment matrix boots it from, and the templates the
|
||||
ansible reports for it, the Galaxy platform name, and the templates the
|
||||
pkgmgr base image and the environment image references are rendered from.
|
||||
The upstream image the development-environment matrix boots each distro
|
||||
from is pinned in ``roles/sys-os/meta/services.yml`` under the distro id,
|
||||
where the docker updater, the mirror and the vulnerability scan find it.
|
||||
|
||||
Consumers MUST read the distro set through this module. A local tuple of
|
||||
distro names or an f-string spelling out ``ghcr.io/.../pkgmgr-...`` is a
|
||||
@@ -26,9 +28,11 @@ from __future__ import annotations
|
||||
|
||||
from utils.cache import PROJECT_ROOT
|
||||
from utils.cache.files import read_text
|
||||
from utils.roles.mapping import ROLE_FILE_META_SERVICES
|
||||
from utils.yaml_bootstrap import load_block
|
||||
|
||||
FILE_META_DISTROS: str = "meta/distros.yml"
|
||||
RUNTIME_IMAGES_ROLE: str = "sys-os"
|
||||
|
||||
IMAGE_PKGMGR = "pkgmgr"
|
||||
IMAGE_PKGMGR_VIRGIN = "pkgmgr_virgin"
|
||||
@@ -82,21 +86,24 @@ def galaxy_platforms() -> list[dict[str, object]]:
|
||||
|
||||
def dev_runtime_images() -> tuple[str, ...]:
|
||||
"""Upstream base images of the development-environment test matrix."""
|
||||
return tuple(spec["dev_runtime_image"] for spec in _specs().values())
|
||||
return tuple(dev_runtime_image(name) for name in distro_names())
|
||||
|
||||
|
||||
def dev_runtime_image(distro: str) -> str:
|
||||
"""Upstream base image the development environment boots a distro from."""
|
||||
return _spec(distro)["dev_runtime_image"]
|
||||
_spec(distro)
|
||||
services = PROJECT_ROOT / "roles" / RUNTIME_IMAGES_ROLE / ROLE_FILE_META_SERVICES
|
||||
entity = load_block(read_text(str(services)))[distro]
|
||||
return f"{entity['image']}:{entity['version']}"
|
||||
|
||||
|
||||
def distro_of_dev_runtime_image(image: str) -> str:
|
||||
"""Distro id whose development environment boots from ``image``."""
|
||||
for name, spec in _specs().items():
|
||||
if spec["dev_runtime_image"] == image:
|
||||
for name in distro_names():
|
||||
if dev_runtime_image(name) == image:
|
||||
return name
|
||||
raise UnknownDistroError(
|
||||
f"{image!r} is not a dev_runtime_image in {FILE_META_DISTROS}. "
|
||||
f"{image!r} is not a runtime image of {RUNTIME_IMAGES_ROLE}. "
|
||||
f"Declared: {', '.join(dev_runtime_images())}."
|
||||
)
|
||||
|
||||
|
||||
@@ -50,3 +50,21 @@ def split_registry_and_name(image: str) -> tuple[str | None, str] | None:
|
||||
return first, rest
|
||||
|
||||
return None, image
|
||||
|
||||
|
||||
def is_dockerhub(image: str) -> bool:
|
||||
parsed = split_registry_and_name(image)
|
||||
if parsed is None:
|
||||
return False
|
||||
registry, _name = parsed
|
||||
return registry is None or registry in DOCKER_HUB_REGISTRIES
|
||||
|
||||
|
||||
def dockerhub_repo(image: str) -> str:
|
||||
parsed = split_registry_and_name(image)
|
||||
if parsed is None:
|
||||
raise ValueError(f"Invalid Docker image reference: {image!r}")
|
||||
registry, name = parsed
|
||||
if registry is not None and registry not in DOCKER_HUB_REGISTRIES:
|
||||
raise ValueError(f"Image is not a Docker Hub reference: {image!r}")
|
||||
return name if "/" in name else f"library/{name}"
|
||||
+7
-20
@@ -34,9 +34,10 @@ from utils.cache.files import read_text
|
||||
from utils.cache.yaml import load_yaml
|
||||
from utils.docker.image.discovery import iter_role_images
|
||||
from utils.docker.image.ref import (
|
||||
DOCKER_HUB_REGISTRIES,
|
||||
GHCR_REGISTRY,
|
||||
MCR_REGISTRY,
|
||||
dockerhub_repo,
|
||||
is_dockerhub,
|
||||
split_registry_and_name,
|
||||
)
|
||||
from utils.docker.registry import fetch_registry_tags
|
||||
@@ -51,6 +52,7 @@ from utils.update.base import (
|
||||
version_flavor,
|
||||
version_key,
|
||||
)
|
||||
from utils.update.follow import followed_tags
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
@@ -78,24 +80,6 @@ class DockerImageVersionUpdate:
|
||||
latest: str
|
||||
|
||||
|
||||
def is_dockerhub(image: str) -> bool:
|
||||
parsed = split_registry_and_name(image)
|
||||
if parsed is None:
|
||||
return False
|
||||
registry, _name = parsed
|
||||
return registry is None or registry in DOCKER_HUB_REGISTRIES
|
||||
|
||||
|
||||
def dockerhub_repo(image: str) -> str:
|
||||
parsed = split_registry_and_name(image)
|
||||
if parsed is None:
|
||||
raise ValueError(f"Invalid Docker image reference: {image!r}")
|
||||
registry, name = parsed
|
||||
if registry is not None and registry not in DOCKER_HUB_REGISTRIES:
|
||||
raise ValueError(f"Image is not a Docker Hub reference: {image!r}")
|
||||
return name if "/" in name else f"library/{name}"
|
||||
|
||||
|
||||
def fetch_dockerhub_tags(image: str, max_pages: int = 5) -> list[str]:
|
||||
repo = dockerhub_repo(image)
|
||||
tags: list[str] = []
|
||||
@@ -356,7 +340,10 @@ def find_outdated_updates(repo_root: Path) -> list[DockerImageVersionUpdate]:
|
||||
)
|
||||
|
||||
for entry in entries:
|
||||
tags = image_tags.get(entry.image, [])
|
||||
config = (load_yaml(str(entry.config_path)) or {}).get(entry.service)
|
||||
tags = followed_tags(
|
||||
entry.image, image_tags.get(entry.image, []), config, entry.version
|
||||
)
|
||||
if not tags:
|
||||
continue
|
||||
latest = latest_semver(
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Hold a Docker Hub pin to the release a moving tag points at.
|
||||
|
||||
A ``meta/services.yml`` entity that sets ``follow: <tag>`` only moves to a
|
||||
newer tag of the same shape whose digest equals the one ``<tag>`` points at.
|
||||
Ubuntu's ``latest`` names the newest LTS release and Fedora's the newest
|
||||
stable one, while the higher numbers (Ubuntu's interim releases, Fedora's
|
||||
rawhide) point elsewhere.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from utils.docker.image.ref import dockerhub_repo
|
||||
from utils.update.base import version_depth, version_flavor, version_key
|
||||
from utils.update.fetch import get
|
||||
|
||||
|
||||
def tag_digest(image: str, tag: str) -> str:
|
||||
"""Return the digest Docker Hub reports for ``image:tag``.
|
||||
|
||||
Args:
|
||||
image: Docker Hub reference.
|
||||
tag: the tag to resolve.
|
||||
"""
|
||||
url = f"https://hub.docker.com/v2/repositories/{dockerhub_repo(image)}/tags/{tag}"
|
||||
return str(json.loads(get(url))["digest"])
|
||||
|
||||
|
||||
def followed_tags(
|
||||
image: str, tags: list[str], config: object, current: str
|
||||
) -> list[str]:
|
||||
"""Narrow *tags* to the newer releases the entity's ``follow`` tag names.
|
||||
|
||||
Args:
|
||||
image: the entity's Docker Hub image.
|
||||
tags: the registry's tags for that image.
|
||||
config: one entity of ``meta/services.yml``; without ``follow`` every
|
||||
tag is kept.
|
||||
current: the pinned version.
|
||||
|
||||
Returns:
|
||||
The tags unchanged without ``follow``; otherwise the newer tags of the
|
||||
pin's depth and flavor that share the followed tag's digest, and none
|
||||
when Docker Hub cannot be read.
|
||||
"""
|
||||
follow = config.get("follow") if isinstance(config, dict) else None
|
||||
if follow is None:
|
||||
return tags
|
||||
newer = [
|
||||
tag
|
||||
for tag in tags
|
||||
if version_depth(tag) == version_depth(current)
|
||||
and version_flavor(tag) == version_flavor(current)
|
||||
and version_key(tag) > version_key(current)
|
||||
]
|
||||
try:
|
||||
target = tag_digest(image, str(follow))
|
||||
return [tag for tag in newer if tag_digest(image, tag) == target]
|
||||
except (OSError, ValueError, KeyError):
|
||||
return []
|
||||
Reference in new issue
Block a user