Files
computer-playbook/.github/workflows/test-deploy-compose.yml
T
kevinveenbirkenbachandClaude Opus 5 0e8c535830 feat(ci): draw the data-root filesystem per distro
The pick was made once per matrix entry, in a workflow step ahead of the drill,
and every distro of that entry then ran on it. Coverage followed: one entry
exercised one filesystem, and the swarm never reached zfs at all, because
resolve.sh's userland_for() kept it out of any entry containing arch, centos or
fedora - which is every automatic sweep. The draw now happens inside the loop in
distros.sh, once per distro, so a five-distro entry covers zfs with 87% and all
three kinds with 62% probability instead of one kind with certainty.

The workflow input changes meaning with it. It was a single kind, offered as a
choice in the manual entry; it is now a space-separated allow-list. Empty draws
from all three, and naming kinds narrows the draw to them and marks the pick
required, so a host that cannot deliver one of them fails rather than quietly
running on something else. A name that is not a filesystem this project puts a
data root on is rejected outright instead of reaching the applying step.

For the draw to mean anything the images have to carry the userlands, so
scripts/install/zfs.sh becomes scripts/install/filesystem.sh and installs ext4
and btrfs alongside zfs. That is not symmetric work: btrfs-progs does not exist
in the EL family at all, because RHEL dropped btrfs, and a plain
`dnf install btrfs-progs` there fails the whole transaction and takes e2fsprogs
with it. EPEL carries it, so the EL family goes through EPEL from the start -
split by distro rather than by a failed first attempt, since Fedora has no
epel-release to fall back to and routing a transient failure there would turn it
into a fatal one. verify() now demands all four binaries, so a packaging
regression fails the build instead of leaving a hole the drill discovers later.
Measured on all five bases: debian trixie, ubuntu 26.04, arch, fedora 44 and
centos 10 each end with mkfs.ext4, mkfs.btrfs, zfs and zpool present and ldd
fully resolved.

Two defects only became reachable once a single job could ask for different
filesystems in turn:

  * docker_dataroot.sh answered "already prepared" and returned for any mount it
    found, so from the second distro on, every draw would have been silently
    ignored on a host that survives the sweep - the runner in compose and host
    mode. It now compares what is mounted against what was asked for and
    dismantles the old root when they differ. Everything docker cached there
    goes with it; that is what covering more than one filesystem per entry
    costs.
  * reclaim_stale_pool ran only when the new pick was zfs, so a zfs to btrfs
    transition left the previous distro's pool imported on the shared kernel,
    pinning a loop device over an image its dead container had already unlinked.
    It now runs on every pick, which needed the /dev/zfs handling lifted out of
    the zfs branch into zfs_device_ready().

The restore_docker trap is armed before anything can stop docker, rather than
after, so a failure while dismantling a data root cannot leave a host without a
daemon. resolve.sh splits its allow-list into an array before testing it, so a
whitespace-only input is the empty list it looks like rather than a narrowed one
that yields an empty pick marked required. pick.sh clears both keys when it
declines to draw: they are declared in default.env, so a generated .env carries
whatever was exported when it was made, and a caller gating its apply on the
value would otherwise repoint the developer's own docker in the very case the
skip exists to prevent.

Verified by execution rather than by reading: a privileged container ran
docker_dataroot.sh with real loop devices and mkfs through ext4 -> ext4 -> btrfs
-> ext4, confirming the recognition, the replacement in both directions and that
no loop leaks across it; a second harness with a recording zpool stub confirmed
that a btrfs pick does reclaim a stale pool, that no pool means no destroy, and
that a host without /dev/zfs still completes an ext4 pick without touching zpool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 13:27:35 +02:00

243 lines
7.8 KiB
YAML

name: "📦 Test: Deploy (compose)"
on:
workflow_call:
inputs:
distros:
description: "Space-separated distro list"
required: true
type: string
whitelist:
description: "Optional space-separated whitelist of app ids"
required: false
type: string
default: ""
lifecycles:
description: "Lifecycle envelope for discovery (comma/space-separated)."
required: false
type: string
default: ""
blacklist:
description: "Optional space-separated app ids to drop from discovery"
required: false
type: string
default: ""
marker:
description: "Optional suffix appended to the discover job name (e.g. ' ⭐' for the priority line)"
required: false
type: string
default: ""
filesystem:
description: "🗄️ Space-separated filesystems the per-distro pick draws from (ext4, btrfs, zfs). Empty: all three."
required: false
type: string
default: ""
permissions:
contents: read
packages: read
env:
INFINITO_DISTROS: ${{ inputs.distros }}
NIX_CONFIG: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
INFINITO_IMAGE_TAG: ci-${{ github.sha }}
INFINITO_GHCR_MIRROR_PREFIX: mirror
INFINITO_PLAYWRIGHT_KEEP: ${{ vars.INFINITO_PLAYWRIGHT_KEEP }}
INFINITO_APP_DISCOVERY_RUNNER: host
INFINITO_DEPLOY_MODE: compose
INFINITO_DOCKER_FILESYSTEM_ALLOWED: ${{ inputs.filesystem }}
INFINITO_LIFECYCLES: ${{ inputs.lifecycles }}
INFINITO_BLACKLIST: ${{ inputs.blacklist }}
jobs:
discover:
name: 🪼 Compose (${{ inputs.distros }})${{ inputs.marker }}
runs-on: ubuntu-latest
timeout-minutes: 60
outputs:
apps: ${{ steps.discover.outputs.apps }}
steps:
- uses: actions/checkout@v7.0.0
with:
fetch-depth: 0
- name: 🔍 Resolve repository owner
shell: bash
run: |
set -euo pipefail
echo "GITHUB_REPOSITORY_OWNER=$(scripts/meta/resolve/repository/owner.sh)" >> "$GITHUB_ENV"
- name: 📦 Install jq
shell: bash
run: ./scripts/install/apt.sh jq
- name: 📦 Install deps (jq + envsubst)
shell: bash
run: ./scripts/install/apt.sh jq gettext-base
- name: 🔧 Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
cache: pip
cache-dependency-path: pyproject.toml
- name: 📦 Install project (host venv)
shell: bash
run: make install
- name: 🔍 Resolve effective whitelist (diff vs origin/main)
id: resolve_wl
shell: bash
env:
INPUT_WHITELIST: ${{ inputs.whitelist }}
run: ./scripts/github/resolve/effective_whitelist.sh
- name: 🔍 Discover apps
id: discover
shell: bash
env:
INFINITO_WHITELIST: ${{ steps.resolve_wl.outputs.whitelist }}
run: ./scripts/github/resolve/output_apps.sh
- name: 🗺️ Render the plan into the run summary
shell: bash
run: |
source scripts/meta/env/load.sh
"${PYTHON}" -m cli.meta.ci.plan \
--distros "${{ inputs.distros }}" \
--whitelist "${{ steps.resolve_wl.outputs.whitelist }}" \
--priority "${{ inputs.blacklist }}" \
--modes "compose" \
--lifecycles "${{ inputs.lifecycles }}" \
>> "$GITHUB_STEP_SUMMARY"
compose:
name: 🐳 ${{ matrix.apps }}${{ matrix.variant && format(' {0}',matrix.variant) || '' }}
if: needs.discover.outputs.apps != '' && needs.discover.outputs.apps != '[]'
runs-on: ubuntu-latest
timeout-minutes: 355
needs: discover
strategy:
fail-fast: false
matrix:
include: ${{ fromJson(needs.discover.outputs.apps) }}
steps:
- name: 🧹 Free disk space
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: true
- uses: actions/checkout@v7.0.0
- name: "📥 Load rescue-diagnostics SPOT into job env"
shell: bash
run: grep '^INFINITO_RESCUE_DIAGNOSTICS_BASE=' default.env >> "$GITHUB_ENV"
- name: 💾 Enlarge swap space
shell: bash
run: ./scripts/github/runner/enlarge_swap.sh
- name: 📦 Relocate Docker data-root onto /mnt when roomier
if: ${{ !env.ACT }}
shell: bash
run: ./scripts/github/runner/relocate_docker_dataroot.sh
- name: 🔍 Resolve repository owner
shell: bash
run: |
set -euo pipefail
echo "GITHUB_REPOSITORY_OWNER=$(scripts/meta/resolve/repository/owner.sh)" >> "$GITHUB_ENV"
- name: 📦 Install jq
shell: bash
run: ./scripts/install/apt.sh jq
- name: 🚀 Deploy app
timeout-minutes: 350
shell: bash
env:
apps: ${{ matrix.apps }}
variant: ${{ matrix.variant }}
ANSIBLE_LOG_PATH: /tmp/ansible-${{ matrix.apps }}.log
run: ./scripts/tests/deploy/ci/all.sh 2>&1 | tee "/tmp/ansible-${{ matrix.apps }}.log"
- name: 🚫 Fail on deprecation warnings
if: always()
shell: bash
run: ./scripts/tests/deploy/ci/assert_no_deprecation_warnings.sh "/tmp/ansible-${{ matrix.apps }}.log"
- name: 📊 Role runtimes
if: always()
shell: bash
run: |
set -euo pipefail
python3 -m cli.meta.ci.roles.runtime "/tmp/ansible-${{ matrix.apps }}.log" \
--format csv --output "/tmp/role-runtimes-${{ matrix.apps }}.csv"
python3 -m cli.meta.ci.roles.runtime "/tmp/role-runtimes-${{ matrix.apps }}.csv" \
--format markdown >> "$GITHUB_STEP_SUMMARY"
- name: ⬆️ Upload role runtimes CSV
if: always()
uses: actions/upload-artifact@v7
with:
name: role-runtimes-${{ matrix.apps }}
path: /tmp/role-runtimes-${{ matrix.apps }}.csv
if-no-files-found: ignore
- name: 🩺 Dump runner state (on failure)
if: failure()
timeout-minutes: 5
shell: bash
run: ./scripts/github/runner/dump_state.sh "post-deploy failure (${{ matrix.apps }})"
- name: ⬆️ Upload rescue diagnostics
if: always()
uses: actions/upload-artifact@v7
with:
name: rescue-diagnostics-compose-${{ matrix.apps }}${{ matrix.variant_slug && format('-{0}', matrix.variant_slug) || '' }}
path: ${{ env.INFINITO_RESCUE_DIAGNOSTICS_BASE }}/
if-no-files-found: ignore
- name: 🗜️ Zip generated inventory for debugging
if: always()
shell: bash
env:
APP_ID: ${{ matrix.apps }}
run: bash scripts/github/inventory/zip/compose_for_upload.sh
- name: ⬆️ Upload inventory artifact
if: ${{ always() && !env.ACT }}
uses: actions/upload-artifact@v7
with:
name: inventory-compose-${{ matrix.apps }}
path: /tmp/inventory-compose-${{ matrix.apps }}.zip
if-no-files-found: ignore
- name: 📋 Playwright step summary
if: always()
shell: bash
run: |
set -euo pipefail
python3 utils/github/playwright_summary.py \
/tmp/playwright-artifacts \
"${{ matrix.apps }}" \
>> "$GITHUB_STEP_SUMMARY"
- name: ⬆️ Upload Playwright artifacts
if: always()
uses: actions/upload-artifact@v7.0.1
with:
name: playwright-compose-${{ matrix.apps }}${{ matrix.variant_slug && format('-{0}', matrix.variant_slug) || '' }}
path: /tmp/playwright-artifacts/
if-no-files-found: ignore