Files
computer-playbook/scripts/meta/resolve/apps.sh
T
kevinveenbirkenbachandClaude Opus 5 2a7455bd90 feat(ci): spread the deploy matrix over amd64 and arm64
Both architectures were claimed to work and only one was ever exercised. The
deploy matrix now carries an architecture the way it carries mode, tor, distro
and filesystem, so every run proves both.

A job cannot ask for an architecture the way it asks for a distro, because the
architecture is the machine rather than something the job installs on it. The
matrix therefore carries a runner label and `runs-on` reads it, and an
architecture without a label in RUNNERS is a build error rather than a row that
lands on the wrong hardware and still reports green. assert_architecture.sh
checks the runner it actually got, because a label that stops resolving to
arm64 would otherwise go unnoticed forever.

Assignment is rotation, not chance: two architectures against five distros and
fifteen distro/filesystem positions are coprime enough that every combination
comes round. A role narrows the pool through `architectures` in its
meta/services.yml; without one it takes whatever the rotation hands it. An
empty intersection aborts instead of silently removing the role from the run.

Images are built per architecture and pushed under an architecture-suffixed
tag, then merged into the plain tag by manifest_all.sh; pushing the plain tag
from both builds would let the last one win and send half the rows to the wrong
platform. cleanup.sh now protects the `ci-<sha>-<architecture>` children of a
protected tag, because the plain tag is an index over them and deleting a child
leaves it unpullable on that platform.

label.py is split out of axes.py, which the 500-line limit no longer fits.
Writing a job title and reading one back belong together: a retrigger that
recovers an axis wrongly replays a different deploy than the one that failed,
and a drifting artifact name makes the failure reporter download nothing. The
leading `.*` in LABEL_RE is greedy on purpose so it anchors on the last mode
glyph; a reusable-workflow caller path carries a glyph of its own, and
test_selection.py guards the case.

Verified: 313 unit tests and 48 subtests across utils/github/variant,
utils/roles and cli/meta/ci. Locally, a deploy pinned to arm64 reaches ok=1444
and its image builds log aarch64-linux-gnu paths.

Not verified here: actionlint over the seven changed workflows, and a full
`make test`. The matrix doubles the row count, so runtime and runner
availability change for every CI run from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 07:02:44 +02:00

107 lines
3.9 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
# Resolve the CI deploy matrix for one chunk of one sweep. The pipeline itself
# (two discovery queries, the axis rotations, the chunk split) lives in
# cli.meta.ci.matrix, shared with the deploy-plan table, so the matrix this
# emits and the plan the summary renders can never disagree. Every entry is one
# role#variant row carrying the mode, onion state, distro, filesystem and CPU
# architecture it was assigned, plus the runner label that architecture needs.
#
# Inputs via env (defaults live in default.env, the single source of truth):
# INFINITO_CI_CHUNK chunk index to emit (required)
# INFINITO_CI_SWEEP sweep number driving the axis rotations
# INFINITO_CI_OFFSET regular rows to skip before filling the chunks
# INFINITO_MODES 'auto' or a subset of 'host compose swarm'
# INFINITO_WHITELIST optional space-separated app ids to keep
# INFINITO_PRIORITY optional space-separated app ids to lead
# INFINITO_TOR auto|enforced|exclusive|disabled
# INFINITO_DISTROS distro pool the rows are spread over; empty: all
# INFINITO_DOCKER_FILESYSTEM_ALLOWED filesystem pool; empty: all
# INFINITO_ARCHITECTURES architecture pool; empty: amd64 and arm64
# INFINITO_LIFECYCLES lifecycle envelope for discovery
# INFINITO_DISCOVERY_SORT complexity --sort spec (coverage-first)
# INFINITO_REQUIRED_STORAGE per-runner CI storage budget
# INFINITO_APP_DISCOVERY_RUNNER host|docker
#
# Output: JSON array of matrix entries to stdout (single line, always valid).
PYTHON="${PYTHON:-python3}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
cd "$REPO_ROOT"
if [[ -f "scripts/meta/env/load.sh" ]]; then
# shellcheck source=scripts/meta/env/load.sh
source "scripts/meta/env/load.sh"
fi
json_compact_array() {
jq -c 'if type=="array" then . else [] end'
}
run_meta_cli() {
case "${INFINITO_APP_DISCOVERY_RUNNER:?INFINITO_APP_DISCOVERY_RUNNER must be set}" in
host)
"${PYTHON}" "$@"
;;
docker)
NIX_CONFIG="${NIX_CONFIG:-}" \
INFINITO_DISTRO="${INFINITO_DISTRO}" \
docker compose exec -T infinito "${PYTHON}" "$@"
;;
*)
echo "apps.sh: unknown INFINITO_APP_DISCOVERY_RUNNER='${INFINITO_APP_DISCOVERY_RUNNER}' (expected: host|docker)" >&2
exit 2
;;
esac
}
matrix_json="$(
run_meta_cli -m cli.meta.ci.matrix \
--index "${INFINITO_CI_CHUNK:?INFINITO_CI_CHUNK must be set to the chunk index to emit}" \
--sweep "${INFINITO_CI_SWEEP}" \
--offset "${INFINITO_CI_OFFSET}" \
--modes "${INFINITO_MODES}" \
--whitelist "${INFINITO_WHITELIST}" \
--priority "${INFINITO_PRIORITY}" \
--lifecycles "${INFINITO_LIFECYCLES}" \
--tor "${INFINITO_TOR}" \
--distros "${INFINITO_DISTROS}" \
--filesystem "${INFINITO_DOCKER_FILESYSTEM_ALLOWED}" \
--architectures "${INFINITO_ARCHITECTURES}"
)"
if [[ -n "${GITHUB_ACTIONS:-}" && -z "${ACT:-}" ]]; then
required_storage="${INFINITO_REQUIRED_STORAGE}"
mapfile -t roles < <(printf '%s\n' "${matrix_json}" | jq -r '.[].apps' | sort -u)
if [[ "${#roles[@]}" -gt 0 ]]; then
run_meta_cli \
-m cli.meta.roles.applications.sufficient_storage \
--roles "${roles[@]}" \
--required-storage "${required_storage}" \
--warnings \
--format json \
>/dev/null || true # nocheck: shell-or-true -- grandfathered: worked in practice; TODO: sharpen to catch only the exact tolerated error
kept_roles="$(
run_meta_cli \
-m cli.meta.roles.applications.sufficient_storage \
--roles "${roles[@]}" \
--required-storage "${required_storage}" \
--format json |
json_compact_array
)"
matrix_json="$(
printf '%s' "${matrix_json}" |
jq -c --argjson keep "${kept_roles}" \
'map(select(.apps as $a | $keep | index($a) != null))'
)"
fi
fi
printf '%s\n' "${matrix_json}" | json_compact_array