computer-playbook/.github/workflows

Workflow Dependency Map

How the workflows under .github/workflows/ trigger and call each other. Solid arrows: workflow_call (uses:) or job needs. Dotted arrows: indirect coupling (CLI dispatch, shared concurrency group). Per-workflow inputs and descriptions: workflows.md.

CI pipeline

flowchart TB
    push["push: main, feature/**, hotfix/**, fix/**"] --> epl["entry-push-latest.yml"]
    pr["pull_request: opened, synchronize, reopened, ready_for_review"] --> eprc["entry-pr-change-orchestrate.yml"]
    dispatch["workflow_dispatch"] --> eman["entry-manual-steer.yml"]

    epl --> orch["call-orchestrator.yml"]
    epl -->|"version tag on main"| relv["call-release-version.yml"]
    eprc --> orch
    eprc -->|"fork PRs: privileged prebuild"| imgbuild["call-images-build-ci.yml"]
    eprc -->|"fork PRs: privileged prebuild"| imgmirror["call-images-mirror-missing.yml"]
    eman --> orch

    subgraph orchestrator["call-orchestrator.yml jobs"]
        waitfork["wait-fork-prereq-run"] --> forkready["fork-prereqs-ready"]

        lintwf["call-lint.yml: make lint + hadolint"]
        testwf["call-test.yml: make test"]
        codeql["cron-security-codeql.yml"]
        buildci["build-ci-images: call-images-build-ci.yml"] --> dns["call-test-dns.yml"]
        mirror["call-images-mirror-missing.yml"]

        subgraph chain["serial chunk chain"]
            chunk0["test-deploy-chunk-0"]
            chunk0 --> chunk1["test-deploy-chunk-1"]
            chunk1 --> chunk2["test-deploy-chunk-2"]
            chunk2 --> chunk3["test-deploy-chunk-3"]
        end

        lintwf --> chain
        testwf --> chain
        dns --> chain
        mirror --> chain
        buildci --> chain

        chunk0 --> smoke["call-test-runner-smoke.yml"]
        chain --> report["report-main-failures"]

        install["call-test-install.yml"]
        mirror --> devenv["test-workspace: call-test-workspace.yml"]

        chain --> donegate["done"]
        smoke --> donegate
        install --> donegate
        devenv --> donegate
    end

    chunk0 --> deploy["call-test-deploy.yml"]
    chunk1 --> deploy
    chunk2 --> deploy

Sweeps and chunks

A sweep is one orchestrator run. It builds a single ordered list of role#variant rows, assigns each row a deploy mode and a network mode, and deploys the list in serial chunk blocks.

Why chunks exist

GitHub cancels a job that has sat queued for 24 hours. That clock starts when the job is queued, not when the run starts, and a needs: edge delays queueing until the dependency completes. A run that queues its whole matrix at once leaves the tail waiting past the cut, where it dies unrun. Deploying in chunks chained on needs: restarts the clock per chunk.

The chunk size is what makes that safe, and cli.meta.ci.slots derives it rather than pinning it:

waves      = floor(INFINITO_CI_QUEUE_HOURS / the deploy job's timeout-minutes)
chunk size = INFINITO_CI_CONCURRENCY * waves

Assuming every job burns its full timeout is deliberately pessimistic — no job can outlast it — so the estimate can only overshoot the real drain time. With the current constants that is 4 waves of 20 runners, so 80 rows per chunk, and the last job of a chunk starts around t=18h, well inside the 24h window.

How many chunks a sweep spends

The run's 256-job cap is the second ceiling. slots counts every non-deploy job the orchestrator chain spawns, subtracts the worst entry-*.yml overhead, and what remains is available — the rows one sweep may deploy across all its chunks. Rows beyond that roll into the next sweep, which starts reading the regular line at an offset (cli.meta.ci.chunks), so consecutive sweeps walk the whole list instead of re-testing the same head forever.

GitHub Actions cannot generate a variable number of jobs, so the chunk blocks are written out in the orchestrator. INFINITO_CI_MAX_CHUNKS must equal how many exist there — slots plans a sweep against that key, and a sweep planned larger than the chain silently drops its tail. Declare one block more than available fills (ceil(available / chunk size) + 1): the split receives every declared block, so after a short priority chunk the regular rows still fill the rest of the budget. Blocks whose slice comes back empty skip themselves and cost nothing.

Run python -m cli.meta.ci.slots --matrix to see the whole budget.

Priority

priority names the roles that lead. They sort to the head of the list and the split forces a chunk boundary at the priority/regular seam, so a chunk is either all priority or all regular — the seam chunk stays short rather than being topped up. That is what guarantees every priority row is deployed before the first regular one starts. Priority rows never move with the sweep offset.

Each chunk block shows up as <keycap> Batch, from 0️⃣ Batch to 3️⃣ Batch, and its discover job as 🔍 Batch <index>. A batch that holds priority rows wears a trailing ⭐ on both (0️⃣ Batch ⭐, 🔍 Batch 0 ⭐); its deploy rows carry no star of their own. The selection job counts those batches before any of them runs and exposes the count as its starred output (cli.meta.ci.matrix --priority-chunks); batch N is starred while that count exceeds N.

They are also the rows a run must not sample. A priority row is deployed in every combination it can take — each variant, in each mode its role offers, and on the modes that carry the network axis once per node network mode the role can be served in (clearnet, tor, multi) — all within the same sweep. A 3-variant role offering compose and swarm therefore becomes 3 × 2 × 3 = 18 jobs, not 3. That is the point of naming a role in priority: it is proven everywhere at once instead of over several sweeps.

An explicit network input still wins over the full coverage: clearnet, tor and multi are operator narrowings. A variant that pins services.tor.enabled to false only ever runs on clearnet regardless, and so does a role without a tor bond unless its deploy pulls the Tor provider in, which adds multi.

Selection tokens

An entry in whitelist or priority is a role name that MAY pin the axes the row would otherwise be assigned. Two spellings are accepted:

Form	Example
ASCII	web-app-nextcloud#0,2@swarm+tor%debian/zfs:arm64
the job label, pasted back out of the failed run	🐳🧅🌀🦓🦾𓏴𓏛Nextcloud#0

Separator	Axis	Values
#	variants	comma-separated indices
@	deploy mode	compose, swarm, host
+	network mode	clearnet, tor, multi
%	distro	arch, debian, ubuntu, fedora, centos
/	filesystem	zfs, btrfs, ext4
:	architecture	amd64, arm64

The network mode is a + suffix rather than a -tor suffix because a role id may itself end in -tor.

What a token leaves open belongs to the line it stands in: priority covers every remaining combination in one sweep, whitelist keeps the rotation and the row's position in the matrix. A pin the row cannot take aborts the matrix rather than dropping the row, because a dropped row would report a green run for a combination that never ran. Such a pin is @swarm on a role without its own stack, +tor on a variant that pins services.tor.enabled false, +multi on a role with reachability.single_mode: true, or any axis that fights the run's own mode, network, distros, filesystem or architectures input, or :arm64 on a role whose meta/services.yml declares architectures: [amd64].

That is also what a retrigger replays: --failed reads the glyphs off the failed job's title and writes them back as a token, so the row comes back in the mode, network mode, distribution and architecture it died on rather than on whatever the rotation would pick next. The architecture is replayed because it is the machine the job ran on, not a preference it could fall back from. The filesystem is deliberately left out. A title states the kind the matrix assigned, which a deploy is allowed to fall back from, so pinning it would both misstate what the job ran on and turn the kind into a demand, failing the retrigger on the very condition the fallback absorbs.

Mode, network, distro, filesystem and architecture

A row deploys its node in one network mode, handed to the inventory as NETWORK_MODE. The modes a row can take follow its role (network_modes.md): a variant that pins services.tor.enabled false only takes clearnet; a role without a tor bond takes clearnet, plus multi when the services closure of its discovery row names the Tor provider, where the node runs Tor and the role stays on clearnet; reachability.modes drops every mode that needs a network the role excludes; reachability.single_mode drops multi; the Tor provider never takes clearnet. The network input decides what the axis is allowed to do at all:

Value	Effect
auto	rotates over the modes the row can take; a priority row covers all of them in one sweep
clearnet	every row runs without Tor, the provider disabled
tor	the rows that can be served onion-only run in tor; the rest are dropped
multi	the rows that can be served on both networks run in multi; the rest are dropped

A multi row runs the role's Playwright suite once, on the clearnet or the onion URL picked at random, together with the shared cross-network leak spec.

The vpn input does the same for the WireGuard mesh on swarm rows: auto rotates it, enforced meshes every swarm row and disabled none. A compose row never takes the mesh. meshes names the WireGuard layout a meshed row deploys, empty meaning INFINITO_SWARM_MESHES from default.env; pointing it at an inventories/examples/*/wireguard.yml with vpn: enforced and mode: swarm is how an example is deployed in CI.

distros, filesystem and architectures narrow the pools the other axes draw from. All three default to empty, which means the whole declared set. That is what a sweep wants, because the rows are spread over the pool rather than all sharing one value. Narrowing is for chasing a single distribution, filesystem or architecture.

The architecture is the one axis a deploy job cannot apply to itself: the distro is installed and the filesystem is built, but the CPU arrives with the runner. The matrix therefore carries the runner label with the row (ubuntu-26.04 for amd64, ubuntu-24.04-arm for arm64), runs-on reads it, and the job asserts uname -m against what it was assigned before deploying anything, so a label that stops resolving to arm64 hardware fails loudly instead of reporting an arm64 row green from an amd64 machine.

A role that only one architecture can run declares it on its primary service entity, next to modes:

<primary_entity>:
  architectures:
    - amd64

The row then draws from that intersection instead of the run's whole pool, and so does every row whose variant deploys that role, as the deploy planner resolves it. A role and a run that permit nothing in common abort the matrix rather than dropping the row.

Whether the assigned filesystem is binding depends on who chose it:

How the row got its kind	The kind is	A host that cannot serve it
the rotation, from a pool of two or more (every automatic run)	a preference	falls back inside the pool, and says so in the step summary
the rotation, from a pool of exactly one (filesystem: zfs)	a demand	fails the row
a selection token (role#0/zfs)	a demand	fails the row

A pool of two or more stays a preference because the matrix already narrowed each row to one kind out of it: falling back to the other kind the run allowed is inside what the operator asked for, while failing the row is not. A run whose whole pool is unservable fails rather than leaving the pool.

The narrow case is what a deploy job actually sees, because the matrix hands every row exactly one kind. Reading that narrowness as a demand would turn every condition the applying step tolerates into a red row on every deploy: a loop device it could not claim, a pool another job still holds. The run's own input decides instead.

A fallback is reported, never silent. The job title and artifact name carry the kind the matrix assigned, while docker_dataroot.sh writes the effective filesystem and the reason into the step summary. When a run has to prove a filesystem-dependent path, read the summary rather than the title, or name the kind on the run to make it a demand. The snapshot mode of svc-bkp-volume-2-local is the case that matters: it only engages on btrfs and zfs.

The userland tools are baked into every distro image by filesystem.sh; the kernel side comes from the host.

For regular rows every axis is a deterministic rotation over the row's position in the global list and the sweep number, never random, so a red job reproduces by re-running the same sweep:

Axis	Rotation
mode	(position + sweep) % len(modes the role offers)
network	(position + sweep // 2) % len(network modes the row can take)
vpn (swarm rows)	(position // 2 + sweep // 4) % 2
distro	(position + sweep) % len(distro pool)
filesystem	(position // len(distro pool) + sweep) % len(filesystem pool)
architecture	(position + sweep) % len(architecture pool)

A role offers at most two modes in practice — swarm needs its own stack, host needs the absence of one — so a row flips between its two modes on consecutive sweeps. The network turns on sweep // 2 so it does not flip in lockstep: a row with two modes and two network modes walks all four combinations over four sweeps instead of only two. Distro and filesystem read the position like an odometer, the distro as the low digit, so consecutive rows walk every pairing of the two pools rather than a diagonal through it. Turning both on the position directly would cover only as many pairings as the pools are long whenever they happen to be the same length, and no sweep would unlock that, because the sweep shifts both by the same amount. The architecture rotates on the position directly rather than as a third odometer digit. Two values are coprime with both five distros and the fifteen distro/filesystem positions, so walking it fastest still reaches every pairing, while making it the slowest digit would hand a role with few rows one architecture forever - and the point of the axis is that a role's own variants split over both. Priority rows skip the mode/network rotation entirely and take the whole cross-product at once; their distro, filesystem and architecture walk on across those combinations, so one priority role proves several distributions in one sweep.

Because the same variant can run several times in one sweep, the network mode is part of what identifies a job: it is in the job label (🌐, 🧅 or 🌈) and in every artifact name. Two jobs uploading under one artifact name is a conflict, not an overwrite. The distro, filesystem and architecture glyphs follow it in the label (🐳🧅🌀🦓🦾), so a title says which combination died without opening the job.

Stopping on failure

chunk_gate (default true) decides whether the chain stops at its first failed chunk:

Value	Behaviour
true	chunk 0 fails → the remaining chunks are skipped
false	every chunk deploys and reports; the run still ends red

skipped counts as passed, so an empty chunk never blocks the chain. After fixing what broke a sweep, resume_from_chunk re-enters at that index instead of re-running the green chunks. Both are inputs on entry-manual-steer.yml; the other entry points take the defaults, except that a release push on main (a vX.Y.Z tag on the pushed commit) runs with chunk_gate: false. A retrigger carries the source run's chunk_gate over unless infinito administration deploy ci trigger --chunk-gate false overrides it; the i8ciallon alias is i8ciall with that override, for sweeps that should report every chunk before anyone looks.

i8civar is i8ciallon without --strict. A role that failed outright still comes back by name, so the rotation re-picks its axes; a job that never reached a verdict comes back as the exact role#variant@mode+tor it was given, instead of waiting for the regular line to reach it. Use it to finish a run that was cancelled with most of its matrix still pending.

Cancellation

flowchart TB
    prclose["pull_request_target: closed, converted_to_draft"] --> eprcancel["entry-pr-closed-cancel-workflows.yml"]
    branchdelete["delete: branch"] --> delbranch["entry-delete-branch.yml"]
    newcommit["push (not main) / pull_request_target: opened, synchronize, reopened, ready_for_review, labeled"] --> superseded["entry-cancel-superseded.yml"]
    eprcancel -.->|"cancels concurrency group"| runningci["running entry + child workflow runs"]
    delbranch -.->|"cancels concurrency group"| runningci
    superseded -.->|"cancels via API what the group left behind"| runningci

The concurrency group stays the primary mechanism; it reaps within seconds of a run being created and needs no runner. entry-cancel-superseded.yml is the fallback for the case where it does not, and carries no group of its own, because the run holding a group cannot be the run that frees it.

A workflow the entries call declares no concurrency of its own: its jobs then belong to the entry's group, and the entry's cancel-in-progress reaches them. call-images-build-ci.yml and call-images-mirror-missing.yml still declare one, so a run whose only live job is theirs stays outside the entry's group and waits for the fallback.

The deploy job of call-test-deploy.yml holds a job-level group keyed on every matrix axis: mode, tor, distro, filesystem, apps and variant. One deployment runs at a time across all branches; an identical one from another run queues behind it (queue: max), and a cancel removes a queued one.

On pull_request and pull_request_target the deploy and workspace matrices run fail-fast: the first failed row cancels its siblings in that matrix. Push and manual runs keep every row.

A job that runs after a failed needs guards its if: with !cancelled(), not always(), so a cancel stops it. Only done and report-main-failures keep always(): they report the cancelled run. Enforced by test_workflow_cancellable_jobs_no_always.py.

Scheduled and standalone

flowchart TB
    daily["schedule: daily 00:00 UTC"] --> mirrorall["cron-images-mirror-all.yml"]
    daily --> stale["cron-cleanup-stale.yml"]
    daily --> relhighest["cron-release-highest.yml"]
    weekly["schedule: weekly Sat 00:00 UTC"] --> updatewf["cron-update.yml"]
    weekly --> cleanupci["cron-images-cleanup-ci.yml"]
    weeklymon["schedule: weekly Mon 00:00 UTC"] --> scorecard["cron-security-scorecard.yml"]
    branchprot["branch_protection_rule"] --> scorecard
    pushmain["push: main"] --> updatewf
    prtarget["pull_request_target: opened, reopened"] --> depclose["entry-pr-open-dependabot-close.yml"]

    relhighest -.->|"gh workflow run"| relver["call-release-version.yml"]
    relver --> imgbuildci["call-images-build-ci.yml"]
    manual["workflow_dispatch"] --> mirrorcleanup["entry-manual-mirror-cleanup.yml"]

Also manually dispatchable: cron-images-mirror-all.yml, cron-images-cleanup-ci.yml, cron-cleanup-stale.yml, cron-update.yml, cron-release-highest.yml, call-release-version.yml, call-lint.yml, call-test.yml, call-test-dns.yml, call-test-workspace.yml, call-test-runner-smoke.yml.