kevinveenbirkenbachandClaude Opus 5
add00234c9
CI deploy jobs were titled with the raw role id, so a run page read as a
column of dash-joined prefixes. They now carry a display name: the category
path as hanzi labels, then the role's README heading or the part of the id
the categories do not already say, whichever is shorter.
utils/roles/display.py owns both directions. Encoding happens once, at
discovery time, as a `label` key on every matrix entry; the workflows render
`<glyph> ${{ matrix.label }}` and keep the glyph as a literal because the
retrigger reads the deploy mode off it.
Decoding is what makes the retrigger keep working. It accepts a display name
with or without its variant shard and a bare role id unchanged, so job names
from runs that predate this still parse and a hand-typed dispatch never has
to spell hanzi. The parser therefore stopped matching an id-shaped tail and
asks the codec instead; a job the codec does not recognise is skipped rather
than counted as a role.
The role selections go out as display names too, so the run title reads like
the jobs it spawns. Four consumers decode them back, each the single place
the value is actually used: the whitelist resolver before it checks the role
directory, the discovery query on its env, and the sequencing and plan CLIs
on their arguments.
Every fragment is joined by `·`, including before the title, and the heading
loses its whitespace and any emoji variation selector. A display name holding
a space would make the space-separated whitelist and priority inputs
unparseable with more than one role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Workflow Dependency Map
How the workflows under .github/workflows/ trigger and call each other.
Solid arrows: workflow_call (uses:) or job needs. Dotted arrows:
indirect coupling (CLI dispatch, shared concurrency group). Per-workflow
inputs and descriptions: workflows.md.
CI pipeline
flowchart TB
push["push: main, feature/**, hotfix/**, fix/**"] --> epl["entry-push-latest.yml"]
pr["pull_request: opened, synchronize, reopened, ready_for_review"] --> eprc["entry-pull-request-change.yml"]
dispatch["workflow_dispatch"] --> eman["entry-manual.yml"]
epl --> orch["ci-orchestrator.yml"]
epl -->|"version tag on main"| relv["release-version.yml"]
eprc --> orch
eprc -->|"fork PRs: privileged prebuild"| imgbuild["images-build-ci.yml"]
eprc -->|"fork PRs: privileged prebuild"| imgmirror["images-mirror-missing.yml"]
eman --> orch
subgraph orchestrator["ci-orchestrator.yml jobs"]
waitfork["wait-fork-prereq-run"] --> forkready["fork-prereqs-ready"]
lintwf["lint.yml: make lint + hadolint"]
testwf["test.yml: make test"]
codeql["security-codeql.yml"]
buildci["build-ci-images: images-build-ci.yml"] --> dns["test-dns.yml"]
mirror["images-mirror-missing.yml"]
seq["sequencing: serial or parallel, per line"]
subgraph prio["priority line, skipped without a priority input"]
swarmprio["test-deploy-swarm-priority"]
swarmprio -->|"serial"| composeprio["test-deploy-compose-priority"]
composeprio -->|"serial"| hostprio["test-deploy-host-priority"]
snprio["test-deploy-single-node-priority: parallel"]
end
subgraph reg["regular line"]
swarmreg["test-deploy-swarm"]
swarmreg -->|"serial"| composereg["test-deploy-compose"]
composereg -->|"serial"| hostreg["test-deploy-host"]
snreg["test-deploy-single-node: parallel"]
end
lintwf --> prio
testwf --> prio
dns --> prio
mirror --> prio
seq --> prio
prio -->|"all priority jobs green"| reg
swarmreg --> smoke["test-runner-smoke.yml"]
prio --> report["report-main-failures"]
reg --> report
instmake["test-install-make.yml"]
instpkgmgr["test-install-pkgmgr.yml"]
mirror --> devenv["test-workspace: test-workspace.yml"]
buildci --> testguide["test-instructions.yml"]
mirror --> testguide
prio --> donegate["done"]
reg --> donegate
seq --> donegate
smoke --> donegate
instmake --> donegate
instpkgmgr --> donegate
devenv --> donegate
testguide --> donegate
end
snprio --> singlenode["test-deploy-single-node.yml"]
snreg --> singlenode
singlenode --> deploycompose["test-deploy-compose.yml"]
singlenode --> deployhost["test-deploy-host.yml"]
composeprio --> deploycompose
composereg --> deploycompose
hostprio --> deployhost
hostreg --> deployhost
swarmprio --> deployswarm["test-deploy-swarm.yml"]
swarmreg --> deployswarm
The priority line runs only when the orchestrator's priority input is set;
with it empty every priority job is skipped and the regular line starts
directly. The regular jobs receive the priority ids as blacklist, so each
role deploys in exactly one line.
Deploy-line sequencing
A line is one half of a run: the priority line (⭐, the roles named in
priority) or the regular line (🔁, everything else). Each line deploys up
to three modes: swarm, compose, host. The lines are always sequential
relative to each other; this section is about the order inside one line.
Why the order exists
GitHub cancels a job that has sat queued for 24 hours. That clock starts when
the job is queued, not when the run starts, and a needs: edge delays queueing
until the dependency completes. A line that starts all three modes at once
queues its whole matrix in one go, so with more roles than runner slots the
tail of that matrix waits past the cut and dies unrun. Deploying the modes one
after the other restarts the clock per mode.
Serialising costs wall-clock, so it is only worth it above a certain size. The run does not pay it on a small diff.
The decision
The sequencing job resolves the effective whitelist (the same
scripts/github/resolve/effective_whitelist.sh the discover steps use) and
calls cli.meta.ci.sequencing once per line, emitting serial or parallel
as a job output.
Job counts are taken on the row basis each mode actually runs on: swarm
selections are role#variant tokens that map 1:1 onto jobs, compose and host
selections are whole roles whose variants pack into bundles
(utils.github.variant.bundles). The count deliberately omits the
runner-storage filter that scripts/meta/resolve/apps.sh applies inside
GitHub Actions, so it can only overestimate — erring towards the sequential
layout, which cannot be cancelled.
sequencing input |
Behaviour |
|---|---|
auto (default) |
serial above INFINITO_CI_SEQUENTIAL_THRESHOLD jobs in that line, parallel at or below it |
serial |
forced; skips the count entirely |
parallel |
forced; skips the count entirely |
The threshold lives in default.env as
INFINITO_CI_SEQUENTIAL_THRESHOLD. Both lines are decided independently: a
small priority line can run parallel while the regular line behind it runs
serial.
The two layouts
Serial order is swarm, then compose, then host — heaviest mode first, so the longest queue drains while the run is youngest.
- parallel —
test-deploy-swarm-priorityplustest-deploy-single-node-priority, which fans compose and host out together. This is the layout the pipeline had before sequencing existed. - serial —
test-deploy-swarm-priority→test-deploy-compose-priority→test-deploy-host-priority, each chained on the previous throughneeds:.
The regular line mirrors this with the unsuffixed job names.
A needs: edge is static, so a workflow cannot make one conditional. Both
layouts therefore exist as separate jobs and the sequencing output skips
one of them. Because a skipped dependency otherwise skips its dependents, the
serial jobs carry always() plus explicit needs.<job>.result checks on the
real gates. The swarm job depends on no sequencing decision at all: it leads
in either layout.
Stopping on failure
mode_fail_fast (default true) decides whether a serial line stops at its
first failed mode:
| Value | Behaviour |
|---|---|
true |
swarm fails → compose and host of that line are skipped |
false |
every mode deploys and reports; the run still ends red |
skipped counts as passed, so a mode absent from modes never blocks the
chain. The parallel layout ignores the switch — with no order there is nothing
to stop. The setting is a checkbox on entry-manual.yml; the other entry
points take the default.
Job budget
cli.meta.ci.slots divides the run's 256-job cap between the deploy
matrices and reads the orchestrator to do it, so every deploy caller —
including the serial twins — must be listed in its _DEPLOY_CALLERS.
A caller missing there is charged a guessed dynamic-matrix estimate instead
of its single discover job.
Cancellation
flowchart TB
prclose["pull_request_target: closed, converted_to_draft"] --> eprcancel["entry-pull-request-cancel.yml"]
branchdelete["delete: branch"] --> delbranch["delete-branch.yml"]
eprcancel -.->|"cancels concurrency group"| runningci["running entry + child workflow runs"]
delbranch -.->|"cancels concurrency group"| runningci
Scheduled and standalone
flowchart TB
daily["schedule: daily 00:00 UTC"] --> mirrorall["images-mirror-all.yml"]
daily --> stale["cleanup-stale.yml"]
daily --> relhighest["release-highest.yml"]
weekly["schedule: weekly Sat 00:00 UTC"] --> updatewf["update.yml"]
weekly --> cleanupci["images-cleanup-ci.yml"]
weeklymon["schedule: weekly Mon 00:00 UTC"] --> scorecard["security-scorecard.yml"]
branchprot["branch_protection_rule"] --> scorecard
pushmain["push: main"] --> updatewf
prtarget["pull_request_target: opened, reopened"] --> depclose["dependabot-close.yml"]
relhighest -.->|"gh workflow run"| relver["release-version.yml"]
relver --> imgbuildci["images-build-ci.yml"]
manual["workflow_dispatch"] --> mirrorcleanup["images-mirror-cleanup.yml"]
manual --> deploywf["test-instructions.yml: run a role README Production command"]
Also manually dispatchable: images-mirror-all.yml, images-cleanup-ci.yml,
cleanup-stale.yml, update.yml, release-highest.yml, release-version.yml,
lint.yml, test.yml, test-deploy-swarm.yml, test-dns.yml,
test-workspace.yml, test-runner-smoke.yml.