The workflows pin `runs-on: ubuntu-NN.NN`, so no job moves to a new Ubuntu
release behind our back, but nothing raised the pin either. A sixth job of
`🔄 Update versions` closes that: `utils/update/runner.py` reads the image
table of actions/runner-images, takes the newest x64 Ubuntu label whose
image carries no preview or beta badge, and raises every older bare
`runs-on: ubuntu-NN.NN` to it; arch-suffixed labels stay untouched. The job
opens the change as one PR over `.github/workflows`, so its CI run tests
the new image before it reaches main.
This commit carries the updater's first result: every job now runs on
ubuntu-26.04 (49 `runs-on`).
actionlint 1.7.12, the newest release, does not know `ubuntu-26.04` yet
(rhysd/actionlint#682, fix proposed in rhysd/actionlint#743), so
`.github/actionlint.yaml` declares the label until a release knows it.
act maps each `ubuntu-NN.NN` label the workflows use to the local platform
image, derived from `.github/workflows`, instead of a fixed list that did
not know ubuntu-26.04.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The duplicate check accepted any open PR whose head branch started with
the dedupe prefix and ended in the fingerprint of the fresh proposal. The
listing also returns PRs from forks, and the fingerprint can be derived
from public data, so a fork PR with such a branch name would have made the
bot skip its own update PR without carrying the change.
The shortcut now only applies to PRs whose head branch lives in this
repository, which only accounts with write access can create. A fork PR
is still compared by the content of its head, as before.
Raised by the CodeRabbit review of PR #781.
Refs #780
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update bot recognised an open PR as a duplicate by hashing the blobs
of the files its commit changed, read at the PR head. A follow-up commit
that edits one of those files changes the hash, so the next run opened a
second PR with the identical bump: PR #778 next to PR #775 for
web-svc-collabora, after a fix had edited the bumped services.yml.
The branch name of a bot PR ends in the first seven characters of the
fingerprint of the change the bot proposed. An open PR whose branch ends
in the fingerprint of the fresh proposal now counts as a duplicate before
its head is compared. The head comparison stays as the second criterion.
The test runs pr.sh in a throwaway repository against a stubbed gh. It
fails on the previous script with exit 1, because that one goes on to
push.
Closes#780
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.
Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.
baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.
The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.
Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.
test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.
Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A push to main whose head commit starts with "Release version " (the subject pkgmgr writes for every release) now cancels the main run in progress, so the release starts at once instead of queueing behind it. Every other push to main still leaves the running pipeline alone.
When the pushed main commit carries a vX.Y.Z tag, the orchestrator runs with chunk_gate false: every chunk deploys and reports, and a failed chunk still turns the run red, so release-guard and release-version stay skipped.
entry-cancel-superseded.yml mirrors the new rule behind its push guard, parenthesised so && does not bind the release clause; the mirror test composes the expected rule the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.
The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.
A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.
The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
71888d86af moved twelve workflow files behind trigger prefixes and updated the
references GitHub validates. Three mentions in prose kept the old names: an echo
line, a file header and a lint's failure message. The same commit already fixed
the REST dispatch URL two lines below the echo it missed.
roles/svc-runner/files/test/external.sh:36 test-runner-smoke.yml
scripts/github/runner/cleanup_docker.sh:4 images-build-ci.yml
tests/lint/repository/test_distros_spot.py:116 test-workspace.yml
None of them executes; the workflow-reference lint deliberately does not reach
them, because outside .github/workflows a bare *.yml collides with services.yml,
compose.yml and variants.yml. A repo-wide grep over all eleven legacy names now
returns nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.
File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.
restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix spent five runners on work that is almost entirely network
wait: the collection install measured 0% CPU for its whole duration, and
the parent-image pull, the package managers and the push are no
different. One runner carries all five concurrently and frees the other
four.
Losing the matrix loses fail-fast: false with it, so push_all.sh awaits
every build and reports each one by name before exiting non-zero. A
broken distro no longer hides the other four, which is what made today's
triage find three causes instead of one.
Starts are spread by fibonacci seconds because each build opens with a
parent-image pull of several hundred MB. Per-distro logs are grouped in
the job output and uploaded as an artifact, and the pressure report now
carries free -h: memory is the one resource this consolidation contends
for, and an OOM kill would otherwise read as a build error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.
A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.
History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.
Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.
The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-net-tor aborts at roles/svc-net-tor/tasks/router.yml:2 when networks.internet.dns is empty, and rightly so: dnsmasq runs with no-resolv, so an empty upstream leaves the node answering .onion and nothing else. That value is set in exactly two places, both under inventories/development/, and the documented production path reaches neither - the provisioner writes no networks key at all. A reader following the role README verbatim therefore cannot deploy a Tor node.
CI has been proving this since 2026-08-31 without anyone seeing it: the README replay rides on the role's smallest variant and is skipped when the deploy is red, so the only row that ever reached it is one where svc-net-tor itself went green. That happened first on Ubuntu, where the local-service defect abb6f37297 fixes never fired.
The template now emits the DNS assignment, the -e passthrough and the networks entry in --vars for a role whose tasks name networks.internet.dns, which today is svc-net-tor alone; the generator derives that rather than taking a declaration, so a second role asking for the value documents it by doing so. scripts/github/guide/compose_deploy.sh substitutes the placeholder from INFINITO_DNS_IP, the same SPOT inventories/development/default.yml reads, next to the existing domain substitution.
This follows 972596a758, which added the administrator key to the same block for the same reason. No fallback inside the role: defaulting to NETWORK_PUBLIC_DNS_RESOLVERS would route a Tor node's clearnet queries to Cloudflare without the operator choosing it, which is what the assert exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The version cron updates docker image tags and repository refs, and tests/external/update covers docker, addons and repository. Neither sees a version inside an Ansible task variable, and dependabot reads pyproject.toml rather than roles/. A pin set once therefore froze and nothing thawed it: backup-docker-to-local has not moved since it was written, and the automtu pin one commit back would have gone the same way.
utils/update/pip mirrors the docker module: collect the pins, ask PyPI, rewrite in place. Only exact == pins are collected, because a range or a VCS reference states an intent an updater must not overrule. The external check warns rather than fails, like its docker sibling, so a slow PyPI cannot redden a run and a release the repository has deliberately not taken is not a regression.
The cron job is the docker job with one script swapped, so a changed pin arrives as its own PR under update/pip-pins.
Executed: the collector finds both pins in the repository; a synthetic file rewrites from 1.0.0 to 1.1.1 with the indent, quoting and neighbouring line untouched; automtu==1.0.0 would be reported while a bare name and a git+ reference are passed over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/roles/guide.py picks "compose" if the role ships a stack and "host"
otherwise. The compose branch runs make compose-up, which starts the coredns
service from compose.yml, so the replay resolves the deployment's own names.
The host branch runs this script, a bare container on the default bridge with
no resolver anywhere.
That only started to matter with 4986e80dd4, which threaded DOMAIN into the
README replay so it deploys under INFINITO_DOMAIN instead of inheriting
infinito.localhost. Before it, sys-svc-nss resolved *.localhost through
nss-myhostname and no DNS server was needed; after it, a stackless role asks
for infinito.test and gets nothing.
web-svc-html shows the whole chain in run 33923190220. Its guide replay logs
"Verify 'html.infinito.test' is served after the reload" with [Errno -2] Name
or service not known, sys-ctl-hlth-webserver probes every NGINX domain and
exits 1 through script.py's `return 1 if errors else 0`, systemd refuses to
start hlth-webserver, and sys-service-terminator fails the play after 25
attempts. The same run's Fider replay, a compose-mode role, answers the
identical task with real HTTP headers.
Rather than rebuild the resolver by hand, the script now brings up the very
service the compose branch uses and reads the network back off the container,
so subnet, gateway, MTU, image, Corefile and command all stay in compose.yml.
The Corefile is rendered through CoreDNSCorefileRenderer so the script does
not depend on an earlier phase having produced it. coredns declares no
depends_on, so the infinito service stays down and INFINITO_IP4 is free for
the guide container; the run's own log shows the CI phase removing
infinito-nexus-core_default seventeen seconds before the replay starts.
-v /srv is the second failure in the same class. svc-storage-nfs-server dies
with "mount.nfs4 ... reason given by server: No such file or directory" on a
host reporting Storage Driver: overlay2, because name_to_handle_at answers
EOPNOTSUPP there and the server builds no export root. A real volume under
the export path takes it off the overlay.
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.
Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.
The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.
A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A cancel request does not stop a running step. The runner asks it to end, and a step that ignores the signal keeps going, bounded only by its own timeout-minutes -- 350 for a deploy. Meanwhile the concurrency group holds the newer run on pending until the occupant is terminal, not until it is cancel-requested. Measured on run 32859801382: the cancel was accepted at 15:27:45 and thirteen minutes later ten deploy jobs were still running and the newer run still had not started.
The sweep now waits FORCE_CANCEL_AFTER_SECONDS and force-cancels whatever is still not completed. Cleanup steps are skipped for those, which is the price: the artefacts of an already superseded run help nobody. Only entry-cancel-superseded.yml sets the variable -- a closed pull request and a deleted branch have nothing waiting on the group, so they keep cancelling and walking away.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A long orchestrator sweep sometimes survives its concurrency group: the occupant sits on queued while its matrix waits for runners, cancel-in-progress does not reap it, and the newer run parks on pending until someone cancels the sweep by hand. PR #380 sat that way for nine hours behind a run from the previous commit.
entry-cancel-superseded.yml cancels those leftovers through the API. The group stays primary and untouched: it reaps within seconds and needs no runner, so the fallback normally arrives to find nothing left to do. It carries no concurrency group of its own, because the run holding a group cannot be the run that frees it.
Selection mirrors what the group would have done: cancel every matched run except the newest in the group instance, keyed the way the group is keyed. Not by head SHA, which goes stale while the job waits for a runner and would cancel the newest commit's CI while sparing the stale one. INCLUDE_PATHS keeps the sweep inside the group's own members, so a push to main cannot take the cron workflows with it.
deleted_branch_runs.sh becomes branch_runs.sh, since it now also serves live branches, and the shared parts move to lib.sh. A rejected cancel is logged and the remaining runs are still attempted; only a denied token fails the job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every scheduled workflow opened with an alarm clock, so the Actions list
read as a column of identical icons that said nothing about the run. The
clock now lives in run-name behind github.event_name == 'schedule', where
it marks the one thing it can mark: a run nobody triggered. The name says
what the run does, in two or three words.
The dispatch menu shows names in one narrow column and truncates them, so
a new lint caps a workflow_dispatch name at 25 characters and points the
longer, explanatory wording at run-name, which carries no such limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.
The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.
Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.
A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two scripts sharing a delimiter-bounded name prefix are a folder in disguise, and tests/lint/repository/structure/test_prefix_grouped_files.py refuses them. A second release_* resolver is coming, so give the group its directory now and drop the prefix from the name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ba8a72309 bounded scripts/install/apt.sh and d50ba46b0 bounded the swarm
bootstrap; these three were the remainder, and one of them was still
costing whole jobs. docker_dataroot.sh hung Nextcloud#7 (job 96179137223,
run 32284644282) for 5h40m: "installing zfsutils-linux for zpool" at
19:50:39 is the last line before "rc=124 after 20400s" at 01:30:39, with
nothing in between. distros.sh's global budget is the only ceiling below
these calls, so one stalled fetch also takes the untried distros with it.
Both update and install are wrapped, not just update - that hang sat in
the install, and apt-get install fetches from the same mirror. Constants
come verbatim from scripts/install/apt.sh:19-20 rather than being
re-derived.
host_deploy.sh runs inside a container via `docker exec ... sh -c`, where
POSIX sh has no arrays, so its options are a word-split string instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merge assertion was a job nobody depended on. ci-orchestrator needed the
scope, the branch prefix and the affected roles -- not it -- so a PR whose base
was not merged in went on to spend five distros on a deploy sweep and reported
a single red job beside a chain that had already finished.
Validation is now a reusable workflow every entry calls, and what it checks
follows the event and the inputs rather than the caller: a pull request gets
its branch prefix and merge base asserted, a run carrying a selection gets that
selection checked, a fork PR waits here for the privileged run's images. The
wait and its gate move in with it, so the orchestrator no longer keeps a second
copy of that logic. Every downstream job depends on the gate.
The selection check is what the last two nights needed. cli.meta.ci.validate
resolves every token against the branch that would deploy it -- role, variant,
mode, onion state -- and reports all offenders at once. Until now the matrix
builder raised on the first bad token from inside a chunk job, so a stale
priority line cost one full CI round per entry and only ever showed the next
problem after the previous one was fixed. Measured on a real list: three
unusable tokens, one pass.
A bare role name that matches nothing is a warning, not an error: the
diff-derived whitelist legitimately names roles the lifecycle envelope filters
out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A whitelist or priority entry was a role name and nothing else, so a run
could not ask for "this variant, on swarm, behind the onion". It either
covered everything the role offers or waited for the rotation to come
around to the combination that broke.
An entry is now a token that may carry the axes, either as
web-app-x#0,2@swarm+tor or as the job label pasted straight back out of
the run that failed. What a token leaves open still belongs to the line
it stands in: priority covers every remaining combination in one sweep,
whitelist keeps the rotation and its position in the matrix. A pin the
row cannot take aborts the matrix rather than dropping the row, because
a dropped row reports a green run for a combination that never ran.
The onion state spells out as +clearnet, not as a -tor suffix: svc-net-tor
would otherwise read as role svc-net on the clearnet.
Chunk membership still follows the discovery ranking, which is what
decides who makes the budget cut, but the rows inside a chunk are sorted
by display name, variant, mode and onion state, so a chunk's job list
reads like the plan table instead of like the sweep's rotation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
call-test-deploy-{compose,host,swarm,single-node}.yml differed only in
which mode they pinned and which artifact prefix they wrote; compose and
host already ran the identical drill. They become one call-test-deploy.yml
whose matrix rows carry the mode, so the swarm-specific steps are guarded
on it and everything else is shared.
The orchestrator loses its eight deploy jobs and the sequencing job for
three serial chunk blocks. GitHub Actions cannot generate a variable
number of jobs, so the blocks are written out and INFINITO_CI_MAX_CHUNKS
tells the budget how many exist; blocks whose slice is empty skip
themselves. chunk_gate stops the chain at the first red chunk,
resume_from_chunk re-enters after a fix instead of re-running the green
chunks, and modes collapses to a single mode choice.
apps.sh now emits matrix entries for one chunk rather than a token list
for one mode, and no longer sorts them: the discovery order is what the
chunker slices on. split_runner_apps.sh reads its runner quota from
INFINITO_CI_CONCURRENCY instead of carrying a second copy of the 20.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The PR-close canceller had no concurrency group at all, so cancelling
depended entirely on scripts/github/cancel/pull_request_runs.sh, which
needs a runner before it can call the API. Runners were 20 minutes to 21
hours out, and run 31451265380 stayed alive that whole time.
The canceller now joins the group entry-pr-change-orchestrate.yml renders
for the same PR. GitHub clears the group when the run is created, before a
runner is assigned, so the orchestrator dies immediately and the script
becomes a backstop instead of the only path.
The group spells pull_request out literally rather than interpolating
github.event_name. This workflow is triggered by pull_request_target, so
the interpolated form would join the twin the orchestrator's own job
guards already skip on a same-repo PR and leave the working run untouched.
A fork PR runs both twins; the pull_request_target one is still left to
the script.
The branch-delete sweep filtered runs by head_branch alone and shot this
canceller mid-cancel. It now excludes the canceller's own path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hourly sweep fought the rest of the CI control plane instead of
helping it: it preempted itself (17 of the last 29 runs concluded
cancelled 1-2s after their successor was created, for 8s of work behind
a 74-80min queue wait) and shipped MAX_ATTEMPTS=0, which the script
reads as no ceiling while its own README promised the opposite. The 24h
age cutoff could not bound it either, because every rerun refreshes the
run's updatedAt and so keeps it inside the window.
Removes the workflow, the two scripts it was the sole consumer of, its
unit test, and the CI_RERUN_* documentation that would otherwise point
at a deleted file. The workflow is disabled in both origin and fork so
no sweep fires before this lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hourly rerun bot looked only at the newest run's conclusion. When that run was cancelled while an older one was still pending, reviving it added a second waiter to the ref's global-ci concurrency group — and cancel-in-progress evicts only in_progress runs, so neither could start and every later dispatch queued behind them.
Query the branch once with status included and leave it alone while anything on it is still active. The selection itself is unchanged; it now reads the same response instead of issuing a second one.
Scope, measured rather than assumed: this does NOT explain today's deadlock. The blocking revival happened at 14:51 on an idle branch, which the guard permits, and the later revival at 15:51 never produced a second attempt (run 31405424413 is still attempt 1). The guard covers a narrower case than the incident that motivated it.
scripts/github/rerun had no tests at all. The harness added here runs the script against a stubbed gh, following the pattern of tests/unit/scripts/tests/deploy/utils/test_rescue_index.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both were created without the executable bit, which every sibling under scripts/ carries and which tests/lint/filesystem/shell/test_executable.py enforces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.
Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.
Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run cancelled by a concurrency group loses the work it had already finished. An hourly sweep now walks every branch, and when its latest run ended cancelled it restarts only the failed jobs through gh run rerun --failed, keeping the successful ones.
CI_RERUN_INTERVAL_MINUTES, CI_RERUN_MAX_ATTEMPTS and CI_RERUN_MAX_AGE_HOURS tune the sweep. A cron expression cannot read a repository variable, so the workflow is triggered every full hour and a gate lets only the hours through that open the configured window; the gate reads the hour, not the minute, so a late start still counts for its hour. An unset attempt ceiling means no ceiling, bounded only by the 50 reruns GitHub allows per run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every conflicted hunk resolves onto the swarm-nfs side, because on all six
files it is the strict superset: it carries the display-name codec
(utils/roles/display.py) the CI role selections now encode through, and the
sequencing glyph move from 910748ec4. svc-net-tor holds no unique content on
any of them -- runs.py even had non-conflicted hunks already calling `codec`,
so keeping its side would have left a NameError.
The lint fallout is settled in the same commit, as 4afc9b770 did for the
previous merge: the shell-or-true lint lives on svc-net-tor (c505ce746) and
swarm-nfs brings two unmarked `|| true` sites with it. Both get a nocheck
reason rather than a rewrite, because neither swallows anything -- the
slaptest call is the measured exception of c4bbbba93 and has_cn_config decides
right below it, and the dockerd restart is followed by an is-active poll that
fatals. In both the exit code was never the authority.
make quality green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI deploy jobs were titled with the raw role id, so a run page read as a
column of dash-joined prefixes. They now carry a display name: the category
path as hanzi labels, then the role's README heading or the part of the id
the categories do not already say, whichever is shorter.
utils/roles/display.py owns both directions. Encoding happens once, at
discovery time, as a `label` key on every matrix entry; the workflows render
`<glyph> ${{ matrix.label }}` and keep the glyph as a literal because the
retrigger reads the deploy mode off it.
Decoding is what makes the retrigger keep working. It accepts a display name
with or without its variant shard and a bare role id unchanged, so job names
from runs that predate this still parse and a hand-typed dispatch never has
to spell hanzi. The parser therefore stopped matching an id-shaped tail and
asks the codec instead; a job the codec does not recognise is skipped rather
than counted as a role.
The role selections go out as display names too, so the run title reads like
the jobs it spawns. Four consumers decode them back, each the single place
the value is actually used: the whitelist resolver before it checks the role
directory, the discovery query on its env, and the sequencing and plan CLIs
on their arguments.
Every fragment is joined by `·`, including before the title, and the heading
loses its whitespace and any emoji variation selector. A display name holding
a space would make the space-separated whitelist and priority inputs
unparseable with more than one role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The REST API answers inputs: null for a finished workflow_dispatch, so
the run name is the only record of what a manual run was dispatched
with. It carried one value, distros; everything else silently fell back
to the workflow defaults, and a failure that only reproduces on swarm
with btrfs came back green from a run that deployed all three modes on
whatever filesystem it drew.
Render every input as its own <emoji><value> segment instead, omitted
when it holds its default, so the title carries exactly what deviates
and an absent input means 'leave it on the default'. run_name derives
those segments from the workflow rather than knowing them, so the format
cannot drift away from the parser without a failing test.
distros no longer defaults to debian -- empty draws one at random, the
way untagged pushes already do. That leaves nothing in the title, so the
effective list is recovered from the discover job names, matched by
content against the meta/distros.yml SPOT rather than by job-name shape.
Priority roles that fell behind the budget cut in every mode hold no job
at all, so failed_roles cannot see them: they are neither green nor red,
they never ran. Carry them along or they never deploy.
--run now resolves before the selection, so it reproduces a run whatever
picks the roles, not only with --failed.
tests/utils/ci_*.py move into tests/utils/ci/ -- two files sharing a
delimiter-bounded prefix are a folder in disguise, and the structure
lint says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The REST API answers inputs: null for a finished workflow_dispatch, so
the run name is the only record of what a manual run was dispatched
with. It carried one value, distros; everything else silently fell back
to the workflow defaults, and a failure that only reproduces on swarm
with btrfs came back green from a run that deployed all three modes on
whatever filesystem it drew.
Render every input as its own <emoji><value> segment instead, omitted
when it holds its default, so the title carries exactly what deviates
and an absent input means 'leave it on the default'. run_name derives
those segments from the workflow rather than knowing them, so the format
cannot drift away from the parser without a failing test.
distros no longer defaults to debian -- empty draws one at random, the
way untagged pushes already do. That leaves nothing in the title, so the
effective list is recovered from the discover job names, matched by
content against the meta/distros.yml SPOT rather than by job-name shape.
Priority roles that fell behind the budget cut in every mode hold no job
at all, so failed_roles cannot see them: they are neither green nor red,
they never ran. Carry them along or they never deploy.
--run now resolves before the selection, so it reproduces a run whatever
picks the roles, not only with --failed.
tests/utils/ci_*.py move into tests/utils/ci/ -- two files sharing a
delimiter-bounded prefix are a folder in disguise, and the structure
lint says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve conflicts, resolved by intent rather than by side.
Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.
web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').
handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.
svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.
Six follow-on failures the merge produced, fixed at the root:
- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
the prefix lint forbids; the pair now lives in utils/github/variant/ as
bundles.py and bundle_limits.py, with the -m entry point, the ruff
per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
three bare timeout literals; both route through gotoOnion and resolveTimeout
now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
distros that ship docker without it, the loop device is already detached when
the probe failed before attaching, and modprobe zfs may fail because the
/dev/zfs check decides.
make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
variant_bundle_limits.py next to variant_bundles.py formed the prefix group
'variant', which the structure lint reads as a folder in disguise. It stayed
green locally because that lint asks git ls-files: the new file was untracked
through every quality run and only became visible once committed. CI saw it
immediately - the sole failure of run 30675284663.
Both move to utils/github/variant/ as bundles.py and limits.py, and the test
mirrors along.
The BLE001 per-file ignore is removed rather than repointed. parse_size raises
humanfriendly.InvalidSize for a malformed string and TypeError only for a
non-string, and the value reaching it is always str - so the catch-all was
never needed. Measured, not assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rescue artifact is a tarball of whatever the collectors managed to grab, so
reading a failed run meant downloading it to find out which files exist at all.
Run rescue_index.sh right after each collection instead, so the job log names
the tree the artifact contains.
All four sites call it without a fallback, which is what the preceding commit's
exit-status contract is for. host_deploy.sh and one.sh hoist the rescue path
into a variable first - it is now needed twice, once for the collector's
INFINITO_RESCUE_DIAGNOSTICS_DIR and once for the index.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CI deploy bootstrap hand-listed two packages and let the ubuntu-latest runner image supply the rest, so a new host-side import surfaced as a red deploy job instead of a failed install. Size [project.dependencies] to what runs outside the project image and move the deployment stack into a deploy extra, then have every install site read that instead of naming packages itself: the image build and make install-python take the deploy extra, the CI host and the mirror and instructions workflows take the base.
Declare cryptography, which utils/manager/value_generator.py imports directly and which only reached the environment transitively through ansible. Drop numpy, which nothing in the repository imports. Guard the boundary with a lint test that blocks every deploy package and asserts the runner-host CLI still builds its parser.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, none resolved by picking a side wholesale:
- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
rc now counts as failure; the previous default(0) let an aborted Playwright
run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
which this branch's lint enforces, and adopted the incoming retry-inside-poll
for the OIDC button plus its networkidle wait.
The pipeline gate only listed twelve of seventeen jobs in its needs, leaving out wait-fork-prereq-run, fork-prereqs-ready, build-ci-images, images-mirror-missing and test-dns. A failure in any of them turns its dependents into skipped rather than failure, and the gate's check matches only failure and cancelled, so the run would have concluded green without images ever being built.
A caller-supplied whitelist was also passed through verbatim. Run 30492550883 was dispatched with web-app-prometheu; discovery dropped the unknown id without a warning, the empty resolution became [], the deploy workflows skipped their whole matrix on that condition and the parent job still concluded success. Validate every id against roles/<id> and exit naming the offenders, so a typo cannot quietly remove an application from a run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The swarm branch predates the shell-or-true lint, so its EXIT-trap collectors
arrived unmarked. They are the legitimate case: best-effort diagnostics and
teardown that must not mask the original exit code, which the trailing
return "${rc}" preserves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
