The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.
Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.
baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.
The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.
Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.
test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.
Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
019fb572a7 minted the app token before the script could decide whether a push
follows, so every repository paid for a job that mostly no-ops. Run 37169357364
showed the cost on the fork: BOT_APP_CLIENT_ID is unset there, the token step
failed, and sync-main went red earlier than before rather than green. On
infinito-nexus/core the script always takes the source-equals-target shortcut,
so every push on every branch would have drawn a Workflows:write token for
nothing.
The script now resolves its own decision and writes sync=true|false to
GITHUB_OUTPUT when CI_SYNC_MAIN_DECIDE_ONLY is set. The workflow asks first and
gates both the token and the push on the answer, which is also what makes the
documented off-switch work: with CI_SYNC_MAIN_SOURCE_REPOSITORY disabled the job
now ends green instead of failing on an absent secret. The decision stays in the
script, so the workflow does not restate the normalization or the disabled-value
list.
The push takes its credential from CI_SYNC_MAIN_PUSH_TOKEN through a credential
helper rather than a remote URL or an extraheader: .git/config holds the
variable name and the helper expands it at call time, so the token reaches
neither argv nor the config file. Checkout runs with persist-credentials: false
because its extraheader would otherwise outrank the helper.
Verified by running all four paths: source equals target and disabled both emit
sync=false and exit 0, a mirroring fork emits sync=true, and push mode without a
token fails loudly. git credential fill against the configured helper returns
username=x-access-token and the value from the environment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourteen real conflicts plus the 66 generated READMEs.
utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.
plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.
roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.
group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.
docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.
inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.
roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.
The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
964f3255f5 made the registry gate judge an image by the architectures it serves rather than by its existence, but the release retag stayed outside that gate: buildx imagetools create happily writes a version tag and latest from an incomplete source, and the gap only surfaced when someone pulled on the missing architecture.
Both freshly written references are now asserted through the same helper, which reports an unreachable manifest as serving nothing and therefore fails rather than passing in silence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A checkout that declared a new cache host rendered the artefacts and reached nothing: the Nexus proxy repository, the frontend's leaf certificate and the nginx upstream map all live in the running stack, which only the owning checkout set up. cache-apply installs all three and reloads the frontend, and compose up runs it for a checkout that merely joins the network, so the manual call is left for a declaration changed between two bring-ups.
The map source is no longer selectable. INFINITO_CACHE_CONF_SOURCE existed to let a worktree read the primary checkout's map instead of its own, but the frontend mounts one map and the install replaces it, so the checkout that applied last wins either way: the knob chose which file to point at, never who prevailed. Dropping it also retires the git subprocess that resolved the owning checkout on every env build.
Two checkouts sharing a stack therefore overwrite each other's map, which the documentation now states instead of implying a selector could prevent it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A push to main whose head commit starts with "Release version " (the subject pkgmgr writes for every release) now cancels the main run in progress, so the release starts at once instead of queueing behind it. Every other push to main still leaves the running pipeline alone.
When the pushed main commit carries a vX.Y.Z tag, the orchestrator runs with chunk_gate false: every chunk deploys and reports, and a failed chunk still turns the run red, so release-guard and release-version stay skipped.
entry-cancel-superseded.yml mirrors the new rule behind its push guard, parenthesised so && does not bind the release clause; the mirror test composes the expected rule the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entity name keys a role's instance directory, its compose project and its container name, and was derived only by stripping the longest matching category prefix. Two roles under different categories whose names share a suffix therefore collapse onto one entity and overwrite each other's rendered stack, which no provides: can repair.
A literal entity_name in vars/main.yml now wins over the derivation. web-app-seaweedfs-console takes its shorter name back and declares the entity seaweedfs-console, which frees the seaweedfs key in its own services map for the engine consumer and removes the provides:, covers: and alias entries that stood in for it. A new lint pairs the declaration with the primary key it has to match, because a drift between them would drop the role from the provider set in silence.
Compared across all 288 roles, exactly one entity changes, and it restores what main already keeps on disk, so no instance directory moves.
The function and the filter lose their get_ prefix. Two lints read that prefix as the discriminator between the variable and the filter that fills it; both discriminate on the pipe now. Ansible's templar resolves a filter and a same-named variable independently, verified against ansible-core 2.20.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
121 lines of restated code, step narration, section banners and rationale
prose, in workflow YAML, compose overrides, group_vars, lookup and action
plugins, role tasks, shell scripts and tests.
Three lines go the other way: where a reason has to survive it is now a
registered nocheck directive carrying it, not free prose. Two of them point at
an invariant further down the same file, the third records why an end-of-life
object store keeps no Playwright surface.
Scope was decided per line rather than per file. Classifying by "no additions"
would have swept roles/web-app-erpnext/tasks along with it, where the removed
lines are delegate_to, environment and cmd — a behaviour change that has no
business in a comment commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every check asked `docker manifest inspect` whether a tag resolved. A tag
serving only amd64 answers yes, so the arm64 half of the pool could be
absent from `latest` while the release, the CI-image wait and the cron
picker all reported the images present.
`architectures.sh` answers the narrower question instead: given a reference,
which architectures must it serve that it does not. It subtracts the
manifest index's platforms from `resolve_architectures()`, so the pool stays
declared in one place and an absent tag and a single-architecture tag give
the same answer to every caller.
`highest_release_tag.sh` checks `latest` before it walks the version tags.
The published install instructions name the image without a tag, so a reader
pulls `latest`, and only a release of the highest version republishes it: an
architecture missing from `latest` is therefore reported as that version
missing. Older tags stay an existence check, because their source predates
the architecture and no rebuild for it is available.
The CI-image wait is handed the run's own architecture list, the same
expression the build matrix and the manifest step already read, so it cannot
demand more than the run produced. The two release workflows pass no list and
therefore require the whole pool, which is what drives the rebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instructions test brings up a machine per role, so it can only ever
replay one guide. Which one it picked was a side effect of ordering: the
lookup returned the first invokable peer the round listed, and on a CI row
that is the role under test only because the row's apps happen to read
<role>,web-app-docs.
It now reads APP_ID, which the deploy workflow already sets to the row's app
and which the swarm routine already documents as the app under test. A round
that carries no APP_ID, or one whose app publishes no Production block, draws
an invokable peer at random instead, so a local round covers a different
guide each time rather than always the first one the inventory lists.
Ansible runs inside the infinito container rather than on the runner, and
that exec passes an explicit allowlist, so the name arrives only where it is
forwarded. Two seams carry it: _FORWARDED_ENV_KEYS, which covers CI and the
local initialize and reinstall verbs, and the update verb's own docker exec,
which assembles its arguments without that CLI.
The pin is tolerant rather than strict. APP_ID is set on every deploy row and
not only on documented ones, so refusing an app that publishes no guide would
let a generic variable abort a deploy.
Measured: the draw is observed over 200 rounds rather than assumed, and the
forwarding is negative-controlled, removing the key and watching the new test
fail on that key alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The package cache was configured in six places at once: the nginx upstream
map, the consumer compose override, pip.conf, .npmrc, the apt sources and the
twenty ensure_proxy_repo calls that bootstrap Nexus each spelled the same
hostnames, repository names and suites by hand. A new upstream meant editing
five files and hoping none was missed, and the docs told contributors to do
exactly that.
A role now declares what it needs under cache: in its meta/networks.yml, and
everything else is derived from it. svc-cache-package holds the general
upstreams; web-svc-libretranslate holds its own. All nine derived files are
generated and gitignored, so a second spelling of a hostname cannot exist.
utils.cache.hosts reads the declarations without PyYAML, because the .env
generator reaches them before any dependency is installed; it raises rather
than truncate when a comment sits at column 0 inside the block, which would
otherwise drop every declaration below it in silence. utils.cache.records
owns the Nexus record format and the Ubuntu mirror resolution, so the dev
stack and the role provision from one rendering.
A build may only resolve a cached host it can verify: the plain-HTTP hosts
are hijacked for every image, and the TLS hosts only for an image whose
Dockerfile installs the frontend CA. Hijacking all of them would fail every
pip and npm build with CERTIFICATE_VERIFY_FAILED.
svc-cache-package deploys the cache and its TLS frontend by calling the same
shell scripts the dev stack calls, rather than reimplementing them as tasks.
The role is invokable through the new svc-cache category and inert until a
host carries it.
Verified: the twenty Nexus payloads, the fifteen nginx server blocks and all
seven package-manager configs are byte-identical to the hand-written files
they replace, and the new category adds one role to the deploy order while
preserving the relative order of the other 180.
The role has not been deployed yet.
The replay of a role's "### Production" block moves out of the CI guide
scripts and into the documentation role itself. The block is read off the
rendered page over HTTP, so what runs is what a reader copies, and the role
that publishes the page is also the role that proves it.
scripts/github/guide/{one,compose_deploy,host_deploy}.sh and
cli/build/docs/readme/production.py are gone with the workflow step that
called them. The instructions marker now only adds web-app-docs to the row's
apps; test-e2e-cli discovers the role through its new templates/test.env.j2
and runs files/test/test.sh.
The block runs in a throwaway machine, never on the stack host. compose.yml
starts one service built from files/test/Dockerfile, whose base is
INFINITO_PARENT_IMAGE out of the mounted checkout's .env, so the machine is
the stack host's own distribution and the replay exercises the installation
instead of a prebuilt image. files/test/init.sh adds an init when the base
carries none and keeps the full -Syu that ef0f03ea98 established for arch.
scripts/tests/workspace/base/01_install.sh then prepares the machine before
the block runs.
Nothing is skipped any more. The previous host-install branch exited 0 and
passed silently over 65 of the 178 invokable roles; a block without a
"docker run" is now replayed against systemd in that same machine, with the
git clone dropped in favour of the mounted tree under test.
The target role comes from the new guide_role lookup, which reads
list_invokable_app_ids() rather than matching role-name prefixes, and falls
back to the asking role when the round deploys nothing else. deployment.py
grows running_apps() so both lookups share one definition of what the round
deploys. required_by_coverage.role_is_invokable drops its own walk over
categories.yml and delegates to the same SPOT; measured equal across all 280
roles.
Carried in the same index, because the refactor pass ran over this change
set: library.py splits into sites.py and build_queue.py mixins, builder.py
hands translated_languages to catalogs.py and folds three repeated log-tail
and state writes, and placeholders.py becomes a facade over spans.py,
repairs.py and damage.py. No test changed for those splits, and the two
ref-resolution dicts builder.py carried are replaced by _wanted_ref, which
also stops hashing the deployed tree on every tag build.
Verified: 7132 unit and lint tests pass over the whole tree, the guide_role
lookup renders through a real Templar, the base image resolves from .env,
and compose config validates the service. The replay itself has never been
executed; that needs a deploy of web-app-docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cosmos, Quick Setup and Credits follow from meta/services.yml, categories.yml
and galaxy_info.author, yet 279 role READMEs carried a copy that went stale
whenever the metadata moved and nobody re-ran make docs. The documentation
build now runs the generators itself, against its own scratch checkout, and
the committed READMEs keep only the prose a person writes.
utils.cache.PROJECT_ROOT derives from the module's location, so putting the
checkout first on PYTHONPATH makes the generator operate on the scratch tree.
i18n.py and builder.py add it; the one remaining coupling, a relative_to over
PROJECT_ROOT in a log line, is gone.
CI extracted the "### Production" block from the committed README with awk.
Stripping the section made guide_deployable() return "" for every role, which
selects no deploy row and turns nothing red: a fail-open regression. It now
asks _is_role_invokable, which matches the old condition exactly (178 roles
either way, both difference sets empty), and cli/build/docs/readme/production.py
renders the block for the two guide scripts. All 178 renders are byte-identical
to what HEAD served, and both scripts' extraction pipelines produce identical
output for a web app, a service and a desktop role.
test_readme.py inverts: the three sections are now a hard failure in a
committed README. readme_md.md documents what each is derived from.
Removed: the docs, cosmos, readme-generate, readme-index and readme-check
targets, the five aliases pointing at them, and docs from make quality.
Verified by running generate_commands() in series against a copy of the
checkout: every command exits 0, no README is left without Cosmos or Credits,
and exactly the 178 invokable roles get a Quick Setup. The ansible_roles step
needs pandoc, which only the docs image carries, so that one link of the chain
is unexercised here; it is unchanged by this commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`roles/` set as inline code reads as a literal path, yet the directory
holds an index page the build resolves. Writing it as a link costs nothing
and gives the reader the page; leaving it as code hides it, which is how
this branch degraded six working references in one pass.
The lint strips markdown links from a line before scanning it, or it would
report the code span inside a finished link, and it skips fenced blocks. It
reads the index filenames from the same place the link lint does.
Two false-positive classes came out of the first run and shaped the rule. A
span of nothing but dots and slashes is a path word, not a path: '/' names a
filesystem in a table of mount points and '../' names the traversal an
argument must not perform. An absolute span is skipped because this
repository writes Keycloak group paths that way, and /roles/web-app-bluesky
is a group, not a folder.
35 spans across 12 files become links, and the documentation build resolves
every one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three shapes of the same failure, taken from the build's own message list
rather than from a reading pass.
Eight unescaped inline characters in prose: a '*', '**', backtick or '|'
opens markup that never closes. 'roles/*/vars/main.yml', the regex
'^(web-|svc-).*', 'scripts/system/tls/trust/*.sh'. In ansible_log an inline
literal ended on a space, which reStructuredText forbids; the space carried
meaning, so it moved into the prose.
Thirteen example blocks after a single ':', which makes the indented block
a block quote instead of a literal one.
Three bullet lists with no blank line to the paragraph above, which
docutils reads as a continuation of that paragraph.
Two markdown files carried a '---' where reStructuredText allows no
transition: directly after a heading, and at the end of a document.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-svc-compose already emits a transient build.extra_hosts override for every
service that has a build, driven by INFINITO_CACHE_PACKAGE_FRONTEND_IP and
gated on the frontend CA being present. Its host list held the three apt hosts
and not argos-net.com, so 3af579449b rebuilt the same thing inside the
libretranslate role.
test_no_ansible_env_lookups found it: a role may not pull a default.env key
with lookup('env', ...), because default.env is the dev and CI contract rather
than something deployment-agnostic Ansible should reach into. Adding the host
to the shared list removes the rule violation and the duplicate in one move,
and with it LIBRETRANSLATE_CACHE_FRONTEND_IP, LIBRETRANSLATE_BUILD_HOSTS and
the container_extra_hosts call.
The URL lint probes every URL the repository mentions and a bare GET on
argos-net.com answers 404, measured, while the model paths Nexus prefixes
underneath /v1/ answer 206. The base URL is what Nexus needs, so the line is
marked the way the deb.debian.org redirector two lines above it already is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
argos-net.com hosts the .argosmodel files the LibreTranslate image pulls
at build time, 70 to 150 MB each and around 100 of them, so every rebuild
refetched several gigabytes from upstream. Route the host through the
existing Nexus raw proxy and the nginx MITM frontend that already covers
pypi, npm, alpine and the GitHub raw hosts.
The argos server block overrides the global proxy_read_timeout. Nexus
emits no byte of an object until it has fetched and stored the whole
thing, so the wait for the response header is the wait for the entire
cold fetch, which outran the global 300s once eight download lanes
competed for the upstream.
INFINITO_CACHE_PACKAGE_FRONTEND_CONF replaces the hardcoded relative bind
for the frontend's upstream map. One cache stack serves every checkout on
the host, so the map follows the primary checkout by default;
INFINITO_CACHE_CONF_SOURCE=worktree points it at the current one to test a
change before it merges. dotenv-force threads that variable through its
env -i allowlist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
server() now resolves a translation endpoint in three steps: use the running
service, rebuild when the host has a GPU but no accelerated deploy left its
stamp, otherwise deploy. A plain run therefore costs nothing, and the rebuild
fires once, on the move from a CPU-built container to an accelerated one.
deploy() brings the runner up, waits for State.Running and deploys the role
inside it. The exec steps address `docker compose exec i18n` directly:
cli.administration.deploy.development exec hardcodes the `infinito` service
(compose.py:261), so every deploy through it silently ran in the main
container, where mailu already holds port 25 and postfix cannot start.
check=True stops a failing step instead of carrying on against the wrong
target.
The deploy router writes its pid to build/deploy.pid, which survives the exec
into the subscripts, and deploying() refuses a translation while that process
lives. The running deploy is left untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router now prints one line before it routes:
=== compose-deploy: mode=reinstall apps=svc-ai-litellm bundles= variant= \
full_cycle=false purge=false disable= playwright_keep=
Rewriting a deploy command is where its other arguments get lost. Switching
apps= from svc-ai-agent-broker to svc-ai-litellm dropped
disable="matomo,dashboard,prometheus,email,css" and
INFINITO_PLAYWRIGHT_KEEP=true in the same edit, after three deploys that had
carried both, and the run went on to deploy matomo and dashboard. That matters
beyond the wasted minutes: test-e2e-cli runs in the destructor stage, after
every role, so the providers are paid for before the CLI test the deploy exists
for even starts.
main.sh is the single entry every compose-deploy passes through, so the line
cannot be skipped and does not depend on remembering to look. It reads the
knobs and exports nothing; 8e293bd921 is on record for what happens when these
same variables reach places that persist them.
compose.md loses the checklist this first carried. The rules were already
stated twice in that file and a third statement is not what was missing; the
misses were not caused by not knowing them. It keeps the two facts that carry
information: why dropping disable= delays the evidence rather than merely
costing resources, and that full_cycle=true is the expensive form rather than
the complete one, since it adds pass 2, credential rotation and the recovery
drill while the CLI tests gate on RUNTIME instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
load.sh treats every already-set INFINITO_* as caller intent and restores it
after sourcing .env. In a worktree those values come from a different checkout,
so make requested the primary instance's subnet and docker refused the network
with "invalid pool request: Pool overlaps with other one on this address
space". Five retries, then the deploy died.
INFINITO_INSTANCE names the slot a value belongs to. When the environment
carries a different one than this checkout's .env, the whole set is foreign and
is dropped before .env loads. A caller in the same slot still wins, which eight
scripts under scripts/ rely on, and an unreadable .env yields no slot and
leaves the previous behaviour untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pull request against main extracts the core and docs domains, fans out
over the languages LibreTranslate supports, and opens one translation PR
per language against the branch that introduced the strings. Only the
entries a catalog leaves empty or fuzzy reach the translator.
The job matrix and the translator read the same list through
translatable(), so a code cannot enter the matrix and then be refused by
the run it starts.
The pipeline stays off until CI_ENABLE_AUTO_TRANSLATIONS is set, and
pull requests from a fork are skipped because the app token cannot push
to their head branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
git diff reports no change for a path whose only change is an untracked
file, while open/pr.sh stages that path with `git add` and commits it.
Every gated job therefore skipped its PR step whenever an updater added
a file instead of editing one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. Twelve pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
769163ca7 computed DASHBOARD_URL and MATOMO_URL from INFINITO_DOMAIN
right after load_repo_env in common.sh. The orchestrator sources
common.sh inside the bare runtime image before Python is installed,
where load_repo_env returns without loading the env, so the guard
aborted the debian, ubuntu and fedora workspace jobs before their first
step.
common.sh now offers stack_url, which reads INFINITO_DOMAIN when it is
called. The compose steps resolve their URLs in plain assignments after
their own source of common.sh, so a missing domain still aborts the step
instead of handing an empty URL to an assertion that expects 000.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The recovery added in 2f5f52bd1 ran 'npx puppeteer browsers clear',
which asks yes/No on stdin and deletes nothing without a terminal. A
cut-off download therefore still left chrome-headless-shell without its
executable, and puppeteer kept refusing every later install of that
build, so lint-mermaid failed on every run.
A failed provisioning now removes the chrome-headless-shell cache folder
itself, the step puppeteer's own error message asks for, and retries.
Against a stack whose cache still held the broken folder, all 287
Markdown files rendered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chrome-headless-shell download had no time limit, so a stalled
network kept lint-mermaid, and with it the whole quality gate, waiting
for 148 minutes. The install now runs under a 900 second timeout.
An interrupted download leaves the browser folder without its
executable, and puppeteer then refuses every later install of that
build. A failed provisioning therefore clears the puppeteer browser
cache once and tries again before falling back to the existing warning.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Building every architecture in one job through QEMU took the image job
from under ten minutes to over two hours: each arm64 layer after
'COPY .' reran emulated on every commit, on a four-vCPU runner.
Each architecture builds on its own runner again, but push.sh now
pushes by digest only and records the digest per distro and
architecture, so no per-architecture tag exists. A merge job collects
every architecture's digests and manifest_all.sh publishes the plain
tag as one manifest list, refusing a distro that lacks a digest for any
architecture in the pool. The build cache is per architecture again so
the two jobs do not overwrite each other's cache. The binfmt mirror
entry goes with the emulation, and ADR 0002 records the measured
reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.
The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.
A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.
The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
default.env now sets INFINITO_DOMAIN to main.infinito.test, and CI
deploys under it, but the workspace suite still asserted
https://dashboard.infinito.test. curl could not resolve that host, so
the debian and fedora compose tracks failed after every deploy had
passed. Both URLs now follow INFINITO_DOMAIN once load_repo_env has
loaded it, and an unset domain aborts instead of probing a guess.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CI images were built by one job per architecture, each pushing a
<tag>-<arch> tag, and a third job merged them into the plain tag. One
job on an amd64 runner now registers QEMU through a pinned
tonistiigi/binfmt and hands the whole architecture pool to a single
buildx --platform build, so each distro gets exactly one tag, a
manifest list from the start, and one build cache.
binfmt installs with 'all' because that list leaves out the native
architecture; naming amd64 explicitly on an amd64 host would route the
host's own binaries through QEMU. The registry assertion moves into
the build job, manifest_all.sh goes, and the cleanup stops protecting
per-architecture children that no longer exist. ADR 0002 records the
changed decision.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The filesystem install ran in the RUN after 'COPY .', so any commit
rebuilt the zfs userland: the AUR build on Arch and, on arm64, the
source RPM rebuild on Fedora and CentOS, all of it under emulation now
that one job builds every architecture. It now runs in its own layer
right after default.env and scripts/install are copied, which only
those two paths invalidate. apt-mirrors.sh moves with it and still runs
first.
Running before the Python step, filesystem.sh no longer finds a system
that step already upgraded, so its pacman call does the full -Syu
instead of a partial -Sy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
zfsonlinux serves no aarch64 metadata for either Fedora or EL. dnf5
skipped the missing repository and fell through to the source rebuild,
but dnf4 on CentOS Stream aborted every transaction that loaded it, so
the arm64 centos image died at the repoquery.
The repoquery now skips an unavailable zfs repository, and the source
path disables it on every call and removes zfs-release right after the
source RPM is downloaded. The pieces that differ between dnf
generations follow the installed one: dnf-plugins-core or dnf5-plugins,
'download --source' (dnf5 keeps it as an alias of --srpm), and 'mark
remove' or 'mark dependency'. EL enables CRB for builddep, where the
devel dependencies live, and the package snapshot moves ahead of the
repoquery so the plugins count as build tooling too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The arm64 Fedora image failed because zfsonlinux publishes Fedora RPMs
for x86_64 only, so repoquery found no zfs to pin the dummy kmod to.
An empty repoquery now downloads the source RPM of the same release,
takes the version from it, rebuilds it and installs the runtime
packages only.
Everything installed from rpm-build onwards is marked as a dependency
and autoremoved afterwards. A plain dnf remove refuses because builddep
pulls the protected systemd, and taking the snapshot after rpm-build
left gcc held by redhat-rpm-config's annobin requirement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stack answered to *.infinito.test, and the host resolver pointed that
zone at 127.0.0.1, so a worktree's stack on its own bind IP could not be opened
in a browser while the primary stack existed.
INFINITO_DOMAIN is now derived per checkout: main.infinito.test for the
primary one, and for a linked worktree its branch, lowercased with everything
outside a-z0-9_- turned into "-", below INFINITO_DNS_DOMAIN
(feature/store-i18n -> feature-store-i18n.infinito.test). The branch is read
from the worktree's gitdir HEAD instead of running git, because git refuses the
mounted checkout inside the stack container. GitHub and act runs keep the
static default, custom.env still wins.
The handler overrides the caller's value on purpose: inside the stack
container that value is the previous .env, which make setup rewrote on every
deploy, so the old domain kept feeding itself back into the next compose up.
make network-dns-setup routes each stack's domain to the INFINITO_BIND_IP in
that checkout's .env, with a local= line so AAAA answers NODATA instead of the
primary stack's ::1. It skips domains outside the base zone, which a stale
.env still carries, and fails when two checkouts claim one domain.
worktree-up prints the domain and the follow-up command.
Existing checkouts switch after make dotenv-force and a redeploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every distribution built FROM `base-debian`, so the arch, ubuntu, fedora and
centos images were all asking the registry for the debian parent.
push.sh exports INFINITO_DISTRO from MATRIX_DISTRO and then sources the env
layer, which is correct as far as it goes: load.sh snapshots caller-set values
and restores them, so the distro survives. INFINITO_PARENT_IMAGE is not set by
the caller, though, and arrives already materialised in .env, rendered for
whichever distro generated that file. Setting the distro in a loop therefore
never reaches the reference derived from it.
The new resolver renders it through utils.distros.base_image() rather than
composing the string, because test_distros_spot rejects an image reference
built anywhere but the SPOT. It sits beside local.sh, ci.sh and service_ref.sh,
which resolve the other image names the same way.
Verified: arch, fedora and ubuntu now resolve to three different parents where
they previously resolved to one. shellcheck is clean and test_distros_spot
passes, which is what proves the resolver stays inside the SPOT.
Not verified: the build step that calls it. push.sh runs only in CI, and the
base images are not published yet, so a real run still stops at the pull before
it reaches this line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The two install workflows differed in a name, one env key and the script they
ran, and the orchestrator wired each separately. They become one file.
The pkgmgr half goes with them. Installing through the package manager is no
longer the plan, and the images have already moved: since the switch to
base-images every container is built FROM base-<distro>, so that test was the
last thing still pulling a pkgmgr image. The pkgmgr CLI itself is untouched;
roles keep calling it.
The mermaid graph in .github/workflows/README.md named both files as nodes and
as edges, which test_workflow_references caught after the rename. The workflow
table in the docs follows.
Verified: actionlint accepts every workflow, and 263 tests pass across
tests/lint/repository, tests/lint/filesystem and tests/integration/meta/workflows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The axis was complete in CI and unusable from a workstation. Four separate
gaps, each invisible on a real runner, and each found by running the swarm loop
against it:
`INFINITO_ARCHITECTURES` as an exported variable did nothing, because the
workflow reads its pool from `inputs.architectures`. Passing the input alone did
not help either: ACT_ENV sets INFINITO_APP_DISCOVERY_RUNNER=host, so the
discovery that builds the matrix runs outside the act container, where a
workflow-level `env:` does not reach. This is why the distro axis is handed over
twice, as `distros=` and as `INFINITO_DISTROS`, and SWARM_ARCH now does the same
for the architecture.
The third gap is the sharpest: act does not resolve `${{ matrix.* }}` inside a
job-level `env:` block, so `INFINITO_ARCHITECTURE` reached the assertion empty
every time. Earlier runs only appeared to work because a stale local `.env`
supplied a value, and the assertion was therefore checking a leftover rather
than the row. It surfaced the moment that file was regenerated.
Fourth, act had no mapping for `ubuntu-24.04-arm`, the label RUNNERS
introduced, and skipped the job with a hint instead of failing. It maps to the
same runner image: locally the architecture is delivered by emulation, and the
runner's own hardware is not the promise being made.
`make binfmt` registers that emulation. The handler already existed for
compose-up; it gains a subcommand so the registration is reachable without a
privileged docker invocation, which is what an agent would otherwise have to
ask for every time. The emulation does not survive a swarm run, because act
starts further containers whose systemd clears the shared binfmt registry, so
this is needed often.
Without `arch=`, SWARM_ARCH resolves to the host's own architecture rather than
staying empty, which would now abort every unpinned local swarm run at the
assertion.
Verified: `make binfmt arch=s390x` takes a platform from `exec format error` to
`s390x`; `make -n swarm-zombie arch=arm64` carries arm64 through all three
levers and defaults to amd64 without it; shellcheck passes; a live run reached
`"architecture":"arm64"` with the arm64 glyph in its job title.
Not verified: that the runner-label mapping carries a swarm deploy to the end.
That run was stopped so a parallel agent could use the shared stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
INFINITO_ARCHITECTURE means two different things. In CI it is the runner's
hardware, delivered by `runs-on`. Locally it is the platform app containers are
created for, delivered by qemu. The check asserted the first everywhere, so a
local run that honoured the assignment by emulation was reported as a mismatch,
with an error blaming RUNNERS where nothing was wrong.
Each context is now checked against its own promise. On a real runner the
uname must match, unchanged. Anywhere else the host must be able to execute the
assigned platform, which is verified by running a container of it rather than
by reading /proc/sys/fs/binfmt_misc, since that directory reads as empty from a
sandboxed shell whether or not a handler is registered.
Nothing is skipped and nothing passes quietly. A host that cannot deliver the
assigned architecture fails here, in the first second of the job, and names the
two commands that fix it. Five arm64 deploys died instead at
`exec /bin/sh: exec format error` deep inside an Ansible role, each after
fifteen to twenty minutes of setup.
This replaces the `exit 0` under act that an earlier version of this change
introduced. That version swallowed exactly what this check exists to catch: a
row assigned arm64 ran entirely on amd64 and would have reported green.
Verified, all four branches: amd64 native passes; arm64 with emulation present
passes as emulated; arm64 without it exits 1 with the remedy; a real-CI
mismatch exits 1 with the RUNNERS hint. shellcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
run.sh drives every suite with -n auto and no scope, so xdist hands out single
test METHODS. Each worker is its own process, so setUpClass runs once per worker
the class happens to be scattered over, and an expensive fixture is paid as many
times as the scheduler split its methods.
The integration report shows the shape without interpretation:
TestMcpEnvRender's nine methods came back as six at 277-287s and three at
0.003s. The fast three landed on workers that had already paid. That is 1776 of
the suite's 2282 CPU-seconds spent computing one identical result six times,
twenty Ansible templar renders of the MCP providers each time.
--dist loadscope groups by class, so a class occupies one worker and its
fixture runs once.
1a908a6e5b introduced -n auto and hoisted fixtures in the same commit, so the
hoisting was undercut from the start: work moved from per-method to per-class,
and the scheduler it arrived with promptly charged it per worker instead. This
finishes that commit rather than correcting it.
Measured against the same tree, four suites, before and after:
integration 296.83s -> 219.32s
lint 105.64s -> 84.27s
unit 65.74s -> 53.02s
external 75.69s -> 71.99s
Phase 1's critical path drops from 4m57s to 3m40s. That unit and lint move too
says the pattern was not unique to the MCP class.
Coverage is unchanged, which is the point of the change rather than a side note.
All four suites report identical counts: unit 5857 passed / 9 skipped / 2301
subtests, external 19 / 2, lint 521 / 1 / 374, integration 328 / 2 / 1562.
A second effect is worth naming because it cuts the right way. Several sweeps
keep their non-empty guard in a separate method, e.g. test_fragments.py:139
proving the corpus is not empty before the sweep asserts over it. Under
--dist load those two ran in different processes with separate caches, so the
guard proved nothing about the process the sweep ran in. loadscope puts them
back together.
Residual: loadscope trades balance for locality, and here one class sets the
floor. TestMcpEnvRender holds a worker for ~250s on its own, so the suite cannot
go below that until the ~12s per render is addressed. That cost is the render
guard in utils/cache/applications.py:415 returning the merged map without
caching it, which is production config resolution and deliberately left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint fan-out listed lint-php twice, once before and once after
lint-playwright. Two commits added it independently: 82218087c1 for the PHP and
Ruby syntax checks, 6fee7ca136 for the per-language unit suites, and the second
did not see the first because lint-playwright had since been inserted between
them.
The wasted slot was the harmless half. parallel.sh keys target_start,
target_dur, target_rc and target_log by target NAME, so a repeat overwrites the
first run's log path and exit code, and the final pass reads each name once.
Two consequences: drain_one prints the second run's log twice and the first
run's output never appears, and a failing first invocation is reported green as
soon as the second one passes. A duplicate therefore turns a target into a blind
spot rather than into a slow one.
The gate showed the symptom plainly, two lint-php rows with an identical
0m 03s, which reads as a rendering quirk rather than as one target run twice.
Both halves are fixed: the list loses the repeat, and parallel.sh refuses a
duplicate before it loads the environment or starts a single make, naming the
offender and why a repeat cannot be tolerated. The two callers are Makefile:440
and Makefile:822; the test list has no duplicates, so the guard is inert there.
Negative control: `parallel.sh lint-php lint-python lint-php` exits 2 with
"duplicate target(s): lint-php" and launches nothing. The full quality-high run
afterwards dispatches 15 lint targets with lint-php once and stays green.
Residual: the guard covers scripts/make/parallel.sh only. scripts/tests/dns/
parallel.sh is a separate script with its own argument handling and was not
examined.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both architectures were claimed to work and only one was ever exercised. The
deploy matrix now carries an architecture the way it carries mode, tor, distro
and filesystem, so every run proves both.
A job cannot ask for an architecture the way it asks for a distro, because the
architecture is the machine rather than something the job installs on it. The
matrix therefore carries a runner label and `runs-on` reads it, and an
architecture without a label in RUNNERS is a build error rather than a row that
lands on the wrong hardware and still reports green. assert_architecture.sh
checks the runner it actually got, because a label that stops resolving to
arm64 would otherwise go unnoticed forever.
Assignment is rotation, not chance: two architectures against five distros and
fifteen distro/filesystem positions are coprime enough that every combination
comes round. A role narrows the pool through `architectures` in its
meta/services.yml; without one it takes whatever the rotation hands it. An
empty intersection aborts instead of silently removing the role from the run.
Images are built per architecture and pushed under an architecture-suffixed
tag, then merged into the plain tag by manifest_all.sh; pushing the plain tag
from both builds would let the last one win and send half the rows to the wrong
platform. cleanup.sh now protects the `ci-<sha>-<architecture>` children of a
protected tag, because the plain tag is an index over them and deleting a child
leaves it unpullable on that platform.
label.py is split out of axes.py, which the 500-line limit no longer fits.
Writing a job title and reading one back belong together: a retrigger that
recovers an axis wrongly replays a different deploy than the one that failed,
and a drifting artifact name makes the failure reporter download nothing. The
leading `.*` in LABEL_RE is greedy on purpose so it anchors on the last mode
glyph; a reusable-workflow caller path carries a glyph of its own, and
test_selection.py guards the case.
Verified: 313 unit tests and 48 subtests across utils/github/variant,
utils/roles and cli/meta/ci. Locally, a deploy pinned to arm64 reaches ok=1444
and its image builds log aarch64-linux-gnu paths.
Not verified here: actionlint over the seven changed workflows, and a full
`make test`. The matrix doubles the row count, so runtime and runner
availability change for every CI run from here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container this project builds installs its own python, ansible and docker
CLI, so the package manager riding along in the image it built FROM was never
used. That layer now lives in its own repository, which publishes one
multi-platform image per distribution, and meta/distros.yml points at it:
`base` replaces `pkgmgr_virgin`, and utils.distros grows base_image() where
pkgmgr_virgin_image() stood. The pkgmgr image itself stays, because the
`pkgmgr install` test still boots it.
Three references carried the pkgmgr-era tag `stable`, which base-images does
not publish: it pushes `latest` and the major release, and for a rolling
distribution only `latest`, so `latest` is the one value that exists for all
five. INFINITO_PARENT_IMAGE_TAG and the two install tests move to it.
The guard that keeps image references inside the SPOT only matched
`pkgmgr-[{$]`. Renaming the family made it blind to exactly half its surface,
which is how scripts/tests/install/make.sh kept `base-${d}:stable` while
pkgmgr.sh was caught. It now matches `base-` too.
INFINITO_ARCHITECTURE and INFINITO_ARCHITECTURES appear here because
default.env is shared with the architecture axis that follows. They are empty
and inert until it lands; the alternative was a commit whose build fails by
construction, with `base-` declared and `stable` still requested.
From here nothing builds until base-images has published: `docker pull` of the
parent answers `manifest unknown`, and every make target that depends on
build-dependency fails there.
Verified: tests/lint/repository/test_distros_spot.py passes, and the widened
pattern is shown to catch the line the old one missed rather than merely not
breaking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
71888d86af moved twelve workflow files behind trigger prefixes and updated the
references GitHub validates. Three mentions in prose kept the old names: an echo
line, a file header and a lint's failure message. The same commit already fixed
the REST dispatch URL two lines below the echo it missed.
roles/svc-runner/files/test/external.sh:36 test-runner-smoke.yml
scripts/github/runner/cleanup_docker.sh:4 images-build-ci.yml
tests/lint/repository/test_distros_spot.py:116 test-workspace.yml
None of them executes; the workflow-reference lint deliberately does not reach
them, because outside .github/workflows a bare *.yml collides with services.yml,
compose.yml and variants.yml. A repo-wide grep over all eleven legacy names now
returns nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The image runs this module from a layer that holds three files of utils:
__init__.py, install/__init__.py and install/collections.py, the slice
ef45d69dc3 cut so the source tree cannot invalidate the collection install.
67f5398985 established that the module therefore stays on the standard
library, and d4ae5cf9f0 broke it 15 minutes later by routing the reads
through utils.cache.files to satisfy the cache-read lint. The import raises
ModuleNotFoundError: No module named 'utils.cache' in that layer, which no
import inside a full checkout can show, because this repository installs
utils into site-packages.
Nothing turned red. The caller reads only the exit code, and the module's
contract is exit 1 with the missing collections on stdout, so a crash - exit
1, nothing on stdout - is indistinguishable from work to do. Every image
build since has reinstalled every collection with --force-with-deps, which
is the cost the check exists to avoid. The deploy job the traceback was
found in finished green.
The reads go back to Path.read_text with a cache-read marker each. The cache
was never earning its place here: read_text is an lru_cache keyed on the
path, and one run reads the requirements file once and one MANIFEST.json per
collection, eight distinct paths, none of them twice. Copying utils/cache
into the slice instead would work - the closure is three stdlib-only files -
but it binds a package with 51 commits in the last twelve months to the
layer that installs ansible, in front of a cache with no possible hit.
The caller now exits 3 when a non-zero status arrives with an empty stdout,
a state the contract cannot produce, so the next crash stops the install
instead of quietly reinstalling everything. all.sh runs under set -e and
touches the stamp only after ansible.sh returns, so the failure also leaves
the stamp stale rather than recording an install that did not happen.
The probe that would have caught this builds the slice in a tempdir from the
Dockerfile's own COPY lines and runs the module inside it. It has to pass
python -S: without it the interpreter resolves every submodule from the
installed checkout however the slice is built, and the test passes on a
module that cannot import in the image. Verified by injecting the import
back into the tempdir copy, which reproduces the traceback verbatim.
Delta measured by loading the committed module and this one side by side:
identical on both requirements files, against the real collections directory
and an empty one, on a missing file, on non-UTF-8 input and on a corrupt
MANIFEST.json. The one behavioural difference is that a file changed within
a process is now read fresh rather than from the memo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
