builder.py imported utils.env.handlers.infinito.ip_range directly, while
ip_range imports EnvBuilder from builder for its annotations. CodeQL
reported that as py/unsafe-cyclic-import on PR #999. The import only
runs under TYPE_CHECKING, so nothing breaks at runtime, but builder now
takes ip_range from utils.env.handlers like ORDERED_HANDLERS, the path
every other handler module already uses.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
package_cache_data is declared backup: false, which claims the volume is
disposable. It is not: /nexus-data holds the repository definitions and
the rotated admin password, and only an Ansible run ever wrote them. The
disaster-recovery drill takes the claim at its word, empties the volume
and recreates the container, and Nexus comes back as a fresh install.
Run 39137606155 shows it end to end in job 117711220177: the deploy
creates all 24 repositories (HTTP 201 each), the drill reports
"13 volume(s) get emptied: … package_cache_data …" and
"must be rebuilt by their own service", and twenty seconds later the
probe reports all 24 as missing. It is not flaky: discover_cli_roles
sorts the roles, so svc-bkp-volume-2-local always runs right before
svc-cache-package.
The same script now also runs as a sidecar that starts whenever the
cache does, so an emptied volume refills itself instead of waiting for
the next deploy. Reaching Nexus from inside the stack needs no docker
exec wrapper and has to address the cache by name, so the script
dispatches on INFINITO_CACHE_PACKAGE_EXEC_MODE and takes its REST host
from the environment. An unknown mode exits rather than guessing.
The host path is unchanged: driven against a recording stub, the
committed and the previous script issue the same eight docker calls in
the same order with the same output.
The sidecar carries no image of its own, reusing the one the package
service already pulled, and stays out of swarm, where the volume is
backed up and the claim never arises.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compose/cache.shared.override.yml attaches every worktree stack's
infinito service to the primary's network (infinito-nexus-core_default,
172.30.0.0/24) without an ipv4_address. The default network declared
no ip_range, so Docker IPAM handed out the lowest free address and the
worktree stacks took 172.30.0.2, .3, .8, .9 and, whenever the primary
container was down, .10, the primary's INFINITO_IP4. The primary's
`make compose-up` then failed with "Address already in use", and with
it `make quality-high` on main.
The default network now declares ip_range: ${INFINITO_IP_RANGE}, the
upper half of INFINITO_SUBNET. Every static address of the stack (.1,
.4 to .6, .10, .53) sits in the lower half, so a container without a
static address can no longer take one.
INFINITO_IP_RANGE is derived from the final INFINITO_SUBNET after
custom.env is applied, because a worktree pins its subnet there and
`make dotenv-force` regenerates without it in the environment. Existing
worktrees therefore get their own slot's range without a new pin.
A lint checks that every static IP in default.env and every
172.30.${slot}.N pin in scripts/system/worktree/up.sh lies below the
pool.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The lane ran litellm but never republished its port, so a host-side probe hit the compose proxy on the same number and read 502. The lane could therefore never report ready, and the oracle suite skipped itself to a green exit instead of asking a judge. A handler reads the port from the role the way translate.py already does, the runner forwards it, and TOOLS_LANE_KEYS takes it up: compose_file_args layers the override in without checking its variables, so an .env written before the key existed would abort every compose command rather than merely drop the lane.
The gateway's master key is pinned in the lane inventory for the reason the System One key already is: the provisioner would otherwise generate it into a vault the checks cannot read back, and the oracle judge is not a deployed consumer with a virtual key of its own.
serving() stops treating reachability as readiness. A lane brought up before AI_LOCAL_MODELS named a different model keeps serving the old one, and a gateway holding a key these checks cannot use looks healthy from outside while refusing every call they make. It now asks /models with the bearer token and requires the configured alias, and requires the gateway to offer every language the catalogs name, because that advertisement is what the translation run waits on. Without this the lane waved itself through and no change to its roles ever shipped.
Two lane-scoped overrides come with it. The ollama role caps itself at four CPUs, which left the host at 361 percent while twenty cores idled, so the lane gives it the host count the way the engine entry above it already does. And the lane pins the router to its declared order: the decider may send a pair the engine serves to the chat model, which a batch over 183 catalogs cannot pay for. Deployments keep the decider.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.
The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.
call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two jobs of run 39040406916 died with the runner's own annotation, 'The
hosted runner lost communication with the server'. Both left every
diagnostic step pending, so no artifact exists for either. For the swarm
one the in-band sampler ruled starvation out — 9324M of 15947M available,
stall60 0.0%, 25 seconds before the cut — and nothing in the repository
can tell the remaining causes apart.
A stuck NFS mount wedges tasks in uninterruptible D state, which shows as
I/O pressure while memory pressure stays flat, and this drill runs its own
NFS node. /proc/pressure/io and /proc/pressure/cpu were read nowhere in
the repository. They are now sampled beside memory, with the D-state count,
every ten seconds instead of thirty, and an I/O stall at or above half the
minute aborts the step while the runner is still alive to upload the
banner.
cpu reads 'some', not 'full': the full line is pinned at 0.00 system-wide,
measured 13.50 against 0.00 on this host, so reading it like memory would
have shipped a metric that can never move.
Dump runner state was switched off for swarm, so the one post-mortem that
survives a step failure never ran there. It runs now, and the swarm drill
gets the step timeout its compose sibling has had all along — 350, the
number test_distro_budget_fits_timeout already derives the distro budget
from.
The sampler moved to its own module: the additions took matrix.py past the
file-size cap, and the lint asks for a split rather than a marker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three faults, one mechanism each, that together let the oracle suite report
a verdict it never reached.
The address. lane_url handed out the runner's compose service name, but the
stack's containers resolve through the deployment's CoreDNS, which never
learns one: getent hosts tools answers nothing inside them, so every check
that asked the lane from there skipped itself. The bring-up now writes the
runner's address, picked as the one inside INFINITO_SUBNET because the
runner sits on every network it was ever brought up on, and the checks read
it from there. It is rewritten on both paths, because a restart gives the
runner a new one.
The readiness. serving judged the engine alone, so a lane whose engine make
i18n had already brought up counted as up while System One was down. It now
probes every role the caller names, which is why the runner republishes
System One's port: a container IP is unreachable from the host that decides
whether to deploy.
The verdict. A silent lane made the suite skip, and a skip exits green.
Only a deploy that is already running leaves the stack alone and exits 0
now; a crashed deploy and a deployed-but-silent role both exit 1.
The override's third :? guard is added to the gate that decides whether the
file is layered in at all, next to the one that was already missing. Without
it an .env written before these keys aborts every compose command, down to
compose-down, instead of dropping the lane. A test now keeps that list and
the override's guards in step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three-arm comparison on web-app-gitea#0 put the meshed arm at 414ms
per task over the direct arm, and the tunnel at 4% of that. What it could not
split is the rest: the arms differ both in connection plugin (docker against
ssh) and in escalation (root with no prompt against administrator with a
password sudo on every task).
A fixed set of module calls, timed either side of the transport switch with
become off and on, gives four numbers that separate the two. It costs about a
minute where a fourth deploy arm would have cost three hours, and it needs no
root login and no sudoers change.
INFINITO_SWARM_TRANSPORT_BENCH guards it, so a normal round times nothing.
The arm existed to answer one question: how much of the meshed arm's
cost is the connection plugin and how much is the tunnel. Runs 37592061960
(docker), 37753506661 (SSH on the underlay) and 37621556682 (SSH over
WireGuard) answered it on web-app-gitea#0, debian/ext4:
pass 1 5964s / 8511s / 8832s
pass 2 5319s / 7472s / 7705s
Leaving ansible_connection: docker costs +4700s, the tunnel +554s - 89.5%
against 10.5%. The answer is recorded; the scaffolding does not need to ship.
Reverts d792519725, 1b28f87b37, 89b8a8c8d1, 0470491e1a, 529c5ace43,
dd3d10a9a2, c20dd0e5f5, 458d8956e1, 63ade1e03b, d281555945 and 4adfb6e3d2.
95cfc8b7b7 is already superseded by bbfd82f89, which sets the profiling limit
upstream.
666312dfbd settled the catalog client on the dedicated LibreTranslate. One
engine cannot be compared against itself, so the direct path had no way to
improve on a language it translates badly: of the 2854 entries of the `ko`
catalog it produced 1 translated entry and 2704 `structure` rejections.
The client now asks the gateway, which fronts several engines and keeps a
history of which one survives a given signature. Protected spans are masked
as `<x id="N"></x>` before the request and declared per item, so the gateway
can reject an answer that altered them instead of passing it on. `_reask`
then retries each rejected entry with the engine that produced it in
`exclude`, which reaches a different backend rather than the same one. The
same catalog measured 17 translated entries with 2649 rejections once the
gateway checked the markup, and 331 with 796 once the retry could exclude.
The full picture a rejection needs is only available in the client: 2431 of
those 2649 structure rejections involve markup the masking hides from the
gateway, so the engine-switching decision belongs on this side of the call.
`server()` reuses a gateway that already answers and otherwise deploys the
bundle the gateway needs. It does not reintroduce the throwaway container
666312dfbd removed: there is no `docker run` here, the roles deploy through
the normal path and their models still come from the role images built
through the package cache. `redeploy=true` is offered because the reuse check
only asks whether a gateway answers, which cannot see that the gateway's own
code changed since it started. A run refuses outright while a deploy is in
progress, because both touch the same container stack, and leaves the deploy
alone.
`deployed()` additionally requires a non-empty catalogue, so a gateway that
answers while serving no pairs counts as not ready rather than as a backend
that refuses everything. `probe(url)` is split out of `deployed(root)` so one
readiness check serves a single engine and the gateway alike.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourteen real conflicts plus the 66 generated READMEs.
utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.
plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.
roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.
group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.
docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.
inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.
roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.
The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A checkout that declared a new cache host rendered the artefacts and reached nothing: the Nexus proxy repository, the frontend's leaf certificate and the nginx upstream map all live in the running stack, which only the owning checkout set up. cache-apply installs all three and reloads the frontend, and compose up runs it for a checkout that merely joins the network, so the manual call is left for a declaration changed between two bring-ups.
The map source is no longer selectable. INFINITO_CACHE_CONF_SOURCE existed to let a worktree read the primary checkout's map instead of its own, but the frontend mounts one map and the install replaces it, so the checkout that applied last wins either way: the knob chose which file to point at, never who prevailed. Dropping it also retires the git subprocess that resolved the owning checkout on every env build.
Two checkouts sharing a stack therefore overwrite each other's map, which the documentation now states instead of implying a selector could prevent it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.
Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
3f0bca4d83 replaced the YAML walk with a line scan and glob-ed the role file
by hand. test_file_spot refuses a hard-coded role-file path: the name belongs
to utils.roles.mapping, which every other reader imports it from.
The import is safe for the bootstrap layer this handler lives on, which is the
whole point of the preceding commit: utils/roles/mapping.py declares nothing
but a future-import, so its closure stays stdlib-only. That is the shape the
handler README allows and the one 77292f9041 established.
The derived registry set is unchanged, asserted against the previous value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
909e08311f derived INFINITO_CACHE_REGISTRIES from iter_role_images. That import
walks straight out of the stdlib:
handlers/__init__ -> infinito/cache/registries -> docker.image.discovery
-> cache.yaml -> yaml
and the .env generator runs on the bare bootstrap python, before any dependency
is installed, so test_bare_bootstrap refused it.
Reaching for iter_role_images was also the wrong question. It answers 'which
images do we mirror': it drops every entry without a version: and every image
whose registry is outside _MIRRORABLE_REGISTRIES. A pull-through cache needs
the opposite set -- a registry we do not mirror is one we pay full upstream
cost for on every pull, which is the case the cache exists for. Four registries
were missing for exactly that reason, all with a version: declared and all
real: docker.gitea.com (web-app-gitea), docker.elastic.co (web-app-zammad),
docker.n8n.io (web-app-n8n) and docker-registry.kixdesk.com (web-app-kix).
The handler now reads the image: values itself with a line scan. Only a literal
plain scalar is needed, a value carrying Jinja is skipped because its registry
is unknowable before the templar runs, and split_registry_and_name from
utils.docker.image.ref still owns the parsing -- that module imports only re
and typing, which is the shape the handler README allows.
Measured against the previous derivation: the old set is contained whole, the
four above are added, nothing is dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rpardini/docker-registry-proxy SSL-bumps and caches only the registries its
REGISTRIES env names; Docker Hub is hardcoded in its entrypoint. A registry the
list omits is still tunnelled, so its pulls succeed at full upstream cost and
leave no line in the proxy log.
The list was the image default and omitted mcr.microsoft.com, registry.gitlab.com
and registry.opencode.de, all three of which roles declare images on. Deriving it
from every image: in roles/**/meta/services.yml keeps it complete by construction.
BASE carries the two registries that were configured before and that no role image
names, so the switch to a derived list cannot drop them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Naming a dns server replaces the runtime's embedded one rather than adding
to it, and that embedded server is the only thing answering compose
aliases. The dev container could therefore not look up `tools`, so every
check reaching a lane service by its service name reported that nothing
answers and skipped itself: the System One suite skipped with 325 usable
pairs behind an address it could not resolve, while its service was up and
healthy.
The embedded resolver goes after the platform one, so platform names
resolve where they always did and only what that server does not know falls
through.
Its address is the one group_vars/all/08_networks.yml already holds, read
through the env pipeline rather than written a second time in compose.yml.
`read_group_path` only ever read the paths file, so the reader it is built
on now takes the file to read and it keeps its own signature.
The two docker handlers share a prefix once a second one exists, which is a
folder; they move into one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lane is no longer LibreTranslate-only, so `i18n` is the wrong name for
the runner, its compose override, its inventory and its env keys. Renames
to `tools` throughout and points the lane at
`svc-ai-libretranslate-engine`, which is the role that actually owns the
`libretranslate-engine` service key.
The six cache host paths were each written out in full. They now derive
from `DIR_CACHE`, one plain string in `group_vars/all/05_paths.yml`, via a
`cache_paths` handler; the generated values are byte-identical. The handler
is registered directly after `cache_stack` rather than late, because
`registry_cache_max_size` reads `INFINITO_CACHE_REGISTRY_HOST_PATH` and
returns early when it is empty, which silently dropped
`INFINITO_CACHE_REGISTRY_MAX_SIZE` from the generated `.env` and broke
every `docker compose` invocation that layers `cache.override.yml`.
`INFINITO_TOOLS_MODELS_HOST_PATH` derives from the same base, so the model
bind the lane mounts follows the cache layout instead of repeating it.
The inventory pins `DOCKER_BIND_HOST=0.0.0.0`: the group_vars default keys
off `DOCKER_IN_CONTAINER`, which evaluates false for the inner deploy and
would leave the lane's services on the runner's loopback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The package cache was configured in six places at once: the nginx upstream
map, the consumer compose override, pip.conf, .npmrc, the apt sources and the
twenty ensure_proxy_repo calls that bootstrap Nexus each spelled the same
hostnames, repository names and suites by hand. A new upstream meant editing
five files and hoping none was missed, and the docs told contributors to do
exactly that.
A role now declares what it needs under cache: in its meta/networks.yml, and
everything else is derived from it. svc-cache-package holds the general
upstreams; web-svc-libretranslate holds its own. All nine derived files are
generated and gitignored, so a second spelling of a hostname cannot exist.
utils.cache.hosts reads the declarations without PyYAML, because the .env
generator reaches them before any dependency is installed; it raises rather
than truncate when a comment sits at column 0 inside the block, which would
otherwise drop every declaration below it in silence. utils.cache.records
owns the Nexus record format and the Ubuntu mirror resolution, so the dev
stack and the role provision from one rendering.
A build may only resolve a cached host it can verify: the plain-HTTP hosts
are hijacked for every image, and the TLS hosts only for an image whose
Dockerfile installs the frontend CA. Hijacking all of them would fail every
pip and npm build with CERTIFICATE_VERIFY_FAILED.
svc-cache-package deploys the cache and its TLS frontend by calling the same
shell scripts the dev stack calls, rather than reimplementing them as tasks.
The role is invokable through the new svc-cache category and inert until a
host carries it.
Verified: the twenty Nexus payloads, the fifteen nginx server blocks and all
seven package-manager configs are byte-identical to the hand-written files
they replace, and the new category adds one role to the deploy order while
preserving the relative order of the other 180.
The role has not been deployed yet.
The README generator read its author URL out of the ## Credits sections it
writes itself. .mailmap now folds every address a person commits under into
their @infinito.nexus one, whose local part names their social profile, and
that profile becomes the single point of truth for an author.
utils/meta/identity.py parses .mailmap rather than shelling out to
git check-mailmap, which refuses the worktree the test container mounts.
Cross-checked against git over all 14 identities in the history: no
disagreement.
A new env handler carries the committing identity into the container, which
mounts the checkout but not the contributor's git configuration. Without it
both new tests would skip exactly where they are meant to fire:
* tests/integration/meta/test_commit_identity.py fails when the configured
address does not fold, and prints the .mailmap line to add.
* tests/external/repository/test_author_profile.py probes that profile
live, behind a canary so an unreachable instance is not reported as a
missing profile.
Both look at the current address only. Probing every declared author would be
red for people who have not created a profile yet, and a suite that is red for
someone else's omission stops being read.
Marko Pjevac is declared as a role author but has no address here, so his
Credits line stays unlinked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`roles/` set as inline code reads as a literal path, yet the directory
holds an index page the build resolves. Writing it as a link costs nothing
and gives the reader the page; leaving it as code hides it, which is how
this branch degraded six working references in one pass.
The lint strips markdown links from a line before scanning it, or it would
report the code span inside a finished link, and it skips fenced blocks. It
reads the index filenames from the same place the link lint does.
Two false-positive classes came out of the first run and shaped the rule. A
span of nothing but dots and slashes is a path word, not a path: '/' names a
filesystem in a table of mount points and '../' names the traversal an
argument must not perform. An absolute span is skipped because this
repository writes Keycloak group paths that way, and /roles/web-app-bluesky
is a group, not a folder.
35 spans across 12 files become links, and the documentation build resolves
every one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three shapes of the same failure, taken from the build's own message list
rather than from a reading pass.
Eight unescaped inline characters in prose: a '*', '**', backtick or '|'
opens markup that never closes. 'roles/*/vars/main.yml', the regex
'^(web-|svc-).*', 'scripts/system/tls/trust/*.sh'. In ansible_log an inline
literal ended on a space, which reStructuredText forbids; the space carried
meaning, so it moved into the prose.
Thirteen example blocks after a single ':', which makes the indented block
a block quote instead of a literal one.
Three bullet lists with no blank line to the paragraph above, which
docutils reads as a continuation of that paragraph.
Two markdown files carried a '---' where reStructuredText allows no
transition: directly after a heading, and at the end of a document.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint accepted a link that resolved to a file or a directory. myst
builds a document reference only when the target is_file(), so a link to
a directory without a README is a bare file listing on GitHub and a dead
cross-reference in the documentation build. It now reports which of the
two reasons applies.
The tightened rule found 41 links. Where a README backs the directory,
bd15eae9fa's resolver carries the link and it stays. Where none exists,
the link becomes a code span rather than an invented README: 15 of them
point at a role's meta/addons/, whose addon table already sits below the
sentence, and the rest at code directories such as utils/ or
cli/administration/deploy/runner/.
The changed sentences are msgids, so these entries return to the
translator on the next extract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
subprocess check=False suppresses a non-zero exit code, not a missing binary: Popen raises FileNotFoundError before an exit code exists. The test-install CI job has no docker CLI on PATH, so make dotenv died on all five distros. Move the probe into a helper guarded by except OSError, matching utils/env/runtime.py and the cache conf handler, and fall back to INFINITO_GPU_COUNT=0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Read through the cached file helpers instead of around them in names.py and the i18n tuning handler, shorten the cache-conf COMMENT below the length limit, declare INFINITO_CACHE_CONF_SOURCE in the passthrough key tuple, sort its default.env entry alphabetically, and spell the nproc fact as ansible_facts['processor_nproc'].
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ac5e7d54c4 removed a duplicated literal by having the tuning handler take
BATCH_SIZE from the client that owns it. That import walks straight out of the
stdlib:
handlers/__init__ -> i18n/tuning -> client -> languages -> cache.yaml -> yaml
and the .env generator runs on the bare bootstrap python, before any dependency
is installed. The client needs SOURCE_LANGUAGE from the language table, so it
can never satisfy that contract itself.
The constant moves into a module with no imports of its own, which the handler
README names as the allowed shape: a handler may import any repo module whose
own closure is stdlib-only. Both the client and the handler read it from there,
so the literal still exists once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three modules reimplemented what utils.cache already offers, and each paid
for it with a suppression.
test_compose_override_services_defined carried a yaml.SafeLoader subclass
whose only job was to swallow compose's own tags, plus a noqa for the
yaml.load rule it then tripped. load_yaml_any handles !override and !reset
already, so the loader and the suppression both go.
test_compose_bind_sources_exist read default.env directly and argued in a
nocheck marker that once per session is fine. Reading it through
utils.cache.files makes the argument unnecessary.
test_gpu_gate loaded the role's vars and meta files with raw yaml over
hand-assembled paths; it now uses load_yaml with ROLE_FILE_VARS_MAIN and
ROLE_FILE_META_SERVICES, the way 50a13c755b did for the playwright
multipliers.
gpu_count drops a noqa ruff no longer raises, and the development env module
an import nothing uses, which lint-python rejects.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The translator sized its batches and lanes from constants and the CPU count,
which is wrong on any host whose bottleneck is the GPU rather than the cores.
`make i18n-tune` sweeps batch sizes 10 to 80 against lane factors 0.5 to 2.0
over 400 real msgids, keeps the pair with the highest entries per second and
writes it to build/i18n-tuning.json, beside the history
cli/contributing/update/collection_source already keeps there.
`make dotenv` turns that file into INFINITO_I18N_BATCH_SIZE and
INFINITO_I18N_LANES, and a translation run prefers them over its own
derivation. With no sweep on disk the handler emits the shipped defaults, so
a checkout that never tunes behaves as before.
The handler had spelled the batch default out a second time; it now takes
BATCH_SIZE from the client that owns it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d1f1fde4c1 added `i18n_tuning` to the registry while the module itself was
never staged, so a fresh checkout of this branch cannot import
utils.env.handlers at all:
ImportError: cannot import name 'i18n_tuning' from 'utils.env.handlers.infinito'
That takes `make dotenv` with it. Commit the module.
test_prefix_groups_belong_in_subfolders flagged the pair it landed beside:
cache_conf.py and cache_stack.py share a delimiter-bounded prefix, which the
repository treats as a folder in disguise. Both move into cache/, and
i18n_libretranslate.py joins the new module in i18n/ before it becomes the
same finding. The registry switches to the `from .infinito.ca import x as
ca_x` form 9b78933153 established; the names and the order in
ORDERED_HANDLERS are unchanged.
primary_checkout() ran git with `check=False`, which suppresses a non-zero
exit but not a missing cwd or a missing binary, so a caller whose repo_root
does not exist broke the whole handler chain rather than falling back. The
docstring already promised the fallback; now the code does it too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
argos-net.com hosts the .argosmodel files the LibreTranslate image pulls
at build time, 70 to 150 MB each and around 100 of them, so every rebuild
refetched several gigabytes from upstream. Route the host through the
existing Nexus raw proxy and the nginx MITM frontend that already covers
pypi, npm, alpine and the GitHub raw hosts.
The argos server block overrides the global proxy_read_timeout. Nexus
emits no byte of an object until it has fetched and stored the whole
thing, so the wait for the response header is the wait for the entire
cold fetch, which outran the global 300s once eight download lanes
competed for the upstream.
INFINITO_CACHE_PACKAGE_FRONTEND_CONF replaces the hardcoded relative bind
for the frontend's upstream map. One cache stack serves every checkout on
the host, so the map follows the primary checkout by default;
INFINITO_CACHE_CONF_SOURCE=worktree points it at the current one to test a
change before it merges. dotenv-force threads that variable through its
env -i allowlist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The i18n toolchain gets its own runner behind the `i18n` compose profile,
with its own inner docker daemon and docker state, so an i18n deploy shares
no ansible run and no container state with `infinito`. It reuses the image
that service builds (pull_policy: never), which starts it in under a second.
cache-consumer.yml holds the package-cache extra_hosts once and both runners
take them through `extends`; a YAML anchor cannot cross a compose file, an
extends can.
GPU reservation moves from a layered override file to INFINITO_GPU_COUNT on
both services. `gpus:` only accepts the literal `all`, so a variable there
invalidates the project on a CPU host, while a device reservation takes
`count: 0`. compose/gpu.override.yml and gpu_runtime_available() go with it.
test_compose_override_services_defined.py fails when an override extends a
service no compose file defines, which renders the whole project invalid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stack answered to *.infinito.test, and the host resolver pointed that
zone at 127.0.0.1, so a worktree's stack on its own bind IP could not be opened
in a browser while the primary stack existed.
INFINITO_DOMAIN is now derived per checkout: main.infinito.test for the
primary one, and for a linked worktree its branch, lowercased with everything
outside a-z0-9_- turned into "-", below INFINITO_DNS_DOMAIN
(feature/store-i18n -> feature-store-i18n.infinito.test). The branch is read
from the worktree's gitdir HEAD instead of running git, because git refuses the
mounted checkout inside the stack container. GitHub and act runs keep the
static default, custom.env still wins.
The handler overrides the caller's value on purpose: inside the stack
container that value is the previous .env, which make setup rewrote on every
deploy, so the old domain kept feeding itself back into the next compose up.
make network-dns-setup routes each stack's domain to the INFINITO_BIND_IP in
that checkout's .env, with a local= line so AAAA answers NODATA instead of the
primary stack's ::1. It skips domains outside the base zone, which a stale
.env still carries, and fails when two checkouts claim one domain.
worktree-up prints the domain and the follow-up command.
Existing checkouts switch after make dotenv-force and a redeploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
In CI the architecture axis is satisfied by the runner: the matrix picks a
row's architecture and `runs-on` lands the job on that hardware. A workstation
has one CPU and cannot be asked for another, so the same INFINITO_ARCHITECTURE
is honoured by telling the docker client which platform to pull and create, and
letting qemu execute it.
Two decisions here are easy to tidy away and should not be:
The key is deliberately not spelled DOCKER_DEFAULT_PLATFORM. The env layer is
sourced into the operator's own shell, where that name would pin every docker
call, including the ones that build the tooling image itself. compose.yml maps
it to docker's name inside the deploy container, so the platform applies where
app containers are created and nowhere else.
builder.py applies the handler a second time, after custom.env. That file
overrides handler output, so an operator naming the architecture there would
otherwise get the architecture without the platform it implies, and the deploy
would run on the host's own CPU under an arm64 label.
The key is emitted even when empty, because compose.yml reads it bare and this
repository forbids an inline default in a compose file. Empty means what unset
always meant: the host's own architecture.
test_walks_full_handler_chain counted calls against handlers as a proxy for
"every handler is reachable", which only holds while each runs exactly once.
The deliberate second application made that 42 != 41. It now compares the two
sets, which is what its own docstring claims to check.
Verified: 116 unit tests under tests/unit/python/utils/env pass; a deploy with
INFINITO_ARCHITECTURE=arm64 carries DOCKER_DEFAULT_PLATFORM=linux/arm64 into
the container and reaches ok=1444, with aarch64-linux-gnu paths in an image
build proving the containers are genuinely arm64.
Known gap: switching architecture needs `rm -f .env` first. Plain `make dotenv`
reads the previous INFINITO_DOCKER_PLATFORM back out of .env and the
contradiction guard treats it as an operator assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container this project builds installs its own python, ansible and docker
CLI, so the package manager riding along in the image it built FROM was never
used. That layer now lives in its own repository, which publishes one
multi-platform image per distribution, and meta/distros.yml points at it:
`base` replaces `pkgmgr_virgin`, and utils.distros grows base_image() where
pkgmgr_virgin_image() stood. The pkgmgr image itself stays, because the
`pkgmgr install` test still boots it.
Three references carried the pkgmgr-era tag `stable`, which base-images does
not publish: it pushes `latest` and the major release, and for a rolling
distribution only `latest`, so `latest` is the one value that exists for all
five. INFINITO_PARENT_IMAGE_TAG and the two install tests move to it.
The guard that keeps image references inside the SPOT only matched
`pkgmgr-[{$]`. Renaming the family made it blind to exactly half its surface,
which is how scripts/tests/install/make.sh kept `base-${d}:stable` while
pkgmgr.sh was caught. It now matches `base-` too.
INFINITO_ARCHITECTURE and INFINITO_ARCHITECTURES appear here because
default.env is shared with the architecture axis that follows. They are empty
and inert until it lands; the alternative was a commit whose build fails by
construction, with `base-` declared and `stable` still requested.
From here nothing builds until base-images has published: `docker pull` of the
parent answers `manifest unknown`, and every make target that depends on
build-dependency fails there.
Verified: tests/lint/repository/test_distros_spot.py passes, and the widened
pattern is shown to catch the line the old one missed rather than merely not
breaking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
chunk_size() multiplied the runner slots a chunk may assume by the waves the
queue window affords, and nothing could say otherwise. The derivation is the
right default -- too few rows leaves runners idle, too many push a chunk's tail
past the queue cancel -- but a run that wants smaller chunks had to edit
default.env to get them.
INFINITO_CI_CHUNK_SIZE now takes precedence when it is greater than zero, and
zero keeps the derivation. The input travels the same path offset already
travels: entry-manual-steer.yml offers the field and maps empty to 0,
call-orchestrator.yml hands it to all four chunk blocks, call-test-deploy.yml
exports it as the env var. It is a string rather than a number for the reason
resume_from_chunk already documents: a workflow_dispatch number input reaches
the caller as a string, and a number-typed workflow_call input then makes
GitHub refuse the job before it exists.
Two places in call-test-deploy.yml set it, not one. The plan summary resolves
the chunk size itself, so exporting it only for the app resolution would print
a plan that disagrees with the matrix the run then deploys.
Measured against the live helper: 0 gives 80 rows over 3 chunks, 25 gives 25
over 4, and returning to 0 restores 80 and 3.
The gate found three faults in the first attempt and all three are in this
commit: the key sorted after INFINITO_CI_CONCURRENCY rather than before it, the
dispatch description ran to 81 characters against a limit of 80, and the
retrigger fixture did not carry the new input. That last one matters beyond the
test: it is the check that would otherwise let a retrigger drop the chunk size
silently. carried_inputs() itself needed nothing, since it derives from the
declared inputs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.
File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.
restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.
All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.
type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.
The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.
The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.
For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.
An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
