Every desk- role becomes dsk-, and the three coding agents move to dsk-gnt- under a category of their own. The category carries a title, hanzi and icon but no invokable flag: an invokable child under an invokable parent is rejected outright, which is also why web itself is not invokable.
Three things a search and replace would have broken without a sound. get_entity_name now strips dsk-gnt-, so the primary entity keys agnt-claude, agnt-cursor and agnt-pi no longer resolved and all three roles dropped out of the CI plan. Eighteen run_once guards carried the old name against a convention its own test enforces. DESKTOP_ROLE_PREFIX in the addon spec lint is a literal, so the desktop exemption it grants would have matched nothing and the lint would have moved to the wrong rule in silence.
docs/requirements/035 is not a historical document but a register a lint holds to every role with an application_id, so its list carries the new ids. In 026 only the links move; the prose stays as a statement about the tree at the time it was written.
Measured against the plan with the rename normalised, the deployed role#variant set is unchanged. A stale id in an inventory is caught before any deploy starts: the pre-flight validator reports Invalid group 'desk-firefox' (not in application_ids).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
role_deploy_modes derived the offered modes from the container shape alone and read modes.<mode>.enabled only for what the shape had already decided. A role could therefore write compose: enabled: true and never be read. The declaration now wins; only a role that declares nothing falls back on its shape.
The complexity matrix hand-rolled the same logic and drifted: host and swarm were gated on the declaration, compose on nothing at all. Every stackless invokable role was handed compose, rotating per sweep, which is why the desktop tier showed up under a whale. Both call sites now read the one helper.
web-svc-legal is the first role to state its own modes. It ships no compose template of its own and reaches compose and swarm through the html server it pulls, which the shape heuristic cannot see. host stays declared as a bool because the host lint requires one on a stackless role.
Measured against the plan: the deployed role#variant set is unchanged, and three mode cells move.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The one-shot walk behind every project-tree test descended into build/, where the make targets keep their caches. Nothing under it is version controlled (.gitignore carries build/*, git ls-files reports none), so the 579 files it contributed were never project files in the sense any caller means.
Two lints already carried build/ in a skip list of their own, which is what a cause treated at each symptom looks like. The prune is anchored at the repository root because the set is matched per directory name and cli/build holds the documentation generators, which are source. A test pins both halves and was watched failing without the prune.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
xgettext counts a space as a format flag, so the prose `95% confidence` parses
as the directive `% c` and the entry is marked python-format. Every translation
moves the word behind the percent sign, msgfmt then reads `% V` and refuses the
catalog. Quoted Jinja blocks and percent-encoded URLs fall into the same trap.
A run over the documentation catalogs left 312 such entries in 19 files, none
of which could be compiled to .mo.
Removing the flag when writing does nothing: babel derives it from the msgid on
every load, so the value in the file is decoration. gettext's own switch for
this is no-python-format, which babel does not touch, and the writer now sets
it on entries the derivation flags but that carry no real placeholder. The
derived flag has to go at the same time, because gettext reads the flag line in
order and `no-python-format, python-format` cancels itself out.
What counts as a real placeholder was already spelled out in PROTECTED, so it
moves into PRINTF and is used from both places rather than written twice.
The repository lint gains the check that would have caught this. It compares
the msgid against itself first and reports only when the translation alone
fails, so a source string that mixes positional and named placeholders, which
gettext accepts, does not raise a finding.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.
Seven conflicts:
- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
in e706170a0 were modified upstream. The deletion stands, which is the whole
premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
and this branch's _write_mesh call after it.
The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
Three modules reimplemented what utils.cache already offers, and each paid
for it with a suppression.
test_compose_override_services_defined carried a yaml.SafeLoader subclass
whose only job was to swallow compose's own tags, plus a noqa for the
yaml.load rule it then tripped. load_yaml_any handles !override and !reset
already, so the loader and the suppression both go.
test_compose_bind_sources_exist read default.env directly and argued in a
nocheck marker that once per session is fine. Reading it through
utils.cache.files makes the argument unnecessary.
test_gpu_gate loaded the role's vars and meta files with raw yaml over
hand-assembled paths; it now uses load_yaml with ROLE_FILE_VARS_MAIN and
ROLE_FILE_META_SERVICES, the way 50a13c755b did for the playwright
multipliers.
gpu_count drops a noqa ruff no longer raises, and the development env module
an import nothing uses, which lint-python rejects.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/i18n/libretranslate.py had grown two jobs: bringing a LibreTranslate
up, and talking to one. The HTTP half moves to utils/i18n/client.py with the
batch size, the retry constants, the SERVER_CODES map that turns the
catalog's bare ISO 639-1 codes into the script-qualified ones the server
answers to, the Outcome dataclass and merge().
The deploy module keeps the name it is imported by and re-exports the three
client symbols, so every existing caller is untouched; the exported class is
the same object, not a wrapper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d1f1fde4c1 added `i18n_tuning` to the registry while the module itself was
never staged, so a fresh checkout of this branch cannot import
utils.env.handlers at all:
ImportError: cannot import name 'i18n_tuning' from 'utils.env.handlers.infinito'
That takes `make dotenv` with it. Commit the module.
test_prefix_groups_belong_in_subfolders flagged the pair it landed beside:
cache_conf.py and cache_stack.py share a delimiter-bounded prefix, which the
repository treats as a folder in disguise. Both move into cache/, and
i18n_libretranslate.py joins the new module in i18n/ before it becomes the
same finding. The registry switches to the `from .infinito.ca import x as
ca_x` form 9b78933153 established; the names and the order in
ORDERED_HANDLERS are unchanged.
primary_checkout() ran git with `check=False`, which suppresses a non-zero
exit but not a missing cwd or a missing binary, so a caller whose repo_root
does not exist broke the whole handler chain rather than falling back. The
docstring already promised the fallback; now the code does it too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The image pulled both directions of every Argos pair because upstream's
scripts/install_models.py matches a package when from_code AND to_code are
both wanted, so a direction cannot be expressed through
--load_only_lang_codes. prefetch_models.py now replaces that script instead
of warming its cache, and takes --directions: `both` keeps the old set,
`from_source` keeps only the pairs translating out of English, which halves
the download for a deployment that never translates back.
Replacing the script means carrying its guards. It refuses a language code
no package offers and refuses an empty selection, and it hands MiniSBD the
iso2model-converted codes rather than the raw ISO ones. Without those a typo
would have built an image with fewer models, or none, and reported success.
The build reaches the package cache over TLS, which needs the frontend CA in
the image trust store. Rather than reimplement the installer in the
Dockerfile, stage scripts/docker/cache/package-frontend/ca.sh into the build
context and run it: it already picks the trust path per distribution,
rebuilds the store and skips when no CA was staged. The trust-store path now
exists in exactly one place.
That step needs root. The base image sets USER libretranslate (uid 1032) and
RUN inherits it, so update-ca-certificates fails with
cannot create /etc/ssl/certs/ca-certificates.crt.new: Permission denied
The CA layers therefore run as root and hand the user back before the model
layers, so the models stay owned by the runtime user.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
argos-net.com hosts the .argosmodel files the LibreTranslate image pulls
at build time, 70 to 150 MB each and around 100 of them, so every rebuild
refetched several gigabytes from upstream. Route the host through the
existing Nexus raw proxy and the nginx MITM frontend that already covers
pypi, npm, alpine and the GitHub raw hosts.
The argos server block overrides the global proxy_read_timeout. Nexus
emits no byte of an object until it has fetched and stored the whole
thing, so the wait for the response header is the wait for the entire
cold fetch, which outran the global 300s once eight download lanes
competed for the upstream.
INFINITO_CACHE_PACKAGE_FRONTEND_CONF replaces the hardcoded relative bind
for the frontend's upstream map. One cache stack serves every checkout on
the host, so the map follows the primary checkout by default;
INFINITO_CACHE_CONF_SOURCE=worktree points it at the current one to test a
change before it merges. dotenv-force threads that variable through its
env -i allowlist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tasks/stages/03_destructor.yml includes test-e2e-cli only `when: RUNTIME in [dev, act, github]`, so a production host skips it by design. The coverage verifier demanded it anyway and failed every partial deploy there with "required role(s) did not execute". It stayed hidden because a full deploy passes no --id, which leaves allowed_applications empty and the guard unarmed.
required_by takes a `runtimes` list beside the existing compose/swarm blocks, read before the mode narrowing replaces rb. An unknown or empty runtime waives nothing, so the guard keeps its teeth wherever the runtime cannot be established; only a known runtime the list omits drops the requirement. This is 48753de136 one axis further: a role that a deploy correctly skips must not be demanded of it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `auto` alias picked a model that could not hold the prompt, and every
layer that should have caught it was comparing this deployment's own
declarations against each other.
Three errors produced one failure. smollm2:135m was declared at 16384 while
the tag trains to 8192, so Ollama clamped num_ctx and then truncated the
prompt to half the window rather than refusing it. The token estimate ran at
three characters per token and was documented as erring high, but the
oversized prompt is built from "x ", which tokenizes at two: 16384 characters
are 8193 tokens on all three served tokenizers against an estimate of 5461.
A window too large and a demand too small cancel into "eligible", and
neither correction works alone; both ship. The third error is what fired
them: at one sample the speed measurement kept its cold load, so a 135M
model measured slower than a 0.5B one and the router ranked on the
inversion. The warm-up is now discarded at every setting.
The probe compared a declared window against an estimated demand, both ours,
so it could not observe a declaration the backend does not honour. It now
reads usage.prompt_tokens and fails when the backend evaluated materially
less than was sent. Against the live stack that check separates the two
models exactly: smollm2 reports 4098 of 8192 sent and fails, llama3.2
reports 8221 and passes, and 4098 is the figure Ollama's own truncation
logged. A 400 now carries its body, without which this failure reads as
"HTTP Error 400" and nothing more.
The strategy is selectable. Where jeff is deployed the decision comes from a
System One encoder reading the prompt; otherwise the routes are ranked on
standing preferences, and each factor carries its own weight. Both were
exercised over six prompt sizes: system_one chose qwen, then smollm2, then
the mock as the prompt grew, weighted chose llama3.2:1b throughout. The
state handed to the decider is capped at 512 characters, because the encoder
holds 512 positions and the previous 20000 came from the server's own
rejection limit, which is a different number: measured through the gateway,
a 2000 character state already exceeded the decider's timeout, and one
timeout leaves the request running so every later one queues behind it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three tasks passed the gateway master key as `container exec -e
LITELLM_MK=<key>`. Ansible echoes that command line, and the masking was
written as `no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"`, which CI and
local deploys turn off, so every run printed the key in clear. That single
pattern is where four live master keys leaked from this branch.
The key is not masked better, it is no longer passed: the gateway container
already carries LITELLM_MASTER_KEY from its own env file, and `container
exec` inherits it, so ensure_keys.py, probe_auth.py and warm_model.py read
it from the environment they run in. Verified against the running container,
whose config env carries the variable.
The consumer keys have no such home, so they move to the shell module's
stdin: the payload for ensure_keys.py and the single key probe_auth.py
revokes and restores. That takes them out of the `cmd:` echo. It does not
take them out of `module_args`, which Ansible prints when a task fails, so
those two tasks keep their masking; a literal `no_log: true` was tried first
and rejected by the repository's own lint, which requires the masking to
stay switchable for diagnosis. The two tasks that now carry no secret at all
lose their `no_log` instead, which makes a failing speed measurement or warm
call readable again.
Exercised by unit tests only. The deploy that would run these tasks needs a
variant with consumer keys, and the litellm-only round has none, so
01_keys.yml was never entered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ca_injected asked three questions, the first being whether the application
has a domain. That decides whether it is SERVED over the self-signed CA. An
application that only CALLS one has no domain of its own, so the predicate
answered false, sys-svc-compose-ca skipped its stack, and the container ran
with an empty trust store.
svc-ai-agent-broker is such a caller: it authorizes every request against
Keycloak group paths over the public HTTPS base. Without the CA every
outbound call died in the TLS handshake, the broker answered 503 "access
check unavailable", Open WebUI turned that into HTTP 400, and the spec
failed on a model it had just listed. Its container was the only one in the
rescue artifact without SSL_CERT_FILE, while litellm and open-webui carried
it.
A service may now declare ca_client: true, and the predicate then answers on
the deployment's TLS mode alone, because the domain question is meaningless
for a client. The three served-side conditions are untouched, so no
application that already received the CA stops receiving it.
The alternative of pointing the broker at an internal Keycloak URL was
rejected: the role declares no keycloak service, so the URL would render
without a route.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_passthrough_swarm quoted and escaped any value containing whitespace,
which contradicts the contract stated at the top of the same module:
docker stack deploy does not strip outer quotes, so the value reaches the
container with them.
Ansible's to_json emits the default ", " separators, so AGENT_PLATFORMS
always carried spaces and arrived as "{\"hermes\": ...}". json.loads then
returned a str rather than a mapping, and sorted(PLATFORMS) in the broker
iterated its characters: the broker advertised one model per character and
never hermes or openclaw, which is why Open WebUI's model list held neither
and the agent-broker specs failed before reaching a completion. The same
encoding mangled AGENT_CONSTRAINT, whose placement expression would have
failed once an agent was reachable.
The branch existed for the opposite consumer: test.env is sourced by bash,
where KEY=one two leaves the variable unset under set -u. Quoting is the
wrong instrument for that, because one filter cannot serve both contracts
keyed on DEPLOYMENT_MODE alone. Both values that join on a space now join
on a comma and their consumers split on one, so no rendered env value needs
quoting to survive the shell.
Two unit tests asserted the quoting while a sibling in the same class
asserted that swarm does not escape a double quote; the pair could not both
hold. They are replaced by one test that renders real JSON and checks the
container receives a mapping. The lint that flags a space-joined value no
longer accepts dotenv_quote as the remedy, since in swarm it no longer
quotes, and its message names the comma instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four defects in the client surfaced in one run. LibreTranslate serves Chinese as zh-Hans and zh-Hant, never as bare zh, so the readiness wait blocked on a target that was never coming and the run stood still for an hour without printing a line. A blanket except turned every transient refusal into a permanently discarded entry, and the fallback path answered a failed batch by firing twenty single-text requests at the same overloaded server; the same catalogs that lost 130 to 295 entries came back with 0 discarded once the batch is retried with backoff first.
The per-catalog count conflated a refusal with a damaged protected span, which is measurably wrong: of 228 entries left pending in core/sw only 36 carried a span at all. Outcome now carries values, refusals, damage and the last refusal message per call rather than on the client, where twenty lanes mixed their numbers into each other. That split is what surfaced the 502 the server had been answering with. The throwaway container also ran with check=True and capture_output=True, so docker error text died inside the swallowed exception and the log showed an argv list and rc 125.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The check read df --output=pcent and judged every filesystem. The NVIDIA container toolkit mounts a 4 KiB tmpfs per CDI hook holding exactly one file, so it sits at 100% on every GPU host, no cleanup can ever change that, and the service failed the whole deploy with it.
A filesystem now counts only from 64 MiB up, which keeps /boot and every real partition in. Smaller ones over the threshold are still reported, as INFO rather than WARNING, so a genuinely tiny filesystem does not vanish from the log; below the threshold they stay quiet. The warning also names the mountpoint now, where it used to print bare percentages that said nothing about which filesystem had filled up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pull.py ran docker compose build and pull through subprocess.run without a timeout. The ansible task keyword eventually kills the task, but the python in between kept waiting, and the handler name says "compose pull" while pull.py also builds, so the phase the deploy sits in is not what the log claims. A first libretranslate build downloading every Argos model outlived the task timeout, and each retry restarted the RUN layer from zero.
run_cmd now passes timeout=RUN_TIMEOUT_SECONDS and turns TimeoutExpired into rc 124 plus whatever the command had already written, which every caller already treats as a failure. The matrix compose path carried no task timeout at all: its compose up has retries: 12, so a hang would never have triggered a single one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first deploy that measured generation rates gave mock/deterministic
4148 tok/s against 70.7 for the fastest real model. That number is the
proxy's echo latency, not a rate: a mock returns its declared string
without generating anything. Being local, free and 59 times faster, it
then won every routing decision, so the alias answered pong to any
prompt and the routing probe would have gone green while proving
nothing.
The exclude list is derived rather than maintained: it is the router's
own alias plus every alias whose provider is the mock provider, so a mock
declared in services.yml or in a variant is covered without a second
place to update.
LITELLM_EXCLUDE also becomes a set. It held a single alias and was
compared with `not in`, which on a string is a substring test, so a
comma-joined list would have silently unranked any model id contained in
it.
A mock stays routable. With nothing preloaded it is the only model the
gateway serves and has to win; it simply no longer wins on a number that
is not a model's. Against the rates this deploy measured, the alias moves
from mock/deterministic back to qwen2.5:0.5b.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
server() now resolves a translation endpoint in three steps: use the running
service, rebuild when the host has a GPU but no accelerated deploy left its
stamp, otherwise deploy. A plain run therefore costs nothing, and the rebuild
fires once, on the move from a CPU-built container to an accelerated one.
deploy() brings the runner up, waits for State.Running and deploys the role
inside it. The exec steps address `docker compose exec i18n` directly:
cli.administration.deploy.development exec hardcodes the `infinito` service
(compose.py:261), so every deploy through it silently ran in the main
container, where mailu already holds port 25 and postfix cannot start.
check=True stops a failing step instead of carrying on against the wrong
target.
The deploy router writes its pid to build/deploy.pid, which survives the exec
into the subscripts, and deploying() refuses a translation while that process
lives. The running deploy is left untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role declares `gpu: true`, which resource.yml.j2 turns into a nvidia
runtime, and appends `-cuda` to the image tag so the container carries the
CUDA libraries the runtime alone does not provide. Both hang off
LIBRETRANSLATE_GPU, read through resource_filter: lookup('config', ...)
returned false for the same declaration and shipped a CPU image twice.
assert_gpu.yml asks CTranslate2 for its CUDA device count after the compose
handlers flush and fails on zero. Without it the container stays healthy and
answers every request while falling back to the CPU, which is how a broken
CUDA stack survived several deploys unnoticed. It is gated on the host
carrying /dev/nvidia0 and on IS_STACK_HOST, since the exec helper exits 64 on
swarm workers.
load_only moves to the variants so CI stays slim while a production deploy
loads every language. An empty list now drops the flag instead of passing an
empty one, whose meaning is ambiguous.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The i18n toolchain gets its own runner behind the `i18n` compose profile,
with its own inner docker daemon and docker state, so an i18n deploy shares
no ansible run and no container state with `infinito`. It reuses the image
that service builds (pull_policy: never), which starts it in under a second.
cache-consumer.yml holds the package-cache extra_hosts once and both runners
take them through `extends`; a YAML anchor cannot cross a compose file, an
extends can.
GPU reservation moves from a layered override file to INFINITO_GPU_COUNT on
both services. `gpus:` only accepts the literal `all`, so a variable there
invalidates the project on a CPU host, while a device reservation takes
`count: 0`. compose/gpu.override.yml and gpu_runtime_available() go with it.
test_compose_override_services_defined.py fails when an override extends a
service no compose file defines, which renders the whole project invalid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
resource.yml.j2 renders `runtime: nvidia` for any service that declares
`gpu: true`, gated a second time on the host carrying /dev/nvidia0 so a
CPU-only host never gets a runtime its daemon cannot resolve. The device
fact is probed in daemon_config.yml next to the kata and gVisor shims, and
daemon.json.j2 lists nvidia alongside them. nvidia stays out of
SANDBOX_RUNTIME: it passes a device through, it does not isolate.
nvidia.yml drops rival apt sources for libnvidia-container before writing
its own. The other order leaves two sources with different Signed-By keys
for one URL, which makes apt refuse the entire source list.
test_gpu_runtime.py covers both polarities and the missing device fact; the
latter caught `x.stat.exists | default(false)`, which raises on the
undefined object before the filter runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router already ranked routes on traits.speed, but nothing ever set
one. No catalogue carries a rate, because a rate is a property of this
machine under this load rather than of the model, and a hand-written
number in services.yml would be a guess that rots.
The deploy now measures it. After the gateway is warm, measure_speed.py
asks it which models it serves and sends router_speed_samples requests to
each model that has no rate yet, discarding the first as a cold load and
taking the median of the rest. The result is stored on the stack host and
rendered back into model_info.traits.speed on the same deploy, so the
gateway restarts once with the numbers it just produced. The stored file
is read back with slurp rather than lookup('file'), because the config
directory lives on the stack host and a lookup reads the controller.
A model that already carries a rate sends no request at all. Re-measuring
everything every deploy would produce results the tolerance discards anyway,
and that traffic is the entire live surface of this step: what can hang,
what can fail the deploy, and what the deploy waits for.
router_speed_remeasure discards the stored numbers once, for the case
where the hardware under them changed.
Two budgets bound a run. A measured request may take
router_speed_timeout; the discarded first request gets upstream_timeout
instead, because it pays the model load and holding it to the sample
budget would time out exactly the models worth measuring. Exceeding a
budget abandons that model, not the deploy, and the model keeps whatever
rate it had.
Every matrix variant overrides the sample count to 1, where the single
sample is kept rather than discarded: a CI round proves that the
measurement, the store and the re-render ran, not what the rate was on a
shared runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router ranked surviving routes lexicographically: local, then price,
then speed, then alias. How much a factor counted was its position in
that tuple, so a deployment could not say that it cares about price more
than about speed.
Each factor now carries a 0..1 weight in services.litellm.router_weights.
A factor is scaled to 0..1 inside the candidate set before its weight
applies, because the raw units do not compare: a per-token price is a
millionth of a cent and a rate is tens of tokens per second. A factor
whose value is identical across the set scores every route 1 and drops
out, rather than deciding on a rounding artefact.
Confidentiality stays out of the score. local_reason removes every remote
candidate before decide() runs, so no weighting can send a request that
must stay local to a third party; a weight is a preference, and a
credential leaving the cluster is not something a preference should be
able to buy. The shipped weights additionally keep locality worth at
least cost and speed combined, and a test reads them out of
meta/services.yml rather than repeating them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stage looped over every role of its group and skipped the ones this host
does not deploy, one task at a time, plus an unconditional handler flush behind
each. Measured on a one-app deploy on 2026-09-23: 132 of 139 entries skipped,
278 task deltas, 41.8 seconds of nothing.
The lookup now takes the `groups` magic variable and drops the entries no host
in the inventory carries, so they never reach the loop.
Which roles get deployed does not change, and that is provable rather than
hoped for: the body deploys a role exactly when application_allowed holds,
which requires the app to be in that host's group_names; if it is, that host is
in groups[app], so groups[app] is non-empty and the lookup keeps the entry. The
filter is therefore strictly weaker than the condition it precedes and can drop
nothing the body would have run.
The filter deliberately reads `groups` and not `group_names`. 8aea5a9b35
measured that a per-host condition above the loop makes ansible insert the
included blocks in host order instead of loop order, after which
svc-storage-nfs-client mounted an export svc-storage-nfs-server had not
deployed yet and the play died on 'mount.nfs4: Connection refused'
(run 31645153977). `groups` is the same mapping on every host, so every host
still walks one identical list, and tasks/utils/setup/role.yml keeps its `when`
exactly where that commit put it. Two of the new cases pin this: a role only
one host carries still enters the loop, and a group_names passed to the lookup
changes nothing.
First written the other way, with group_names in the lookup and the `when`
removed. A single-host deploy cannot show that defect, and it came back green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
031 made the gateway the one place backends are wired, but not the place the
choice is made: every consumer still names one alias and LITELLM_CHAT_MODEL
resolves it once per deploy for all of them. That is wrong in both directions.
A prompt needing a large window reaches a model that cannot hold it, which is
how web-app-hermes answers HTTP 500 when the agent model declares fewer than
64000 tokens, and a one-line question reaches whatever the deploy picked.
This adds the System One half: fast, typed, no generation, deciding which
System II model answers. Requirement 038 records the decisions; the shape
follows prismhq/jev-router with a deterministic decision in place of a model,
and decision 7 records why no learned one is in scope.
A pre-call hook, not a semantic router. LiteLLM's own auto_router matches
embeddings against example utterances, which answers "what is this about"
rather than "which model can serve this", and it pulls in the semantic-router
extra plus an embedding backend the gateway does not otherwise need.
The hook ships as a rendered file beside config.yaml rather than a derived
image: proxy_server.py resolves a callback through get_instance_fn(value,
config_file_path), which spec_from_file_location's it out of the config's own
directory. services.litellm.image stays on its pin. It is a template because
ROUTER_ALIAS has to match what the route builder publishes.
Capabilities come from LiteLLM's own catalog, 1595 models deep in the image,
keyed on the route's litellm_params.model and then on that name without its
first segment, because ollama/llama3 is filed prefixed while openai/gpt-4o-mini
is filed bare. A traits mapping overrides it for what the catalog does not
know. Declaring what the catalog already states is forbidden: it duplicates a
fact this project does not own and rots in the dangerous direction, since a
stale supports_vision keeps a model eligible for requests it can no longer
serve. Cost is not declared either; a route with an api_base or a mock_response
is served by this deployment and is free, and a keyed route the catalog does
not price sorts behind every route it does.
It fails loud, in both halves. The hook returns a string naming the demand when
no served model can take the request, and proxy/utils.py turns that into a
RejectedRequestError carrying the text. The alias's own model_list entry exists
only so the name is listable and raises when reached. Pointing it at the first
route instead, as this first did, made a router that never ran look like one
that worked.
services.litellm.router_alias is the single switch. Empty publishes no route and
renders no callbacks entry, so the hook is not even a condition of starting the
proxy; a separate enable flag would let services.yml read as on while the
router is off.
The routing probe had to learn the alias, which is backed by no backend and was
read as a route standing without one. It now also sends a prompt the smallest
declared window cannot hold and fails when the answer names the alias itself or
a model whose window is too small. That lookup tries the prefixed and the bare
name, because litellm sets model_response.model to the resolved model: matching
only the literal name would find no window and pass every answer.
svc-ai-ollama variant 1 serves three models so the routes differ in something a
request can demand. smollm2:135m is deliberately not the smallest window:
8dbdd31b22 measured it running to n_gen=22011 with context shifts at
n_left=4091, and a route's max_tokens is a default a consumer can override
rather than a cap.
Open, and only a deploy closes them: that the proxy accepts the raising route
at startup, that the alias routes across three windows, and swarm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The harness derived LITELLM_SERVED_PROVIDERS from the keys argument, i.e. the
providers that hold a key. vars/main.yml:16-18 splits that in two: the keyed
list feeds env.j2, which indexes LITELLM_PROVIDER_KEYS and fails the render on
a keyless entry, while the served list is the keyed one plus AI_MOCK_PROVIDER
and decides what config.yaml.j2:4 publishes. The harness carried the name of
the second and the meaning of the first, so it reproduced the very defect it
was meant to catch: a mock filtered out of model_list, an empty list, and the
gateway answering 400 for the chat model while every name-level variable still
resolves to it. That cost a deploy and every warm task in run 35875571687.
TestMockModels stayed green through it by passing keys={"mock": "none-needed"},
a key the mock provider does not have and never needs, which walked around the
broken assumption instead of asserting against it.
The harness now derives the list the way vars/main.yml does, the fake key is
gone, and a new case states the failure directly: a mock must publish a route
although no provider holds a key. Negative control: reverting
LITELLM_SERVED_PROVIDERS to LITELLM_KEYED_PROVIDERS fails
test_the_mock_is_served_without_any_key with "'mock' not found in []".
meta/services.yml declares no mock in remote_models, so the default-argument
cases keep their old outcome; TestNoBackend still renders an empty model_list.
Residual: template and vars are pinned by two separate tests against one
hand-copied derivation rather than a shared source. A drift in vars/main.yml is
caught only by test_keyed_providers.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bb78ab3fb8 moved the handler loader in front of the settings render so the
play stopped dying on "handler 'compose-up' was not found". Registering them
was necessary and not sufficient: the notification now survives, and the next
task kills the play a different way.
render_replicated_templates.yml hangs notify: compose-up on the render
(_render_one_template.yml:23, default from role_templates_expanded.py:17).
00_core.yml then enters utils/network/routine.yml, whose step 1 pulls in
sys-svc-container, and that role ends on an unconditional meta: flush_handlers
at 00_core.yml:52. Job 107238119066 of run 35875571687 shows the whole chain:
NOTIFIED HANDLER ... Validate Docker Compose configuration at log line 13787,
META: triggered running handlers at 13801, and the handler at 13923 with
rc 127, "compose: command not found". /usr/bin/compose is installed by
sys-svc-compose/tasks/00_core.yml:30, which is step 3 of that same routine and
is never reached, because a failed handler aborts the play (ok=228 failed=1).
A guarded bootstrap ahead of the render absorbs that flush against an empty
queue and installs the wrapper first, the shape 252b3e58b2 applies in
sys-svc-rdbms and 7f1025e4a7 in svc-db-qdrant. Routine step 1 then falls
through on run_once_sys_svc_container, and step 3 renders compose.yml and
flushes under searxng's own application_id. The when: guard is load-bearing:
without it 01_reset.yml tears down DIR_COMPOSITIONS again under MODE_RESET.
litellm carries the same shape and survives only because it loads
sys-svc-rdbms first, which bootstraps compose as a side effect; searxng is a
leaf with no database and is therefore the first compose consumer in its own
CI row.
The existing bootstrap case in test_handler_registration_order.py finds the
first notify through task.get("notify"), a literal key in the same file, so it
cannot see a role that notifies two include levels down. The new case anchors
on the render include itself and compares indices instead. The sweeping case
checks topic ownership, not order, which is why searxng was green here while
it failed in CI.
Same class, pre-existing, not fixed here: svc-ai-agent-broker renders at :39
and enters the routine at :61, latent because it depends on litellm and is
never the first compose consumer. 252b3e58b2 names web-app-magento,
web-app-seaweedfs and svc-runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every rule so far named a construct to protect, and every run found one that
had been forgotten: paths, underscores, bracketed URLs, then the bracket
itself. 7338 sources still reached the translator carrying markup, and one in
eight came back broken.
Mask the delimiters themselves, so nothing a translator would reformat is left
in the text: emphasis, backticks, brackets, braces, parentheses, double quotes,
quoted identifiers and digits. Single quotes stay exposed on purpose, because
1419 of the 1494 sources that carry one are English contractions, and masking
those would hand over fragments instead of words.
Costs nothing: the 2606 entries that end up without translatable text were
already empty under the old rules, and the newly masked ones are galaxy_tags
lists and JSON fragments.
Two defects survive masking because they alter no protected span, so unmask
repairs them against the source: a translator reading a trailing span as
sentence-final swallows the boundary behind it, gluing the next sentence to a
code span, and it spaces out emphasis, which markdown then renders literally.
The standing checks assert the property rather than enumerate constructs, and
cover core and docs, where before only the span-parity check did, which is why
none of this was ever caught.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy warmed the chat model and got HTTP 400. The rendered config in the running container read 'model_list: []' while every name-level variable resolved to mock/deterministic: the alias list had been taught about the mock, the list that decides what is actually published had not.
One list could not answer both questions. config.yaml.j2 and test.env.j2 ask which models are served and want the mock; env.j2 asks which providers hold a key and emits <PROVIDER>_API_KEY by indexing LITELLM_PROVIDER_KEYS, so a keyless entry there kills the render. LITELLM_KEYED_PROVIDERS keeps its original meaning and LITELLM_SERVED_PROVIDERS is that set plus the mock.
The new test renders both from the role's own vars and pins the distinction from either side: the mock is served without a key, and the mock is never keyed - the second assertion carries the reason, because that is the one whose violation only shows up at deploy time.
The first attempt at this fix put the mock into the keyed list and would have broken the env render. It was caught by enumerating the variable's consumers before naming a confidence number, not by a test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy refused at ok=771 failed=1 with 'services.litellm.remote_models names provider(s) mock; known providers are openai, anthropic, openrouter'. Declaring the mock among the remote models, which is what makes it an ordinary model rather than a config key of its own, runs into a guard that demands a key slot for every provider. A mock needs none: it answers from its own declared response.
The guard keeps its teeth for real providers and now names the exception and its reason, instead of tolerating it silently.
AI_MOCK_PROVIDER holds the name once. The guard, AI_REMOTE_ALIASES and AI_MOCK_ALIASES read it, and the template hands it to the route builder, which lost its own constant. Two literals spelling the same thing is how the LM Studio branch came to drop the context window its README still advertised; a parameter cannot drift.
Static checking never touched this line. It took a real deploy, 772 steps in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every AI consumer asks for LITELLM_CHAT_MODEL, so a CI round paid CPU inference for tests that only check wiring. This follows the shape svc-net-tor already uses: variant 0 runs the whole logic against chutney, variant 1 against the public network.
A mock is an ordinary remote_models entry with provider mock rather than a config key of its own, because that list already means 'an alias and a way to call it' and a mock needs no key. AI_AGENT_MODEL prefers it so the agent broker stays deterministic in every round, including the ones where the consumers get a real model.
The preload switch lives in svc-ai-ollama's variants, not in the gateway's. The gateway's services.ollama is a bare consumer flag carrying bond, enabled and shared; a preload_models key there would have been silently inert and the models would have downloaded anyway.
Variant 0 keeps every backend enabled and empties its preloads instead of disabling it, because variant 0 must stay the maximal closure - and because tor does not disable its service either, it changes the flavor.
Production is untouched: with no mock declared the chain resolves exactly as before, which is asserted rather than assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The proxy config grew a loop per model source, so the rule for how an alias is called lived in four places - and they had drifted: the LM Studio branch published no model_info.max_input_tokens although the role's README says a model entry carrying context publishes exactly that. Serving anything new meant a fifth branch.
The sources now meet in one filter that returns {alias, params, context}, and the template renders a single loop. Every value goes through to_json, so a response carrying a colon, a quote or a newline cannot break the YAML it is written into.
Publication order is a contract, not an accident of how the loops were stacked: LITELLM_CHAT_MODEL takes the first alias, so a locally served model has to outrank a declared one. That is now stated in the module and asserted, instead of being implied.
The gateway lint had to widen with it. Its canary checks that some gateway template still matches the provider-URL pattern, because without one the rule would pass vacuously on every consumer; the pattern demanded {{ … }} while the template now sets the URL in {% set … %}. Widening it to {{ or {% makes the rule stricter, since a provider URL in a set block is just as direct an address and would have slipped through anywhere else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The trigger could only derive a selection: --failed from a source run's verdicts, --apps as a whitelist, and every axis carried over from that run. A verification run has no source run whose axes it wants to inherit; it wants the combination that answers an open question. --priority states the tokens, and five overrides state the axes.
--priority composes with --apps but collides with --failed, which builds its own line, and it skips the resume-offset computation: that only means something when a source run says where the regular line stopped.
Every dispatch is now validated first. call-orchestrator.yml already runs the same check as its first job and every other job needs it, so a bad token never reaches a chunk. What it reaches is the branch's concurrency group, which entry-manual-steer.yml declares with cancel-in-progress: a doomed dispatch cancels the run currently in flight and then fails its own validation. That cancellation is what this refuses, not the deploy minutes.
The whitelist is passed as empty when it is the all-sentinel, which is not a role name and would be reported as an unknown one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cheapest combination that still answers a question was picked by assertion, because nothing recorded the cost. Deploy job titles already carry the whole row and the API gives every job a start and an end, so the cost per axis value is read back off the runs that already happened.
Measured over six runs of this branch: compose 47m against swarm 83m, clearnet 52m against tor 79m. Choosing the axis is therefore worth minutes per row, not seconds.
Medians, not means, so one queue wait cannot outvote a hundred normal jobs, and a value below the sample minimum is dropped rather than ranked. fastest() answers None there, which means the caller leaves that axis to the rotation instead of guessing it.
The job labels are parsed with the same primitives the retrigger uses, so a title format change breaks both together rather than leaving this one silently measuring nothing. A filesystem row states the kind the matrix assigned, not the kind the deploy ran on, because a fallback is not recorded in the title.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The target span started at the closing bracket, so masking [RSA 4096-bit](url)
left an unbalanced [RSA 4096-bit in the translatable text. The translator
closed it, and restoring the span produced [RSA 4096-bit]](url). Parity never
noticed, because the extra bracket sits outside every protected span, and
1248 German entries carried it.
A lookbehind now keeps the bracket in the text and masks the target alone.
The existing damage is repaired in place rather than discarded: the sources
hold no ]]( at all, so every occurrence in a translation was spurious, and
the German around it was sound.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The URL pattern stopped at the first closing parenthesis, so a Wikipedia
link such as .../Social_engineering_(security) was only masked up to the
bracket. The translator then glued the remainder to the following word and
produced .../Social_engineering_(security)auf, which the external URL check
caught as HTTP 404. Markdown targets clipped the same way, and such targets
exist in several role READMEs.
Both patterns now accept a balanced parenthesis group, so the whole link
becomes one protected span. Prose in parentheses still masks to nothing.
Pruning both domains against the stricter protection emptied nine German
entries, four in core and five in docs, including the two glued links. They
refill on the next translation run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
LibreTranslate rewrote anything that read like prose: assets/img/ became
Vermoegenswerte/img/ and ](path) collapsed to )path). Mask multi-segment
repository paths, bare filenames and markdown link targets alongside the
spans that were already protected, so the translator never sees them.
The catalogs still carry the earlier damage, so add a prune subcommand. It
compares the protected spans of source and translation and empties every
entry whose translation altered them, which lets the next run refill it. A
pass over the German docs catalog cleared 85 entries without losing a single
intact translation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ephemeral LibreTranslate container starts against the -cuda image with
--gpus all wherever docker registers an nvidia runtime, and keeps the CPU
image everywhere else, so runners without the container toolkit behave as
before. Measured on an RTX 3080 Ti: 44% utilisation, 1744 MiB resident,
482 sentences per 30 seconds.
Only the tool derives the suffix. web-svc-libretranslate stays pinned to the
CPU image, so no deploy host is required to install the toolkit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The header test writes a .po file into its tempdir and reads it back. The
shared cache is keyed by path, so a cached read answers with the content
from before the write. The marker sits on the call's own line because the
lint accepts it nowhere else, and the shorter comment survives the
formatter that had detached it once already.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
reap() read the idle time before it took the agent's lock, and ensure()
set _last_used only once the agent answered its health endpoint. Two
requests lost their agent to that order. A turn longer than idle_minutes
counted from the moment the agent became ready, so the sweep stopped it
mid-request; a start slower than idle_minutes let the sweep queue on the
lock and stop the agent the moment ensure() released it. A turn on a
CPU-served model costs minutes per model call, so neither case is
theoretical.
The broker now counts in-flight chats per agent. server._chat brackets
the forwarded request in begin()/end(), and end() also stamps
_last_used, so idle time counts from the end of a request rather than
its start. The sweep skips a busy agent, and after taking the lock it
re-reads the agent and its idle time before stopping anything, which is
what closes the start race.
The relay deliberately does not stamp anything: an agent's own
background traffic would otherwise hold it open forever.
Unit tests cover the sweep: an idle agent is stopped, idle_stop false
stops nothing, a busy agent survives its idle time, and a prompt landing
during the sweep keeps its agent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. Twelve pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ollama served qwen2.5:0.5b with its own default window of 4096 tokens and
truncated what did not fit: "truncating input prompt limit=2050
prompt=17997". The agents meanwhile assumed windows of their own, Hermes
131072 and OpenClaw 200000, so the model answered from the tail of a cut
prompt, which is where the tool-definition JSON came from. Hermes also
probed for the window it did not have, and LiteLLM answers one of those
probes, GET /v1/models/{id}, with 401 for a virtual key.
A model entry now declares its window: `context` on an AI_LOCAL_MODELS
entry or on a services.litellm.remote_models entry. The gateway sends it
to Ollama as num_ctx, so the runner serves that window instead of its
default, and publishes it as model_info.max_input_tokens. The field is
optional, because neither LM Studio's JIT load nor openrouter/auto can be
held to a declared number.
The broker gains services.agent-broker.agents.model, defaulting to the
gateway's chat model, so agents can run on a larger model than Open WebUI
serves by default. It looks the window of that alias up in the inventory
and writes it into the agent config as Hermes model.context_length and
OpenClaw contextWindow; an unknown window leaves both keys out. A deploy
refuses a model that resolved to an empty string or to an unrendered
template, because every agent would ask the gateway for it.
Measured on the local stack: a request carrying num_ctx moved the runner
from 4096 to 8192, a 14771-token prompt at 32768 answered without
truncation in 168 s, and the runner grew from 484 MB to 936 MB. Both
agents resolve the declared window from their config.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
