sys-svc-compose writes compose.ca.override.yml for exactly lookup('ca_injected', application_id), which since 1254e27c2d also answers true for a service that declares ca_client and only calls the self-signed endpoint. compose_file_args rebuilt that predicate instead of asking it, and its copy still demanded a domain, so for a domainless ca_client the file was written and never reached --compose-file. Swarm takes its file list from that lookup; compose finds the file on disk by itself, which is why only the swarm jobs of run 36064589578 failed while the compose job passed.
The broker's container in that run carries no SSL_CERT_FILE and no CA mount, while openwebui, same job and same node, carries seven CA env vars and four binds. Every outbound call to Keycloak then fails the handshake, which the CLI test reports as 503 and Open WebUI relays as HTTP 400.
Two corrections in ca_injected itself. Its templar.template() call was dead: as_str strips the TrustedAsTemplate tag and ansible-core 2.19+ refuses to render an untrusted string, so the line could not resolve what it advertised, and tls.py and cert.py never template TLS_MODE either. And the ca_client branch short-circuited on the global TLS_MODE, skipping resolve_mode's per-app server.tls.mode override; it now applies only where there is no domain and so no per-app answer to get.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router settings become one mapping. Nine scalars sharing a router_ prefix were a dict spelled out in key names, and the sampling section about to arrive would have added five more. Carried along: fifteen lookups in vars, three variant overrides, four test files, and the two rejection messages that name the settings path, which would otherwise have pointed a reader at a key that no longer exists.
On a configurable trigger, seconds or requests or probability, the same request goes to several eligible routes at once. jeff is shown the replies rather than the routes that wrote them, with the aliases hidden behind option labels so the verdict rests on the text. In shadow mode the caller keeps the route already decided; in serve mode the winning reply is handed back through mock_response, so the fan-out is the answer path and no further call follows.
Each comparison writes one row per offered route to the proxy's own database, grouped by a request class of token band and capability demands. Grouping finer would spread the evidence too thin to decide, coarser would let a verdict about a vision request speak for a plain chat. A route that raises is left out rather than recorded as a loss: failing to answer is a fact about the moment.
The decider now also sees the price of this request, the measured rate, the quota and prompt caching, plus how each route fared before. A figure the catalog does not state is left out rather than defaulted, because a decider cannot tell an absent number from a zero one.
Sampling ships off. Every history and comparison path is caught wide and logged: neither may ever fail a route.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every desk- role becomes dsk-, and the three coding agents move to dsk-gnt- under a category of their own. The category carries a title, hanzi and icon but no invokable flag: an invokable child under an invokable parent is rejected outright, which is also why web itself is not invokable.
Three things a search and replace would have broken without a sound. get_entity_name now strips dsk-gnt-, so the primary entity keys agnt-claude, agnt-cursor and agnt-pi no longer resolved and all three roles dropped out of the CI plan. Eighteen run_once guards carried the old name against a convention its own test enforces. DESKTOP_ROLE_PREFIX in the addon spec lint is a literal, so the desktop exemption it grants would have matched nothing and the lint would have moved to the wrong rule in silence.
docs/requirements/035 is not a historical document but a register a lint holds to every role with an application_id, so its list carries the new ids. In 026 only the links move; the prose stays as a statement about the tree at the time it was written.
Measured against the plan with the rename normalised, the deployed role#variant set is unchanged. A stale id in an inventory is caught before any deploy starts: the pre-flight validator reports Invalid group 'desk-firefox' (not in application_ids).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
role_deploy_modes derived the offered modes from the container shape alone and read modes.<mode>.enabled only for what the shape had already decided. A role could therefore write compose: enabled: true and never be read. The declaration now wins; only a role that declares nothing falls back on its shape.
The complexity matrix hand-rolled the same logic and drifted: host and swarm were gated on the declaration, compose on nothing at all. Every stackless invokable role was handed compose, rotating per sweep, which is why the desktop tier showed up under a whale. Both call sites now read the one helper.
web-svc-legal is the first role to state its own modes. It ships no compose template of its own and reaches compose and swarm through the html server it pulls, which the shape heuristic cannot see. host stays declared as a bool because the host lint requires one on a stackless role.
Measured against the plan: the deployed role#variant set is unchanged, and three mode cells move.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The one-shot walk behind every project-tree test descended into build/, where the make targets keep their caches. Nothing under it is version controlled (.gitignore carries build/*, git ls-files reports none), so the 579 files it contributed were never project files in the sense any caller means.
Two lints already carried build/ in a skip list of their own, which is what a cause treated at each symptom looks like. The prune is anchored at the repository root because the set is matched per directory name and cli/build holds the documentation generators, which are source. A test pins both halves and was watched failing without the prune.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `auto` alias picked a model that could not hold the prompt, and every
layer that should have caught it was comparing this deployment's own
declarations against each other.
Three errors produced one failure. smollm2:135m was declared at 16384 while
the tag trains to 8192, so Ollama clamped num_ctx and then truncated the
prompt to half the window rather than refusing it. The token estimate ran at
three characters per token and was documented as erring high, but the
oversized prompt is built from "x ", which tokenizes at two: 16384 characters
are 8193 tokens on all three served tokenizers against an estimate of 5461.
A window too large and a demand too small cancel into "eligible", and
neither correction works alone; both ship. The third error is what fired
them: at one sample the speed measurement kept its cold load, so a 135M
model measured slower than a 0.5B one and the router ranked on the
inversion. The warm-up is now discarded at every setting.
The probe compared a declared window against an estimated demand, both ours,
so it could not observe a declaration the backend does not honour. It now
reads usage.prompt_tokens and fails when the backend evaluated materially
less than was sent. Against the live stack that check separates the two
models exactly: smollm2 reports 4098 of 8192 sent and fails, llama3.2
reports 8221 and passes, and 4098 is the figure Ollama's own truncation
logged. A 400 now carries its body, without which this failure reads as
"HTTP Error 400" and nothing more.
The strategy is selectable. Where jeff is deployed the decision comes from a
System One encoder reading the prompt; otherwise the routes are ranked on
standing preferences, and each factor carries its own weight. Both were
exercised over six prompt sizes: system_one chose qwen, then smollm2, then
the mock as the prompt grew, weighted chose llama3.2:1b throughout. The
state handed to the decider is capped at 512 characters, because the encoder
holds 512 positions and the previous 20000 came from the server's own
rejection limit, which is a different number: measured through the gateway,
a 2000 character state already exceeded the decider's timeout, and one
timeout leaves the request running so every later one queues behind it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three tasks passed the gateway master key as `container exec -e
LITELLM_MK=<key>`. Ansible echoes that command line, and the masking was
written as `no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"`, which CI and
local deploys turn off, so every run printed the key in clear. That single
pattern is where four live master keys leaked from this branch.
The key is not masked better, it is no longer passed: the gateway container
already carries LITELLM_MASTER_KEY from its own env file, and `container
exec` inherits it, so ensure_keys.py, probe_auth.py and warm_model.py read
it from the environment they run in. Verified against the running container,
whose config env carries the variable.
The consumer keys have no such home, so they move to the shell module's
stdin: the payload for ensure_keys.py and the single key probe_auth.py
revokes and restores. That takes them out of the `cmd:` echo. It does not
take them out of `module_args`, which Ansible prints when a task fails, so
those two tasks keep their masking; a literal `no_log: true` was tried first
and rejected by the repository's own lint, which requires the masking to
stay switchable for diagnosis. The two tasks that now carry no secret at all
lose their `no_log` instead, which makes a failing speed measurement or warm
call readable again.
Exercised by unit tests only. The deploy that would run these tasks needs a
variant with consumer keys, and the litellm-only round has none, so
01_keys.yml was never entered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ca_injected asked three questions, the first being whether the application
has a domain. That decides whether it is SERVED over the self-signed CA. An
application that only CALLS one has no domain of its own, so the predicate
answered false, sys-svc-compose-ca skipped its stack, and the container ran
with an empty trust store.
svc-ai-agent-broker is such a caller: it authorizes every request against
Keycloak group paths over the public HTTPS base. Without the CA every
outbound call died in the TLS handshake, the broker answered 503 "access
check unavailable", Open WebUI turned that into HTTP 400, and the spec
failed on a model it had just listed. Its container was the only one in the
rescue artifact without SSL_CERT_FILE, while litellm and open-webui carried
it.
A service may now declare ca_client: true, and the predicate then answers on
the deployment's TLS mode alone, because the domain question is meaningless
for a client. The three served-side conditions are untouched, so no
application that already received the CA stops receiving it.
The alternative of pointing the broker at an internal Keycloak URL was
rejected: the role declares no keycloak service, so the URL would render
without a route.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_passthrough_swarm quoted and escaped any value containing whitespace,
which contradicts the contract stated at the top of the same module:
docker stack deploy does not strip outer quotes, so the value reaches the
container with them.
Ansible's to_json emits the default ", " separators, so AGENT_PLATFORMS
always carried spaces and arrived as "{\"hermes\": ...}". json.loads then
returned a str rather than a mapping, and sorted(PLATFORMS) in the broker
iterated its characters: the broker advertised one model per character and
never hermes or openclaw, which is why Open WebUI's model list held neither
and the agent-broker specs failed before reaching a completion. The same
encoding mangled AGENT_CONSTRAINT, whose placement expression would have
failed once an agent was reachable.
The branch existed for the opposite consumer: test.env is sourced by bash,
where KEY=one two leaves the variable unset under set -u. Quoting is the
wrong instrument for that, because one filter cannot serve both contracts
keyed on DEPLOYMENT_MODE alone. Both values that join on a space now join
on a comma and their consumers split on one, so no rendered env value needs
quoting to survive the shell.
Two unit tests asserted the quoting while a sibling in the same class
asserted that swarm does not escape a double quote; the pair could not both
hold. They are replaced by one test that renders real JSON and checks the
container receives a mapping. The lint that flags a space-joined value no
longer accepts dotenv_quote as the remedy, since in swarm it no longer
quotes, and its message names the comma instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first deploy that measured generation rates gave mock/deterministic
4148 tok/s against 70.7 for the fastest real model. That number is the
proxy's echo latency, not a rate: a mock returns its declared string
without generating anything. Being local, free and 59 times faster, it
then won every routing decision, so the alias answered pong to any
prompt and the routing probe would have gone green while proving
nothing.
The exclude list is derived rather than maintained: it is the router's
own alias plus every alias whose provider is the mock provider, so a mock
declared in services.yml or in a variant is covered without a second
place to update.
LITELLM_EXCLUDE also becomes a set. It held a single alias and was
compared with `not in`, which on a string is a substring test, so a
comma-joined list would have silently unranked any model id contained in
it.
A mock stays routable. With nothing preloaded it is the only model the
gateway serves and has to win; it simply no longer wins on a number that
is not a model's. Against the rates this deploy measured, the alias moves
from mock/deterministic back to qwen2.5:0.5b.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router already ranked routes on traits.speed, but nothing ever set
one. No catalogue carries a rate, because a rate is a property of this
machine under this load rather than of the model, and a hand-written
number in services.yml would be a guess that rots.
The deploy now measures it. After the gateway is warm, measure_speed.py
asks it which models it serves and sends router_speed_samples requests to
each model that has no rate yet, discarding the first as a cold load and
taking the median of the rest. The result is stored on the stack host and
rendered back into model_info.traits.speed on the same deploy, so the
gateway restarts once with the numbers it just produced. The stored file
is read back with slurp rather than lookup('file'), because the config
directory lives on the stack host and a lookup reads the controller.
A model that already carries a rate sends no request at all. Re-measuring
everything every deploy would produce results the tolerance discards anyway,
and that traffic is the entire live surface of this step: what can hang,
what can fail the deploy, and what the deploy waits for.
router_speed_remeasure discards the stored numbers once, for the case
where the hardware under them changed.
Two budgets bound a run. A measured request may take
router_speed_timeout; the discarded first request gets upstream_timeout
instead, because it pays the model load and holding it to the sample
budget would time out exactly the models worth measuring. Exceeding a
budget abandons that model, not the deploy, and the model keeps whatever
rate it had.
Every matrix variant overrides the sample count to 1, where the single
sample is kept rather than discarded: a CI round proves that the
measurement, the store and the re-render ran, not what the rate was on a
shared runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The router ranked surviving routes lexicographically: local, then price,
then speed, then alias. How much a factor counted was its position in
that tuple, so a deployment could not say that it cares about price more
than about speed.
Each factor now carries a 0..1 weight in services.litellm.router_weights.
A factor is scaled to 0..1 inside the candidate set before its weight
applies, because the raw units do not compare: a per-token price is a
millionth of a cent and a rate is tens of tokens per second. A factor
whose value is identical across the set scores every route 1 and drops
out, rather than deciding on a rounding artefact.
Confidentiality stays out of the score. local_reason removes every remote
candidate before decide() runs, so no weighting can send a request that
must stay local to a third party; a weight is a preference, and a
credential leaving the cluster is not something a preference should be
able to buy. The shipped weights additionally keep locality worth at
least cost and speed combined, and a test reads them out of
meta/services.yml rather than repeating them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every stage looped over every role of its group and skipped the ones this host
does not deploy, one task at a time, plus an unconditional handler flush behind
each. Measured on a one-app deploy on 2026-09-23: 132 of 139 entries skipped,
278 task deltas, 41.8 seconds of nothing.
The lookup now takes the `groups` magic variable and drops the entries no host
in the inventory carries, so they never reach the loop.
Which roles get deployed does not change, and that is provable rather than
hoped for: the body deploys a role exactly when application_allowed holds,
which requires the app to be in that host's group_names; if it is, that host is
in groups[app], so groups[app] is non-empty and the lookup keeps the entry. The
filter is therefore strictly weaker than the condition it precedes and can drop
nothing the body would have run.
The filter deliberately reads `groups` and not `group_names`. 8aea5a9b35
measured that a per-host condition above the loop makes ansible insert the
included blocks in host order instead of loop order, after which
svc-storage-nfs-client mounted an export svc-storage-nfs-server had not
deployed yet and the play died on 'mount.nfs4: Connection refused'
(run 31645153977). `groups` is the same mapping on every host, so every host
still walks one identical list, and tasks/utils/setup/role.yml keeps its `when`
exactly where that commit put it. Two of the new cases pin this: a role only
one host carries still enters the loop, and a group_names passed to the lookup
changes nothing.
First written the other way, with group_names in the lookup and the `when`
removed. A single-host deploy cannot show that defect, and it came back green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
031 made the gateway the one place backends are wired, but not the place the
choice is made: every consumer still names one alias and LITELLM_CHAT_MODEL
resolves it once per deploy for all of them. That is wrong in both directions.
A prompt needing a large window reaches a model that cannot hold it, which is
how web-app-hermes answers HTTP 500 when the agent model declares fewer than
64000 tokens, and a one-line question reaches whatever the deploy picked.
This adds the System One half: fast, typed, no generation, deciding which
System II model answers. Requirement 038 records the decisions; the shape
follows prismhq/jev-router with a deterministic decision in place of a model,
and decision 7 records why no learned one is in scope.
A pre-call hook, not a semantic router. LiteLLM's own auto_router matches
embeddings against example utterances, which answers "what is this about"
rather than "which model can serve this", and it pulls in the semantic-router
extra plus an embedding backend the gateway does not otherwise need.
The hook ships as a rendered file beside config.yaml rather than a derived
image: proxy_server.py resolves a callback through get_instance_fn(value,
config_file_path), which spec_from_file_location's it out of the config's own
directory. services.litellm.image stays on its pin. It is a template because
ROUTER_ALIAS has to match what the route builder publishes.
Capabilities come from LiteLLM's own catalog, 1595 models deep in the image,
keyed on the route's litellm_params.model and then on that name without its
first segment, because ollama/llama3 is filed prefixed while openai/gpt-4o-mini
is filed bare. A traits mapping overrides it for what the catalog does not
know. Declaring what the catalog already states is forbidden: it duplicates a
fact this project does not own and rots in the dangerous direction, since a
stale supports_vision keeps a model eligible for requests it can no longer
serve. Cost is not declared either; a route with an api_base or a mock_response
is served by this deployment and is free, and a keyed route the catalog does
not price sorts behind every route it does.
It fails loud, in both halves. The hook returns a string naming the demand when
no served model can take the request, and proxy/utils.py turns that into a
RejectedRequestError carrying the text. The alias's own model_list entry exists
only so the name is listable and raises when reached. Pointing it at the first
route instead, as this first did, made a router that never ran look like one
that worked.
services.litellm.router_alias is the single switch. Empty publishes no route and
renders no callbacks entry, so the hook is not even a condition of starting the
proxy; a separate enable flag would let services.yml read as on while the
router is off.
The routing probe had to learn the alias, which is backed by no backend and was
read as a route standing without one. It now also sends a prompt the smallest
declared window cannot hold and fails when the answer names the alias itself or
a model whose window is too small. That lookup tries the prefixed and the bare
name, because litellm sets model_response.model to the resolved model: matching
only the literal name would find no window and pass every answer.
svc-ai-ollama variant 1 serves three models so the routes differ in something a
request can demand. smollm2:135m is deliberately not the smallest window:
8dbdd31b22 measured it running to n_gen=22011 with context shifts at
n_left=4091, and a route's max_tokens is a default a consumer can override
rather than a cap.
Open, and only a deploy closes them: that the proxy accepts the raising route
at startup, that the alias routes across three windows, and swarm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The harness derived LITELLM_SERVED_PROVIDERS from the keys argument, i.e. the
providers that hold a key. vars/main.yml:16-18 splits that in two: the keyed
list feeds env.j2, which indexes LITELLM_PROVIDER_KEYS and fails the render on
a keyless entry, while the served list is the keyed one plus AI_MOCK_PROVIDER
and decides what config.yaml.j2:4 publishes. The harness carried the name of
the second and the meaning of the first, so it reproduced the very defect it
was meant to catch: a mock filtered out of model_list, an empty list, and the
gateway answering 400 for the chat model while every name-level variable still
resolves to it. That cost a deploy and every warm task in run 35875571687.
TestMockModels stayed green through it by passing keys={"mock": "none-needed"},
a key the mock provider does not have and never needs, which walked around the
broken assumption instead of asserting against it.
The harness now derives the list the way vars/main.yml does, the fake key is
gone, and a new case states the failure directly: a mock must publish a route
although no provider holds a key. Negative control: reverting
LITELLM_SERVED_PROVIDERS to LITELLM_KEYED_PROVIDERS fails
test_the_mock_is_served_without_any_key with "'mock' not found in []".
meta/services.yml declares no mock in remote_models, so the default-argument
cases keep their old outcome; TestNoBackend still renders an empty model_list.
Residual: template and vars are pinned by two separate tests against one
hand-copied derivation rather than a shared source. A drift in vars/main.yml is
caught only by test_keyed_providers.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bb78ab3fb8 moved the handler loader in front of the settings render so the
play stopped dying on "handler 'compose-up' was not found". Registering them
was necessary and not sufficient: the notification now survives, and the next
task kills the play a different way.
render_replicated_templates.yml hangs notify: compose-up on the render
(_render_one_template.yml:23, default from role_templates_expanded.py:17).
00_core.yml then enters utils/network/routine.yml, whose step 1 pulls in
sys-svc-container, and that role ends on an unconditional meta: flush_handlers
at 00_core.yml:52. Job 107238119066 of run 35875571687 shows the whole chain:
NOTIFIED HANDLER ... Validate Docker Compose configuration at log line 13787,
META: triggered running handlers at 13801, and the handler at 13923 with
rc 127, "compose: command not found". /usr/bin/compose is installed by
sys-svc-compose/tasks/00_core.yml:30, which is step 3 of that same routine and
is never reached, because a failed handler aborts the play (ok=228 failed=1).
A guarded bootstrap ahead of the render absorbs that flush against an empty
queue and installs the wrapper first, the shape 252b3e58b2 applies in
sys-svc-rdbms and 7f1025e4a7 in svc-db-qdrant. Routine step 1 then falls
through on run_once_sys_svc_container, and step 3 renders compose.yml and
flushes under searxng's own application_id. The when: guard is load-bearing:
without it 01_reset.yml tears down DIR_COMPOSITIONS again under MODE_RESET.
litellm carries the same shape and survives only because it loads
sys-svc-rdbms first, which bootstraps compose as a side effect; searxng is a
leaf with no database and is therefore the first compose consumer in its own
CI row.
The existing bootstrap case in test_handler_registration_order.py finds the
first notify through task.get("notify"), a literal key in the same file, so it
cannot see a role that notifies two include levels down. The new case anchors
on the render include itself and compares indices instead. The sweeping case
checks topic ownership, not order, which is why searxng was green here while
it failed in CI.
Same class, pre-existing, not fixed here: svc-ai-agent-broker renders at :39
and enters the routine at :61, latent because it depends on litellm and is
never the first compose consumer. 252b3e58b2 names web-app-magento,
web-app-seaweedfs and svc-runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy warmed the chat model and got HTTP 400. The rendered config in the running container read 'model_list: []' while every name-level variable resolved to mock/deterministic: the alias list had been taught about the mock, the list that decides what is actually published had not.
One list could not answer both questions. config.yaml.j2 and test.env.j2 ask which models are served and want the mock; env.j2 asks which providers hold a key and emits <PROVIDER>_API_KEY by indexing LITELLM_PROVIDER_KEYS, so a keyless entry there kills the render. LITELLM_KEYED_PROVIDERS keeps its original meaning and LITELLM_SERVED_PROVIDERS is that set plus the mock.
The new test renders both from the role's own vars and pins the distinction from either side: the mock is served without a key, and the mock is never keyed - the second assertion carries the reason, because that is the one whose violation only shows up at deploy time.
The first attempt at this fix put the mock into the keyed list and would have broken the env render. It was caught by enumerating the variable's consumers before naming a confidence number, not by a test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy refused at ok=771 failed=1 with 'services.litellm.remote_models names provider(s) mock; known providers are openai, anthropic, openrouter'. Declaring the mock among the remote models, which is what makes it an ordinary model rather than a config key of its own, runs into a guard that demands a key slot for every provider. A mock needs none: it answers from its own declared response.
The guard keeps its teeth for real providers and now names the exception and its reason, instead of tolerating it silently.
AI_MOCK_PROVIDER holds the name once. The guard, AI_REMOTE_ALIASES and AI_MOCK_ALIASES read it, and the template hands it to the route builder, which lost its own constant. Two literals spelling the same thing is how the LM Studio branch came to drop the context window its README still advertised; a parameter cannot drift.
Static checking never touched this line. It took a real deploy, 772 steps in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every AI consumer asks for LITELLM_CHAT_MODEL, so a CI round paid CPU inference for tests that only check wiring. This follows the shape svc-net-tor already uses: variant 0 runs the whole logic against chutney, variant 1 against the public network.
A mock is an ordinary remote_models entry with provider mock rather than a config key of its own, because that list already means 'an alias and a way to call it' and a mock needs no key. AI_AGENT_MODEL prefers it so the agent broker stays deterministic in every round, including the ones where the consumers get a real model.
The preload switch lives in svc-ai-ollama's variants, not in the gateway's. The gateway's services.ollama is a bare consumer flag carrying bond, enabled and shared; a preload_models key there would have been silently inert and the models would have downloaded anyway.
Variant 0 keeps every backend enabled and empties its preloads instead of disabling it, because variant 0 must stay the maximal closure - and because tor does not disable its service either, it changes the flavor.
Production is untouched: with no mock declared the chain resolves exactly as before, which is asserted rather than assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The proxy config grew a loop per model source, so the rule for how an alias is called lived in four places - and they had drifted: the LM Studio branch published no model_info.max_input_tokens although the role's README says a model entry carrying context publishes exactly that. Serving anything new meant a fifth branch.
The sources now meet in one filter that returns {alias, params, context}, and the template renders a single loop. Every value goes through to_json, so a response carrying a colon, a quote or a newline cannot break the YAML it is written into.
Publication order is a contract, not an accident of how the loops were stacked: LITELLM_CHAT_MODEL takes the first alias, so a locally served model has to outrank a declared one. That is now stated in the module and asserted, instead of being implied.
The gateway lint had to widen with it. Its canary checks that some gateway template still matches the provider-URL pattern, because without one the rule would pass vacuously on every consumer; the pattern demanded {{ … }} while the template now sets the URL in {% set … %}. Widening it to {{ or {% makes the rule stricter, since a provider URL in a set block is just as direct an address and would have slipped through anywhere else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The trigger could only derive a selection: --failed from a source run's verdicts, --apps as a whitelist, and every axis carried over from that run. A verification run has no source run whose axes it wants to inherit; it wants the combination that answers an open question. --priority states the tokens, and five overrides state the axes.
--priority composes with --apps but collides with --failed, which builds its own line, and it skips the resume-offset computation: that only means something when a source run says where the regular line stopped.
Every dispatch is now validated first. call-orchestrator.yml already runs the same check as its first job and every other job needs it, so a bad token never reaches a chunk. What it reaches is the branch's concurrency group, which entry-manual-steer.yml declares with cancel-in-progress: a doomed dispatch cancels the run currently in flight and then fails its own validation. That cancellation is what this refuses, not the deploy minutes.
The whitelist is passed as empty when it is the all-sentinel, which is not a role name and would be reported as an unknown one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cheapest combination that still answers a question was picked by assertion, because nothing recorded the cost. Deploy job titles already carry the whole row and the API gives every job a start and an end, so the cost per axis value is read back off the runs that already happened.
Measured over six runs of this branch: compose 47m against swarm 83m, clearnet 52m against tor 79m. Choosing the axis is therefore worth minutes per row, not seconds.
Medians, not means, so one queue wait cannot outvote a hundred normal jobs, and a value below the sample minimum is dropped rather than ranked. fastest() answers None there, which means the caller leaves that axis to the rotation instead of guessing it.
The job labels are parsed with the same primitives the retrigger uses, so a title format change breaks both together rather than leaving this one silently measuring nothing. A filesystem row states the kind the matrix assigned, not the kind the deploy ran on, because a fallback is not recorded in the title.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
reap() read the idle time before it took the agent's lock, and ensure()
set _last_used only once the agent answered its health endpoint. Two
requests lost their agent to that order. A turn longer than idle_minutes
counted from the moment the agent became ready, so the sweep stopped it
mid-request; a start slower than idle_minutes let the sweep queue on the
lock and stop the agent the moment ensure() released it. A turn on a
CPU-served model costs minutes per model call, so neither case is
theoretical.
The broker now counts in-flight chats per agent. server._chat brackets
the forwarded request in begin()/end(), and end() also stamps
_last_used, so idle time counts from the end of a request rather than
its start. The sweep skips a busy agent, and after taking the lock it
re-reads the agent and its idle time before stopping anything, which is
what closes the start race.
The relay deliberately does not stamp anything: an agent's own
background traffic would otherwise hold it open forever.
Unit tests cover the sweep: an idle agent is stopped, idle_stop false
stops nothing, a busy agent survives its idle time, and a prompt landing
during the sweep keeps its agent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ollama served qwen2.5:0.5b with its own default window of 4096 tokens and
truncated what did not fit: "truncating input prompt limit=2050
prompt=17997". The agents meanwhile assumed windows of their own, Hermes
131072 and OpenClaw 200000, so the model answered from the tail of a cut
prompt, which is where the tool-definition JSON came from. Hermes also
probed for the window it did not have, and LiteLLM answers one of those
probes, GET /v1/models/{id}, with 401 for a virtual key.
A model entry now declares its window: `context` on an AI_LOCAL_MODELS
entry or on a services.litellm.remote_models entry. The gateway sends it
to Ollama as num_ctx, so the runner serves that window instead of its
default, and publishes it as model_info.max_input_tokens. The field is
optional, because neither LM Studio's JIT load nor openrouter/auto can be
held to a declared number.
The broker gains services.agent-broker.agents.model, defaulting to the
gateway's chat model, so agents can run on a larger model than Open WebUI
serves by default. It looks the window of that alias up in the inventory
and writes it into the agent config as Hermes model.context_length and
OpenClaw contextWindow; an unknown window leaves both keys out. A deploy
refuses a model that resolved to an empty string or to an unrendered
template, because every agent would ask the gateway for it.
Measured on the local stack: a request carrying num_ctx moved the runner
from 4096 to 8192, a 14771-token prompt at 32768 answered without
truncation in 168 s, and the runner grew from 484 MB to 936 MB. Both
agents resolve the declared window from their config.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With svc-ai-agent-broker in the inventory, Open WebUI adds the broker as a
second OpenAI connection (OPENAI_API_BASE_URLS/KEYS). It forwards the
user's id and email headers and waits up to the broker's request_timeout,
so a cold agent start can finish.
BYPASS_MODEL_ACCESS_CONTROL turns false while the broker is enabled.
agent_model_grants.py then runs inside the container and:
- grants every LiteLLM model read access for all users;
- grants hermes and openclaw read access for their
/roles/web-app-<platform>/agent-user group only;
- reads grants back from the database, because the API view is filtered by
the admin access control.
The grant rows also give every gateway model the database entry whose
absence made check_model_access answer "Model not found".
Open WebUI shows a model without a grant to administrators only. So a
svc-ai-litellm deploy now reruns the grant step in a running Open WebUI,
and a model added to the gateway is visible without an Open WebUI deploy.
The include sets application_id explicitly, because the app loader's
include var would otherwise resolve Open WebUI's lookups against LiteLLM.
Without the broker the rendered connection, key and bypass are unchanged;
only the variable name moves to the plural form Open WebUI falls back from.
The Playwright spec:
- revokes biber in Keycloak;
- removes the stale Open WebUI membership, which the OIDC sync keeps when
the groups claim turns empty;
- asserts every served gateway model is listed for biber and no agent
model is;
- grants the groups and expects both agents to answer through the broker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add svc-ai-agent-broker, an OpenAI-compatible endpoint for Open WebUI that
serves the models hermes and openclaw. On a user's first prompt it checks
the Keycloak group /roles/web-app-<platform>/agent-user, creates that
user's own agent under the sandbox runtime, waits for its health endpoint
and streams the answer back. A caller without the group gets 403 and any
running agent of theirs is stopped.
Each agent gets its own container (compose) or single-replica service
(swarm), volume, network shared only with the broker, and bearer key. Its
image, runtime, limits and mounts are fixed by the broker. An agent that
comes up under runc is stopped and refused.
The broker reaches the engine only through wollomatic/socket-proxy. The
proxy has no network, serves a unix socket in a shared volume, allowlists
the calls the broker makes and rejects host bind mounts.
Agents hold no model credential. They call the broker's /llm/v1 relay,
which forwards to LiteLLM with the broker's virtual key, sets the OpenAI
user field to the owner, and logs a relay event with owner, platform, path
and upstream status.
agents.idle_stop, idle_minutes, max_running, start_timeout and
access_cache_seconds in meta/services.yml control the lifecycle.
The CLI test covers:
- broker-key refusal (401) and RBAC refusal (403 with no container)
- two owners getting separate runsc containers, volumes and networks
- file and network isolation between them
- a model answer relayed for both hermes and openclaw, proven from the
relay events so the check does not depend on what the model writes
Keycloak membership changes in the test run inside the broker container,
because curl will not resolve the .onion Keycloak URL of a Tor deployment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
registeredServices() derives each slug from its env key, so
AGENT_BROKER_SERVICE_ENABLED registers agent_broker, while
assertKnownService() compared the raw name. isServiceEnabled("agent-broker")
therefore threw "Unknown service" before the spec ran. The check now
normalises the name through envKey(), the same path the registry uses;
names made of [a-z0-9_] resolve exactly as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles_with_service drops every role without a canonical domain, which
suits consumers that need a URL but made LiteLLM skip headless
consumers: svc-ai-agent-broker never got its key registered and every
relayed call came back 401 "Unable to find token".
The lookup gains include_headless, which keeps such roles with an empty
canonical_domain and canonical_url, and LiteLLM's consumer key list
passes it. Callers that do not pass it behave as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test-e2e-cli loads test.env with `set -a && . test.env && set +a`. The
unquoted `ONION_PORTS=25 80 110 ...` is not an assignment there but the
command `80` with a one-word env prefix: bash prints `command not found`,
leaves ONION_PORTS unset, and onion_ports.py reports "no onion-forwarded
ports declared; nothing to probe". The port check passed vacuously in every
tor job with more than one forwarded port (OpenResty, Listmonk, Pixelfed in
run 35661520542) and only ran where a single port was declared.
Pass the value through dotenv_quote, which quotes a space-separated value in
both compose and swarm and keeps the env-file lint satisfied. A unit test
renders the template in both modes and sources it the way test-e2e-cli does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tor attaches a stream to an onion to the rendezvous circuit that matches its
isolation and keeps it there until SocksTimeout; it never moves the stream to a
fresh circuit. The probe connected without SOCKS authentication and gave up
after 60s, so a dead rendezvous circuit would absorb both passes. That is the
most likely reading of run 35661520542 (web-app-mastodon#1, public Tor), where
<onion>:80 timed out twice right after curl had reached the same onion port.
Authenticate every connect with fresh random username/password credentials.
Tor accepts any and, under its default IsolateSOCKSAuth, builds a separate
rendezvous circuit per credential set, so a retry can no longer land on the
circuit that failed the attempt before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OpenAI, Anthropic and OpenRouter routes were a role var, so an
inventory could not add a model (for example
prism-ml/ternary-bonsai-2-27b on OpenRouter) without --extra-vars. The
list now lives in services.litellm.remote_models (alias, model,
provider), and an inventory replaces it through
applications.svc-ai-litellm.services.litellm.remote_models.
- vars: LITELLM_PROVIDER_KEYS maps each provider to its credential,
LITELLM_KEYED_PROVIDERS keeps the ones with a key.
- config.yaml.j2, env.j2 and test.env.j2 publish the models of keyed
providers; the env var stays <PROVIDER>_API_KEY, now written once per
provider instead of once per model.
- AI_REMOTE_ALIASES reads the same list, so the LITELLM_CHAT_MODEL
fallback follows an inventory override.
- The deploy refuses a model whose provider has no key slot, instead of
silently leaving it unpublished.
Default output is unchanged: rendering HEAD and this commit side by side
over all 8 key combinations (40 cases across config, env, test env and
AI_REMOTE_ALIASES) gives zero differences.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings in the LMStudio loopback fix (LMS_SERVER_HOST=0.0.0.0 plus the
readiness probe changes), which is what failed the LiteLLM model warm and took
Matrix, Mattermost, Moodle, XWiki and Zammad down with it on the last run.
Also folds CLAUDE.md and GEMINI.md into AGENTS.md, and carries upstream's own
regeneration of the MCP audit report, which matches ours.
No conflicts.
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.
The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.
A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.
The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stubs native_metrics_apps and asserts the intersection, including the case the
refactor exists for: a role that ships a fragment and is deployed but is not
scraped must not reach the rule file.
Review had these removed across the role once already, and the 11 September
pass cleared every file that was flagged. The seal-key pin and the
recovery-key work then reintroduced the same pattern in nine new places.
Also removes two that predate the review and were never flagged: the
healthcheck note in meta/services.yml and the entrypoint note in
compose.yml.j2. Both are the same narrative-exception shape.
Param docs, file headers and the plain contract notes in the Playwright specs
stay: separate categories, none of them objected to. What the deleted
comments recorded lives in the role README.
Six assertions: the key is persisted on the deploy that creates it, it is
loaded before the logins that may need it, recovery is gated on both logins
having failed and a key existing, the rebuilt AppRole is the inventory one
rather than the recorded one, the recovery path ends by re-pinning, and a
rejected key fails loudly.
Recovering to the recorded pair instead of the inventory pair would strand
the next deploy the same way, which is why that one is asserted directly.
Nothing stopped the seal key drifting back into the per-deploy rotation, and
the regression is invisible until a deploy is skipped and the node cannot
unseal. Assert the pin on the seal key and its absence on both AppRole
credentials.
Brings in Kevin's AI-agent work, the svc-net-firewall role and the update
tooling. One conflict, in the MCP proxy audit: both sides appended a role to
privileged_control_plane, ours web-app-openbao and upstream svc-net-firewall.
Taken as a union at 25, which is what test_mcp_audit_completeness requires -
neither role declares meta/mcp.yml, so the audit is the only place they are
classified.
_all_project_files carried @lru_cache(maxsize=1) and read PROJECT_ROOT from the
module global inside its body, so the root was not part of the cache key. The
first call decided the answer for the rest of the process: point PROJECT_ROOT at
a fixture afterwards and the walk still reports the real repository, or, in the
other order, a sweep after the fixture is torn down enumerates a directory that
no longer exists.
Both failure modes are silent. iter_project_files, iter_project_files_with_content
and iter_non_ignored_files all sit on this one walk, 202 files in the tree use
them, and a lint whose corpus collapses to zero paths reports zero offenders and
passes. Nothing raises.
The root is now a parameter and therefore the key, with room for four of them.
_reset() still clears it through cache_clear(), and the only other caller is
iter_project_files, which passes str(PROJECT_ROOT) explicitly.
Live impact today is none, and that is worth stating: exactly one test patches
PROJECT_ROOT in this module, and it calls _reset() in both setUp and
addCleanup, so the defect was fully masked. Every other patch site in the repo
targets PROJECT_ROOT in a different module. This is about the next test that
forgets, not about a failure anyone has seen.
A regression test covers it rather than a comment. Negative control: with the
key removed the new test fails on the second root with
Items in the first set but not the second: '/tmp/.../first.txt'
and passes once the key is back; the file is byte-identical to its pre-probe
state afterwards.
The lint docstring that quoted the old cache shape is deleted rather than
updated, because a cross-module implementation reference goes stale the moment
that module changes, which is exactly what happened here.
Same class as c4556fe72d, which keyed the applications SPOT by the groups its
payload reads. That one was a live defect in utils/cache/base.py; this is the
latent twin one module over.
Residual: maxsize=4 is a choice, not a measurement. More than four roots alive
at once evicts the oldest, which re-walks rather than answers wrongly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:
Missing default for web-svc-cdn: services.dashboard.shared
Missing default for web-svc-mirror: services.dashboard.shared
[ERROR] Inventory validation failed. Aborting deployment.
_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.
Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.
Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.
Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.
Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ONLYOFFICE listed onlyoffice 10.2.0 in the Nextcloud appstore on 2026-09-18 at
10:16:31Z and never published the matching GitHub artifact; the tag itself does
not exist. `occ app:install` takes the newest compatible release, so it dies on
a 404 that no retry can heal. CI proved that twice independently: upstream run
35329488812 and fork run 35330645276 both fail Nextcloud#4 at that task after
six attempts, on the same commit, for the same reason.
The fault had three layers, each visible only once the one above it was gone.
Install. meta/addons/onlyoffice.yml pins `config.archive` to v10.1.2, whose
artifact answers 200 and which the appstore also lists for this server major.
01_install.yml stages that archive on the node and unpacks it into custom_apps
through a stdin pipe: the webroot is a named volume, so the node can only reach
the app directory through the container, and piping avoids both `container cp`
and a cleanup step that would have to run as a user which does not own the file.
Enable and configure needed no change at all. With the release already on disk
`occ app:install` reports "already installed", which nextcloud_install_status
has always mapped to runnable, so 02_enable.yml and 03_configure.yml proceed as
they always did. That is why this commit adds no branch to the plugin pipeline.
Update. `occ app:update --all` then dragged the pinned app back to the broken
10.2.0 and killed the deploy. occ offers no exclusion flag, but it does offer
`--showonly` and single app ids, so a pinned app is simply not updated. The old
`--all` call stays byte-identical and now carries a `when`: with nothing pinned,
thirteen of the fourteen variants take exactly the path they took before. This
layer cannot appear in CI at all, because it needs a second deploy with the app
already installed, and the install always died first.
The classifier test is inverted rather than deleted:
test_an_advertised_release_whose_artifact_is_gone_stays_fatal pins the decision
that a dead artifact fails loudly instead of degrading to a silent skip.
Verified by a local compose deploy of variant 4: ok=2508 changed=149 failed=0
and `✓ 36 tests/addons/onlyoffice.spec.js:12:1`, which drives the whole
JWT-authenticated round trip to the document server over its onion address, not
merely an enabled plugin. In the live container beforehand: `occ app:enable`
returned "onlyoffice 10.1.2 enabled", `occ app:install` returned "already
installed", and `occ app:update onlyoffice` reproduced the 404 that the pin now
avoids.
The loop expression was wrong before this commit and is fixed here: Jinja reads
'\1' as an octal escape, so the backreference produced chr(1) instead of an app
id. Evaluated against ansible-core's own filters over four shapes, the tail is
now cut instead of captured, which removes the escaping hazard entirely.
occ output is not localised: with default_language=de and a de_DE.UTF-8 locale
the strings are unchanged, so parsing them is as safe as the five markers the
role already relied on.
Residual: the per-app update branch has never run with a real item inside a
deploy. Its chain and `occ app:update <id>` are each verified separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The image runs this module from a layer that holds three files of utils:
__init__.py, install/__init__.py and install/collections.py, the slice
ef45d69dc3 cut so the source tree cannot invalidate the collection install.
67f5398985 established that the module therefore stays on the standard
library, and d4ae5cf9f0 broke it 15 minutes later by routing the reads
through utils.cache.files to satisfy the cache-read lint. The import raises
ModuleNotFoundError: No module named 'utils.cache' in that layer, which no
import inside a full checkout can show, because this repository installs
utils into site-packages.
Nothing turned red. The caller reads only the exit code, and the module's
contract is exit 1 with the missing collections on stdout, so a crash - exit
1, nothing on stdout - is indistinguishable from work to do. Every image
build since has reinstalled every collection with --force-with-deps, which
is the cost the check exists to avoid. The deploy job the traceback was
found in finished green.
The reads go back to Path.read_text with a cache-read marker each. The cache
was never earning its place here: read_text is an lru_cache keyed on the
path, and one run reads the requirements file once and one MANIFEST.json per
collection, eight distinct paths, none of them twice. Copying utils/cache
into the slice instead would work - the closure is three stdlib-only files -
but it binds a package with 51 commits in the last twelve months to the
layer that installs ansible, in front of a cache with no possible hit.
The caller now exits 3 when a non-zero status arrives with an empty stdout,
a state the contract cannot produce, so the next crash stops the install
instead of quietly reinstalling everything. all.sh runs under set -e and
touches the stamp only after ansible.sh returns, so the failure also leaves
the stamp stale rather than recording an install that did not happen.
The probe that would have caught this builds the slice in a tempdir from the
Dockerfile's own COPY lines and runs the module inside it. It has to pass
python -S: without it the interpreter resolves every submodule from the
installed checkout however the slice is built, and the test passes on a
module that cannot import in the image. Verified by injecting the import
back into the tempdir copy, which reproduces the traceback verbatim.
Delta measured by loading the committed module and this one side by side:
identical on both requirements files, against the real collections directory
and an empty one, on a missing file, on non-UTF-8 input and on a corrupt
MANIFEST.json. The one behavioural difference is that a file changed within
a process is now read fresh rather than from the memo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A service flag may be written as `{{ '<role>' in group_names }}`; 114
meta/services.yml files do exactly that, and it is the only host-dependent
variable any of them uses. The rendered payload therefore differs per host
while the cache key did not: it was built from applications, users,
DOMAIN_PRIMARY and SYSTEM_EMAIL_DOMAIN alone, and in the failing job all
five host_vars files were byte-identical.
113113e553 moved the render out of every task on the premise that Ansible
forks a fresh worker per task, so the in-process cache is always cold. That
holds for workers and not for the strategy's main process, which renders a
task's name once for the first host of a batch. Anything cached there is
inherited by every worker forked afterwards, and applications.py consults
that process cache before the host's own carrier fact, so an inherited
entry outranks the correct answer.
Pretix#2 of run 35152776835 shows both sides in one task: `skipping:
[web-app-pretix-swarm-mgr-01]` against `included: dedicated.yml for
wrk-01, wrk-02`, eleven seconds before mgr-01 resolved its database to
`pretix_database`. The swarm was running the central `postgres_postgres`,
so the wait never converged and the job died after 600 attempts. Variants 0
and 1 pin the flag to a literal and both pass; only variant 2 inherits the
Jinja.
The groups join the key as a sorted comma-joined string rather than a
tuple, because carrier.py stringifies only one side of its key comparison.
A non-scalar component would make the carrier silently stop matching and
every lookup re-render.
domains derives from the applications view and inherits the same
dependency, so this closes a second instance that had not surfaced yet.
users reads group_names nowhere. No extra rendering results: group_names is
constant inside a worker, and the one process that sees several hosts is
the one the split is for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nothing here was typed by hand. The docs generator rebuilt the MCP audit
table against pins that had already landed, so fider reads v0.36.0 instead
of stable, jenkins 2.568.3-lts instead of lts, pretix 2026.7.0 instead of
stable, and flowise 24-slim instead of 24-bookworm-slim. That is the table
catching up with da665ab69 and 10c5b35e8, reading the same declaration the
updater reads.
Autoformat wrapped three test files from the same series: one condition
pulled onto a single line, three assertEqual calls broken across lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_file_spot failed the gate on the entry fixture, which spelled
meta/services.yml out by hand. The path now comes from utils.roles.mapping
as ROLE_FILE_META_SERVICES, following 4a0fca2bcb, which fixed the same
lint on the addon fixture next door.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pin outside semver was given its own 'docker-ref' marker on the theory
that every class needs one. That holds for a digest and not for a tag. A
digest pin declares something about the registry, that bytes move under a
standing tag, which no other choice says. A tag outside semver declares
that upstream publishes nothing orderable, which is exactly what
'docker-version' already means and what minio and the act runner already
carry on their version lines.
Keeping both would have put '# nocheck: docker-version, docker-ref' on
every such line: two keys, two checks, one sentence said twice. 'docker-ref'
is dropped instead. It lived in this one mapping, was never entered in the
suppression catalog, and no check ever read it.
The semver entry goes with it, because a semver pin never needs a
declaration. The test that asserted three distinct rules is replaced by two
that pin the separation where it is real: a digest marker differs from a
tag marker, and a semver pin has no marker at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven roles pin several images at one shared version, because those images
are one product: the four Jitsi components, six GitLab CNG images, eight
BigBlueButton services, three Penpot, plus Mastodon, Funkwhale and
Nextcloud Talk. The bump job walks entries one by one and takes each
image's own newest tag, so a run landing between upstream's first and last
push raises the components already there and leaves the rest behind. What
deploys afterwards is a mixed set, and for Jitsi a mixed set breaks video.
The window is real rather than theoretical. Measured on Docker Hub,
'stable-9457' exists for jitsi/prosody, jitsi/jicofo and jitsi/jvb but not
for jitsi/web, one build out of sixty.
Candidates are now grouped by role and current version, and the group is
held to the newest tag the intersection of all their tag lists still
offers. A group with nothing newer in common yields no update at all rather
than a partial one. A lone entry passes through untouched, so nothing
changes for the large majority.
The key is deliberately the role and the version, not the image namespace.
A namespace rule does not survive the data: prom/* carries four components
at four different versions, opentalk/* three, and minio, taigaio and
markoshust each two. Grouping by namespace would hold back pins that are
correctly independent.
Five of the seven roles have carried this exposure all along; the channel
grammar in the previous commit added the sixth by making Jitsi orderable at
all. This closes all of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Upstreams that ship several channels in parallel put the channel first:
LiteLLM publishes 'main-v1.77.7-stable' next to 'main-v1.77.7-nightly',
Jitsi ships 'stable-9646'. The grammar only allowed a leading 'v', so both
shapes parsed as nothing at all, sat at depth 0 next to 'latest', and the
bump job never looked at them.
The numeric core now takes an optional leading channel, and the channel
folds into the flavor the same way the patch letter already does.
Everything that is not the number is family; only the number is order. That
is what keeps '-stable' away from '-nightly' and 'stable-9646' away from a
bare '9646', which would leave the channel and change what the role
deploys.
The change is additive. Only a tag starting with letters followed by a dash
can match the new group, and every such tag failed the grammar outright
before, so no tag that parses today parses differently. The regression case
pins the four shapes that already worked, and the new cases pin the two
directions that must not happen along with the tags that must stay refused:
'act-latest', 'bookworm-slim', 'main-latest' and 'buildx-stable-1' carry a
channel but no version.
Two pins in this repository become orderable as a result: the four Jitsi
components and the LiteLLM gateway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
