Swarm always runs it, a compose host owning a mesh address runs it, a
compose host without one stays untouched, and an entry another host owns,
as the lab's credential reset leaves behind, does not count.
--include filters the generated inventory to the ids it is given literally, so a provider pulled in through the dependency closure runs without joining group_names. svc-ai-agent-broker's README therefore documented a deploy that stops on its own assert: litellm comes along, nothing it can serve does, AI_AGENT_MODEL resolves to '' and 00_core.yml refuses the agents. Run 36064589578 shows it as 'Including only application_ids: svc-ai-agent-broker' followed by the assert.
meta/services.yml gains guide_companions, read through utils/roles/meta_lookup.py beside get_role_run_after, and the generator joins it onto the role id as application_deploy_ids for the three places the Quick Setup block spells the id out. A role without companions renders byte-identical, which the README lint confirms across every role.
This declares where e74a585402 derived: a role asking for networks.internet.dns names it in its own tasks, so the generator can read it. A companion cannot be read that way. The broker's tasks state only that the model must resolve, and deploying a backend is one of two remedies, the other being a provider key on svc-ai-litellm; which one the documented path takes is a decision rather than a derivation, so it is declared and the fail_msg names both.
Declaring services.ollama on the broker instead was tried and reverted: it pulls svc-ai-ollama into the closure, which a probe confirms, but the literal include filter leaves it out of group_names all the same, so the model server would run while the assert still fired.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy refused at ok=771 failed=1 with 'services.litellm.remote_models names provider(s) mock; known providers are openai, anthropic, openrouter'. Declaring the mock among the remote models, which is what makes it an ordinary model rather than a config key of its own, runs into a guard that demands a key slot for every provider. A mock needs none: it answers from its own declared response.
The guard keeps its teeth for real providers and now names the exception and its reason, instead of tolerating it silently.
AI_MOCK_PROVIDER holds the name once. The guard, AI_REMOTE_ALIASES and AI_MOCK_ALIASES read it, and the template hands it to the route builder, which lost its own constant. Two literals spelling the same thing is how the LM Studio branch came to drop the context window its README still advertised; a parameter cannot drift.
Static checking never touched this line. It took a real deploy, 772 steps in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every AI consumer asks for LITELLM_CHAT_MODEL, so a CI round paid CPU inference for tests that only check wiring. This follows the shape svc-net-tor already uses: variant 0 runs the whole logic against chutney, variant 1 against the public network.
A mock is an ordinary remote_models entry with provider mock rather than a config key of its own, because that list already means 'an alias and a way to call it' and a mock needs no key. AI_AGENT_MODEL prefers it so the agent broker stays deterministic in every round, including the ones where the consumers get a real model.
The preload switch lives in svc-ai-ollama's variants, not in the gateway's. The gateway's services.ollama is a bare consumer flag carrying bond, enabled and shared; a preload_models key there would have been silently inert and the models would have downloaded anyway.
Variant 0 keeps every backend enabled and empties its preloads instead of disabling it, because variant 0 must stay the maximal closure - and because tor does not disable its service either, it changes the flavor.
Production is untouched: with no mock declared the chain resolves exactly as before, which is asserted rather than assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OpenAI, Anthropic and OpenRouter routes were a role var, so an
inventory could not add a model (for example
prism-ml/ternary-bonsai-2-27b on OpenRouter) without --extra-vars. The
list now lives in services.litellm.remote_models (alias, model,
provider), and an inventory replaces it through
applications.svc-ai-litellm.services.litellm.remote_models.
- vars: LITELLM_PROVIDER_KEYS maps each provider to its credential,
LITELLM_KEYED_PROVIDERS keeps the ones with a key.
- config.yaml.j2, env.j2 and test.env.j2 publish the models of keyed
providers; the env var stays <PROVIDER>_API_KEY, now written once per
provider instead of once per model.
- AI_REMOTE_ALIASES reads the same list, so the LITELLM_CHAT_MODEL
fallback follows an inventory override.
- The deploy refuses a model whose provider has no key slot, instead of
silently leaving it unpublished.
Default output is unchanged: rendering HEAD and this commit side by side
over all 8 key combinations (40 cases across config, env, test env and
AI_REMOTE_ALIASES) gives zero differences.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 104155149606 died at the warm request with
litellm.APIConnectionError: OllamaException - Cannot connect to host
ollama:11434 [Connect call failed ('192.168.244.10', 11434)]
192.168.244.10 is not ollama. It is the manager node on the lab bridge: the
name never resolved through the overlay, because the litellm task was attached
to ingress, litellm and postgres and to nothing else. The lmstudio round is the
control, where the same replica does carry the lmstudio network.
The attachment comes from _is_consumer, which reads svc-ai-litellm's own
services.<backend>.enabled and .shared. Variant 1 pins both false. The config
template meanwhile asked `'svc-ai-ollama' in group_names`, which answers a
different question: is ollama deployed anywhere in the round. It is, six edges
away, because litellm pulls postgres, postgres pulls tor, tor pulls the mirror,
the mirror pulls prometheus and prometheus declares flowise, which declares
ollama. So the gateway advertised a route it had no network for.
The template and LITELLM_CHAT_MODEL_SERVED now read the same two flags the
network attachment reads, through one constant each. Publication and routing
cannot diverge any more, and in a variant that declines a deployed backend the
warm task no longer runs at all.
LITELLM_CHAT_MODEL and LITELLM_BACKEND_ROLES are deliberately left alone.
5324eadd8b made them inventory-scoped because web-app-mattermost's assert, the
one task in its file carrying neither delegate_to nor run_once, re-templates in
each worker's own scope; web-app-discourse's addon tasks carry no delegate_to
at all. A host-scoped model name fails there again. Both templates here render
delegate_to STACK_HOST plus run_once, where group_names is authoritative.
meta/services.yml keeps its `in group_names` flags, so the five separate
substring detectors that decide whether a flag counts as dynamic keep seeing
svc-ai-litellm.
Measured over every (deployed, pinned) combination against the real
expressions and the real template: the model consumers are told to ask for is
published whenever the warm task runs, and nothing is published without its
flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.
All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.
type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.
The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.
The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.
For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.
An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the
once-flag and the container network, nothing else. The gateway published
lmstudio/default anyway, so the route existed and nothing could answer it.
wp-compose-tor.log:48994 shows the route written into the config, :51770 shows
the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine
spec ran against that. LM Studio was deployed, published and never prompted.
The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the
hosting node and container id, loops the declared models through
`lms get --yes --gguf`, overlaps them with async/poll and reaps before
playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not
assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start,
and `lms get --help` documents -y for scripting and --gguf for the format.
Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM
Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry:
`alias` is what the gateway publishes and what Ollama pulls, `name` is what
`lms get` resolves. Both roles now reference that one list, and config.yaml.j2
publishes each backend under the alias, routing Ollama to ollama/<alias> and LM
Studio to openai/<name>. The lmstudio loop skips an alias Ollama already
publishes, so two backends never produce a duplicate model_name and Ollama
keeps precedence. lmstudio/default is gone; the model name is now the same
whichever backend answers it.
Ollama's preload_models changes shape from a list of strings to the {alias,
name} entries. Every consumer moves with it: the pull command, the template
loop, and both expressions in 16_ai.yml. An inventory that overrides
`applications.svc-ai-ollama.services.ollama.preload_models` with the old string
list will break at map(attribute='alias'); this is a deliberate shape change,
in the spirit of 7b15c61dfc.
The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI
lives in inventories/development/default.yml, whose own header scopes that file
to the cross-cutting environment knobs. test_variable_definitions only knew
group_vars, so it reported the constant as undefined; the shipped inventories
are a definition source too, because the deploy CLI bakes them into host_vars,
and the guard's own docstring already claimed "defined somewhere in the
repository".
A latent crash came out of this: `| list | first` raises on an empty sequence,
so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama
with only embedding models could already hit it; the existing test dodged it by
rendering only _SERVED. `| default('')` now yields the empty string that
_SERVED already reports as false.
Tests: config.yaml.j2 is rendered rather than read, over ollama-only,
lmstudio-only, both and neither, asserting the published model_name set, that
both backends name the same alias, that a shared alias is published once and
routed to Ollama, and that the render is valid YAML. test_litellm_chat_model
gains the lmstudio branch, the no-preload case and the same-model assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway publishes only models it has a backend for and renders model_list: [] when neither ollama nor lmstudio is deployed and no OpenRouter key is set. LITELLM_CHAT_MODEL fell back to openrouter/auto without checking that key, so consumers were configured for a model the gateway does not carry and every prompt failed with a 400.
Observed in the Mattermost#1 CI job: litellm.log carries 12 occurrences of "Invalid model name passed in model=openrouter/auto" next to 400 Bad Request, and tests/test-litellm-agent.js failed on every attempt while the rest of the suite passed.
Make the fallback conditional on the key so the name is empty when nothing is served, promote LITELLM_CHAT_MODEL_SERVED from the role vars to group_vars where consumers can see it (it was the third copy of this logic and the only correct one), and switch the 14 consumers from "litellm is deployed" to "litellm serves a model". The Playwright service flag reads the same lookup, so the spec skips together with the Ansible gate. The bool filter carries weight: the folded scalar yields the string False, which Jinja treats as truthy.
A new unit test renders both expressions against Jinja and pins the invariant that a model name exists exactly when a model is served, including the case where ollama preloads embedding models only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbach
evangelostsak
