kevinveenbirkenbachandClaude Opus 5 b18710e188 fix(litellm): publish a backend only when the gateway's own flag enables it ...

Job 104155149606 died at the warm request with

    litellm.APIConnectionError: OllamaException - Cannot connect to host
    ollama:11434 [Connect call failed ('192.168.244.10', 11434)]

192.168.244.10 is not ollama. It is the manager node on the lab bridge: the
name never resolved through the overlay, because the litellm task was attached
to ingress, litellm and postgres and to nothing else. The lmstudio round is the
control, where the same replica does carry the lmstudio network.

The attachment comes from _is_consumer, which reads svc-ai-litellm's own
services.<backend>.enabled and .shared. Variant 1 pins both false. The config
template meanwhile asked `'svc-ai-ollama' in group_names`, which answers a
different question: is ollama deployed anywhere in the round. It is, six edges
away, because litellm pulls postgres, postgres pulls tor, tor pulls the mirror,
the mirror pulls prometheus and prometheus declares flowise, which declares
ollama. So the gateway advertised a route it had no network for.

The template and LITELLM_CHAT_MODEL_SERVED now read the same two flags the
network attachment reads, through one constant each. Publication and routing
cannot diverge any more, and in a variant that declines a deployed backend the
warm task no longer runs at all.

LITELLM_CHAT_MODEL and LITELLM_BACKEND_ROLES are deliberately left alone.
5324eadd8b made them inventory-scoped because web-app-mattermost's assert, the
one task in its file carrying neither delegate_to nor run_once, re-templates in
each worker's own scope; web-app-discourse's addon tasks carry no delegate_to
at all. A host-scoped model name fails there again. Both templates here render
delegate_to STACK_HOST plus run_once, where group_names is authoritative.

meta/services.yml keeps its `in group_names` flags, so the five separate
substring detectors that decide whether a flag counts as dynamic keep seeing
svc-ai-litellm.

Measured over every (deployed, pinned) combination against the real
expressions and the real template: the model consumers are told to ask for is
published whenever the warm task runs, and nothing is published without its
flag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-15 02:06:38 +02:00

..

__init__.py

…

test_litellm_chat_model.py

fix(litellm): publish a backend only when the gateway's own flag enables it

2026-09-15 02:06:38 +02:00