mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
fix(ai): pick the chat model from the deployed closure, not the whitelist
LITELLM_BACKEND_ROLES decides which backend branch LITELLM_CHAT_MODEL takes:
the first non-embedding ollama preload model, lmstudio/default, or the
openrouter/auto vendor fallback. It read lookup('deployment').running, which
plugins/lookup/deployment.py resolves as `whitelist or groups`.
On any targeted apps= deploy that whitelist holds only the roles named on
the command line, while backends like svc-ai-ollama arrive as dependencies
of the round. Both branches therefore missed and every consumer was
configured with openrouter/auto, a vendor model the in-cluster gateway does
not serve. The gateway answered each prompt with
400 Invalid model name passed in model=openrouter/auto
while /v1/models listed smollm2:135m and lmstudio/default, so the surface
looked correctly wired while no consumer could complete a prompt.
Read .groups instead, the round's deployed closure, which is what the
`'<role>' in group_names` form used across meta/services.yml already means.
This reverses the selector introduced in 46a16127fc, whose rationale in
requirement 036 assumed the two selectors agree on a single-host round;
they do not once apps= is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0f6728a4c1
commit
b149ad0d22
1 file changed
+1
-1
@@ -11,7 +11,7 @@ LMSTUDIO_BASE_LOCAL_URL: "http://{{ lookup('config','svc-ai-lmstudio', 'services
|
||||
|
||||
LITELLM_OPENAI_BASE_LOCAL_URL: "{{ LITELLM_BASE_LOCAL_URL }}/v1"
|
||||
|
||||
LITELLM_BACKEND_ROLES: "{{ lookup('deployment').running }}"
|
||||
LITELLM_BACKEND_ROLES: "{{ lookup('deployment').groups }}"
|
||||
|
||||
LITELLM_CHAT_MODEL: "{{ (lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') | reject('search', 'embed') | list | first) if 'svc-ai-ollama' in LITELLM_BACKEND_ROLES
|
||||
else ('lmstudio/default' if 'svc-ai-lmstudio' in LITELLM_BACKEND_ROLES else 'openrouter/auto') }}"
|
||||
|
||||
Reference in new issue
Block a user