mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 03:38:23 +00:00
kevinveenbirkenbachandClaude Opus 5
dac61edbc6
svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the
once-flag and the container network, nothing else. The gateway published
lmstudio/default anyway, so the route existed and nothing could answer it.
wp-compose-tor.log:48994 shows the route written into the config, :51770 shows
the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine
spec ran against that. LM Studio was deployed, published and never prompted.
The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the
hosting node and container id, loops the declared models through
`lms get --yes --gguf`, overlaps them with async/poll and reaps before
playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not
assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start,
and `lms get --help` documents -y for scripting and --gguf for the format.
Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM
Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry:
`alias` is what the gateway publishes and what Ollama pulls, `name` is what
`lms get` resolves. Both roles now reference that one list, and config.yaml.j2
publishes each backend under the alias, routing Ollama to ollama/<alias> and LM
Studio to openai/<name>. The lmstudio loop skips an alias Ollama already
publishes, so two backends never produce a duplicate model_name and Ollama
keeps precedence. lmstudio/default is gone; the model name is now the same
whichever backend answers it.
Ollama's preload_models changes shape from a list of strings to the {alias,
name} entries. Every consumer moves with it: the pull command, the template
loop, and both expressions in 16_ai.yml. An inventory that overrides
`applications.svc-ai-ollama.services.ollama.preload_models` with the old string
list will break at map(attribute='alias'); this is a deliberate shape change,
in the spirit of 7b15c61dfc.
The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI
lives in inventories/development/default.yml, whose own header scopes that file
to the cross-cutting environment knobs. test_variable_definitions only knew
group_vars, so it reported the constant as undefined; the shipped inventories
are a definition source too, because the deploy CLI bakes them into host_vars,
and the guard's own docstring already claimed "defined somewhere in the
repository".
A latent crash came out of this: `| list | first` raises on an empty sequence,
so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama
with only embedding models could already hit it; the existing test dodged it by
rendering only _SERVED. `| default('')` now yields the empty string that
_SERVED already reports as false.
Tests: config.yaml.j2 is rendered rather than read, over ollama-only,
lmstudio-only, both and neither, asserting the published model_name set, that
both backends name the same alias, that a shared alias is published once and
routed to Ollama, and that the render is valid YAML. test_litellm_chat_model
gains the lmstudio branch, the no-preload case and the same-model assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
42 lines
1.7 KiB
YAML
42 lines
1.7 KiB
YAML
---
|
|
- name: "🔍 Resolve {{ application_id }} hosting node + container id"
|
|
ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/swarm/resolve_host_cid.yml') }}"
|
|
vars:
|
|
resolve_service_key: "{{ entity_name }}"
|
|
resolve_output_prefix: OLLAMA
|
|
resolve_compose_cid: "{{ OLLAMA_CONTAINER }}"
|
|
|
|
- name: ⬇️ Pre-pull Ollama models
|
|
ansible.builtin.shell: > # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node
|
|
container exec -i {{ OLLAMA_LOCAL_CID | quote }} ollama pull {{ model.alias }}
|
|
register: pull_result
|
|
loop: "{{ OLLAMA_PRELOAD_MODELS }}"
|
|
loop_control:
|
|
loop_var: model
|
|
label: "{{ model.alias }}"
|
|
# rationale: async; model pulls run overlapped, results reaped below before playwright
|
|
async: "{{ OLLAMA_PRELOAD_ASYNC_TIME if ASYNC_ENABLED | bool else omit }}"
|
|
poll: "{{ ASYNC_POLL if ASYNC_ENABLED | bool else omit }}"
|
|
retries: "{{ 0 if ASYNC_ENABLED | bool else 5 }}"
|
|
delay: 15
|
|
until: >
|
|
(pull_result.rc | default(0)) == 0 or
|
|
('up to date' in (pull_result.stdout | default('')))
|
|
changed_when: false
|
|
failed_when: >-
|
|
(pull_result.failed | default(false))
|
|
and 'up to date' not in ((pull_result.stdout | default('')) ~ (pull_result.stderr | default('')))
|
|
delegate_to: "{{ OLLAMA_HOST_NODE }}"
|
|
|
|
- name: "⏳ Reap the Ollama model pulls"
|
|
ansible.builtin.include_tasks:
|
|
file: utils/async/reap.yml
|
|
apply:
|
|
delegate_to: "{{ OLLAMA_HOST_NODE }}"
|
|
vars:
|
|
async_reap_jobs: "{{ pull_result.results | default([]) }}"
|
|
async_reap_label: "Ollama model pre-pull"
|
|
async_reap_tolerate:
|
|
- "up to date"
|
|
ASYNC_TIME: "{{ OLLAMA_PRELOAD_ASYNC_TIME }}"
|