mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-09 20:26:33 +00:00
kevinveenbirkenbachandClaude Opus 5
cd05368c9b
Ollama unloads an idle model after five minutes by default. Preloading pulls the model at deploy time but only the first inference loads it into memory, so a consumer prompting a while after the deploy pays a cold load that eats into its own provider timeout. Declare keep_alive next to preload_async_time in the services SPOT, read it in vars like its neighbours, and pass it to the container as OLLAMA_KEEP_ALIVE. `-1` keeps the model loaded indefinitely; `ollama ps` then reports UNTIL: Forever instead of Stopping... This removes real latency but it is not what fixed the zammad timeout that prompted it: that was a model which never emitted a stop token, addressed separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
10 lines
561 B
YAML
10 lines
561 B
YAML
---
|
|
application_id: "svc-ai-ollama"
|
|
entity_name: "{{ application_id | get_entity_name }}"
|
|
|
|
OLLAMA_CONTAINER: "{{ lookup('config', application_id, 'services.ollama.name') }}"
|
|
OLLAMA_PORT: "{{ lookup('config', application_id, 'services.ollama.ports.internal.http') }}"
|
|
OLLAMA_PRELOAD_MODELS: "{{ lookup('config', application_id, 'services.ollama.preload_models') }}"
|
|
OLLAMA_PRELOAD_ASYNC_TIME: "{{ lookup('config', application_id, 'services.ollama.preload_async_time') }}"
|
|
OLLAMA_KEEP_ALIVE: "{{ lookup('config', application_id, 'services.ollama.keep_alive') }}"
|