kevinveenbirkenbachandClaude Opus 5 cd05368c9b feat(svc-ai-ollama): keep the preloaded model resident ...

Ollama unloads an idle model after five minutes by default. Preloading pulls
the model at deploy time but only the first inference loads it into memory,
so a consumer prompting a while after the deploy pays a cold load that eats
into its own provider timeout.

Declare keep_alive next to preload_async_time in the services SPOT, read it
in vars like its neighbours, and pass it to the container as
OLLAMA_KEEP_ALIVE. `-1` keeps the model loaded indefinitely; `ollama ps`
then reports UNTIL: Forever instead of Stopping...

This removes real latency but it is not what fixed the zammad timeout that
prompted it: that was a model which never emitted a stop token, addressed
separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-28 11:56:16 +02:00

..

compose.yml.j2

feat(svc-ai-ollama): keep the preloaded model resident

2026-08-28 11:56:16 +02:00