mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 11:46:35 +00:00
kevinveenbirkenbachandClaude Opus 5
cd05368c9b
Ollama unloads an idle model after five minutes by default. Preloading pulls the model at deploy time but only the first inference loads it into memory, so a consumer prompting a while after the deploy pays a cold load that eats into its own provider timeout. Declare keep_alive next to preload_async_time in the services SPOT, read it in vars like its neighbours, and pass it to the container as OLLAMA_KEEP_ALIVE. `-1` keeps the model loaded indefinitely; `ollama ps` then reports UNTIL: Forever instead of Stopping... This removes real latency but it is not what fixed the zammad timeout that prompted it: that was a model which never emitted a stop token, addressed separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
19 lines
674 B
Django/Jinja
19 lines
674 B
Django/Jinja
{% include 'roles/sys-svc-compose/templates/base.yml.j2' %}
|
|
|
|
{% set service_name = 'ollama' %}
|
|
{{ service_name }}:
|
|
{{ lookup('container_healthcheck', service_name) | indent(4) }}
|
|
{% include 'roles/sys-svc-container/templates/base.yml.j2' %}
|
|
{{ lookup('container_image', application_id, 'ollama') }}
|
|
{{ lookup('compose_only', 'container_name', OLLAMA_CONTAINER) }}
|
|
expose:
|
|
- "{{ OLLAMA_PORT }}"
|
|
environment:
|
|
OLLAMA_KEEP_ALIVE: "{{ OLLAMA_KEEP_ALIVE }}"
|
|
{{ lookup('container_volumes', service_name) | indent(4) }}
|
|
{{ lookup('container_networks') | indent(4) }}
|
|
|
|
{{ lookup('compose_networks') }}
|
|
|
|
{{ lookup('compose_volumes') }}
|