fix(svc-ai-litellm): check a host consumer against the URL it actually uses

The contract asserted one shape for every consumer: an undotted service name on
the container network, at the gateway's internal port. That is right for a
containerised app and unreachable for a workstation, which is not on that
network and reaches the gateway over the port the engine publishes. A host
consumer therefore had to either fail the contract or skip it.

modes.host.enabled now routes the assert. The virtual key stays unconditional,
because both shapes carry one; the container assert gains its negation, and the
host case gets its own: plain http, a hostname from LITELLM_LOOPBACK_HOSTS, and
the published port rather than the internal one. The intent is unchanged in
both branches, that a workstation's key never leaves the machine and an app's
inference never leaves the deployment.

LITELLM_BASE_LOOPBACK_URL derives from the first entry of LITELLM_LOOPBACK_HOSTS
rather than from DOCKER_BIND_HOST, which is 0.0.0.0 whenever DOCKER_IN_CONTAINER
is set and so describes where the engine binds, not where a client connects.
Built from the bind address the assert would have rejected its own URL on every
local stack. Binding to 0.0.0.0 covers loopback, so 127.0.0.1 reaches the
published port both on a real host and inside the nested container, and the
list is the single place the allowed set is written.

Inert until a host role calls the contract: no role at this commit enables
modes.host and invokes it, so every current consumer takes the container branch
it already took. 345 ansible lints pass with the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed 2026-09-25 13:37:50 +02:00
1 parent 7a4f7e7658
commit 95bf5898a8
2 files changed
+30

No files matched your search

+9
View File
@@ -13,6 +13,15 @@ S1_BASE_LOCAL_URL: "http://{{ lookup('config','svc-ai-s1', 'services.s1.name') }
LITELLM_OPENAI_BASE_LOCAL_URL: "{{ LITELLM_BASE_LOCAL_URL }}/v1"
LITELLM_LOOPBACK_HOSTS:
- "127.0.0.1"
- "::1"
- "localhost"
LITELLM_BASE_LOOPBACK_URL: "http://{{ LITELLM_LOOPBACK_HOSTS | first }}:{{ lookup('config','svc-ai-litellm', 'services.litellm.ports.local.http') }}"
LITELLM_OPENAI_BASE_LOOPBACK_URL: "{{ LITELLM_BASE_LOOPBACK_URL }}/v1"
LITELLM_BACKEND_ROLES: "{{ lookup('deployment').inventory }}"
LITELLM_OLLAMA_BACKEND: >-
@@ -1,4 +1,8 @@
---
- name: "🔧 Read how '{{ application_id }}' reaches the gateway"
ansible.builtin.set_fact:
_litellm_consumer_on_host: "{{ lookup('config', application_id, 'services.' ~ (application_id | get_entity_name) ~ '.modes.host.enabled', false) | bool }}"
- name: "✅ Assert '{{ application_id }}' carries a gateway virtual key"
ansible.builtin.assert:
that:
@@ -23,3 +27,20 @@
ingress), a host that is not this role's own domain, and the gateway's
internal port {{ lookup('config', 'svc-ai-litellm', 'services.litellm.ports.internal.http') }}
rather than its published one.
when: not (_litellm_consumer_on_host | bool)
- name: "✅ Assert the AI base URL of '{{ application_id }}' stays on this machine"
ansible.builtin.assert:
that:
- (LITELLM_BASE_LOOPBACK_URL | urlsplit('scheme')) == 'http'
- (LITELLM_BASE_LOOPBACK_URL | urlsplit('hostname')) in LITELLM_LOOPBACK_HOSTS
- (LITELLM_BASE_LOOPBACK_URL | urlsplit('port') | int) == (lookup('config', 'svc-ai-litellm', 'services.litellm.ports.local.http') | int)
fail_msg: >-
{{ application_id }} runs on the host rather than in the deployment's
container network, so it would send inference traffic to
'{{ LITELLM_BASE_LOOPBACK_URL }}'. A workstation reaches the gateway only
over the loopback address the engine publishes it on: plain http, one of
{{ LITELLM_LOOPBACK_HOSTS | join(', ') }}, and the published port
{{ lookup('config', 'svc-ai-litellm', 'services.litellm.ports.local.http') }}.
Anything routable would carry the workstation's key off this machine.
when: _litellm_consumer_on_host | bool