mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-09 04:06:42 +00:00
kevinveenbirkenbachandClaude Opus 5
12306dc696
The harness derived LITELLM_SERVED_PROVIDERS from the keys argument, i.e. the
providers that hold a key. vars/main.yml:16-18 splits that in two: the keyed
list feeds env.j2, which indexes LITELLM_PROVIDER_KEYS and fails the render on
a keyless entry, while the served list is the keyed one plus AI_MOCK_PROVIDER
and decides what config.yaml.j2:4 publishes. The harness carried the name of
the second and the meaning of the first, so it reproduced the very defect it
was meant to catch: a mock filtered out of model_list, an empty list, and the
gateway answering 400 for the chat model while every name-level variable still
resolves to it. That cost a deploy and every warm task in run 35875571687.
TestMockModels stayed green through it by passing keys={"mock": "none-needed"},
a key the mock provider does not have and never needs, which walked around the
broken assumption instead of asserting against it.
The harness now derives the list the way vars/main.yml does, the fake key is
gone, and a new case states the failure directly: a mock must publish a route
although no provider holds a key. Negative control: reverting
LITELLM_SERVED_PROVIDERS to LITELLM_KEYED_PROVIDERS fails
test_the_mock_is_served_without_any_key with "'mock' not found in []".
meta/services.yml declares no mock in remote_models, so the default-argument
cases keep their old outcome; TestNoBackend still renders an empty model_list.
Residual: template and vars are pinned by two separate tests against one
hand-copied derivation rather than a shared source. A drift in vars/main.yml is
caught only by test_keyed_providers.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>