mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 19:56:53 +00:00
kevinveenbirkenbachandClaude Opus 5
11fef84a2f
The healthcheck is http_status on healthz, which answers as soon as the HTTP server binds, while the model loads on the first real question. Everything that waits on the healthcheck therefore treats jeff as ready during a window in which it cannot answer. svc-ai-litellm's router hook allows one attempt of services.jeff.request_timeout and refuses to route on any error rather than substituting a decider, so a consumer arriving in that window gets a 400 saying the system_one strategy could not reach a decision with TimeoutError: timed out. Run 36064589578 carries that twice in litellm's container log, after which the same route answered in 6.7 seconds. 01_manager_ops.yml puts one real choice question after the handler flush and repeats until it is answered, in the shape svc-ai-litellm already uses to warm its chat model: resolve the hosting node and container id, then pass the script in through python3 -c. Raising request_timeout or retrying inside the hook would only move the window, and the refusal itself is correct and stays. The script reads the port, the alias and the key from jeff's own environment, so nothing reaches the command line that a task echo would carry. It checks that the answers block holds the question it asked, because a 200 without an answer means the model did not decide and reporting success there hands the cold load back to the first caller. The mirrored test pins that exit and pins the budget above request_timeout, since a warm-up on the router's budget would time out exactly where the router did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>