mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 19:56:53 +00:00
kevinveenbirkenbachandClaude Opus 5
eed9215aed
The `auto` alias picked a model that could not hold the prompt, and every layer that should have caught it was comparing this deployment's own declarations against each other. Three errors produced one failure. smollm2:135m was declared at 16384 while the tag trains to 8192, so Ollama clamped num_ctx and then truncated the prompt to half the window rather than refusing it. The token estimate ran at three characters per token and was documented as erring high, but the oversized prompt is built from "x ", which tokenizes at two: 16384 characters are 8193 tokens on all three served tokenizers against an estimate of 5461. A window too large and a demand too small cancel into "eligible", and neither correction works alone; both ship. The third error is what fired them: at one sample the speed measurement kept its cold load, so a 135M model measured slower than a 0.5B one and the router ranked on the inversion. The warm-up is now discarded at every setting. The probe compared a declared window against an estimated demand, both ours, so it could not observe a declaration the backend does not honour. It now reads usage.prompt_tokens and fails when the backend evaluated materially less than was sent. Against the live stack that check separates the two models exactly: smollm2 reports 4098 of 8192 sent and fails, llama3.2 reports 8221 and passes, and 4098 is the figure Ollama's own truncation logged. A 400 now carries its body, without which this failure reads as "HTTP Error 400" and nothing more. The strategy is selectable. Where jeff is deployed the decision comes from a System One encoder reading the prompt; otherwise the routes are ranked on standing preferences, and each factor carries its own weight. Both were exercised over six prompt sizes: system_one chose qwen, then smollm2, then the mock as the prompt grew, weighted chose llama3.2:1b throughout. The state handed to the decider is capped at 512 characters, because the encoder holds 512 positions and the previous 20000 came from the server's own rejection limit, which is a different number: measured through the gateway, a 2000 character state already exceeded the decider's timeout, and one timeout leaves the request running so every later one queues behind it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>