6 Commits

Include renames

Author	SHA1	Message	Date
kevinveenbirkenbachandClaude Opus 5	f10d281533	fix(lmstudio): read the models mount from the registry, not from memory ... CI job 105881436871 on 0b7a8a95d: A container path declared in meta/volumes.yml is written out again. roles/svc-ai-lmstudio/tasks/utils/preload/node.yml:7: /root/.lmstudio/models (declared as 'lmstudio_models') The disk guard typed the mount target instead of deriving it. That matters more since the split: the role now carries two nested volumes, and which of them holds models/ is decided solely by meta/volumes.yml. A second copy of the path drifts the moment the mount moves, which is the drift 29015311b6 built this lint for. LMSTUDIO_MODELS_TARGET resolves it through lookup('volume', …).target, next to LMSTUDIO_SHARED_MODELS_PATH, which already reads .name from the same entry. The test script held the same path a second time. The lint only reads yaml and templates, so it would never have reported that copy, and it would have gone stale just as quietly. It now receives the value through test.env. Not caught locally, and the reason is worth recording: the preceding `make quality` was red on three formalities of the new test assets. I fixed those, re-ran only those three lints, and deployed. This finding sat a few lines below them in the same output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 12:36:35 +02:00
kevinveenbirkenbachandClaude Opus 5	0b7a8a95dc	feat(lmstudio): seed the model store once, before the daemons index it ... c6e8990e25 kept the daemon's auth key off the shared volume and made the red job green, but it left multi-replica operation as broken as it found it: three daemons still shared one /root/.lmstudio, where LM Studio rewrites extensions/plugins with rm -rf at every start, writes settings.json in place and resets it to defaults on a torn read, and re-copies a 102 MB bin/lms while peers execute it. Splitting the tree by access pattern fixes that. /root/.lmstudio is node-local, because it is per-daemon state the daemon rewrites in place. models/ stays shared, because a blob is written once and read forever. That leaves one question, and it is temporal rather than mechanical: the daemon discovers models by an unconditional scan at startup and by an inotify watcher, and inotify never sees writes made by another NFS client. So a model seeded after a daemon started is invisible to it, whichever tool wrote the file. The previous design answered that with `container service update --force`, which restarts the whole inference backend svc-ai-litellm points at, on every deploy. The seed now runs before the stack does. 00_core.yml places the blob on the NFS export ahead of the compose include that deploys the stack, so every daemon finds a complete tree at its own startup scan and nothing has to be restarted. Measured on a three-node swarm with NFS: Fetch the models into the shared store 136.14s one 531 MB download Pre-pull on mgr-01 0.69s Pre-pull on wrk-01 0.50s Pre-pull on wrk-02 0.49s Half a second is not a download. All three daemons recognised the pre-placed model, so the cluster now stores and transfers it once instead of three times. The nested mount across two devices raises no EXDEV, and the per-node disk guard correctly skipped itself four times because the pre-seed covers that check. Vendor coupling is gone from the role. `repo` and `url` are declared fields rather than pieces cut out of a HuggingFace URL with a regex; a model on any host needs only those two. `lms get` still knows only HuggingFace, so `source` stays for it, and a non-HF model simply leaves it unset: the blob is seeded and the startup scan picks it up. The external check follows, comparing against the HuggingFace API only for entries whose url actually points there. The guard arithmetic was wrong and is now derived rather than guessed. Peak usage is the missing bytes plus the largest single model, not twice the declared total, so the read-back moved ahead of the check and subtracts what the store already holds. Before, the requirement grew after every successful seed. Staging moved beside the tree instead of inside it, so an aborted transfer cannot leave orphans where the indexer scans. A deployment test ships with it and states its findings rather than its exit code: it fails on `authPacket` by naming the shared key, on a missing or mismatching blob by naming path and digest, and on a model that is on disk but absent from /v1/models by saying the startup scan did not happen. Gone with the restart: import.sh and its container-cp suppression, seed.yml, rescan.yml, and the disk-copy factor of three that only existed because the import staged through the container. Residual: compose is untested here. There the pre-seed is off, the node-local guard is the one that runs, and `lms get` is the only fetch, so a non-HF model is not loadable in that mode. Seeding node-local volumes is the follow-up that closes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 11:50:29 +02:00
kevinveenbirkenbachandClaude Opus 5	eef3c3be71	fix(lmstudio): bound the model fetch by size, disk and a fallback source ... A wedged LM Studio daemon cost CI job 105558224377 three hours: three kills at the global 60-minute `CONTAINER_EXEC_TIMEOUT`, for a 531 MB model that a healthy node pulls in 27 seconds. Nothing in the path was sized against the thing being fetched, and nothing checked whether it could land at all. Size. Each model now declares `bytes`, taken from the value the deploy already printed as "OK (531068224 bytes)", and the pre-pull derives its own timeout per model: max(preload_floor_seconds, bytes / preload_min_bytes_per_second). The floor of 600s protects a small model from a brief stall; the rate of 1 MB/s sits twenty times below the 21.98 MB/s a healthy pull reached and seven thousand times above the 142 B/s the wedged one managed, so it separates the two without guessing. Evaluated against ansible-core's filters: 531 MB and 10 MB both land on the floor, 4 GB gives 4000s. Disk. The seed measures free space with `df -P -B1` before fetching and refuses with a message naming both figures in GiB, the factor and the file to edit. The factor is three because the node cache, the staging copy inside the container and the models volume all exist at once during the import; that is measured, not padding. Fallback. A model may list `mirrors`, extra bases tried after the source. The mirror task is a no-op once the source delivered, because get_url skips a dest whose checksum already matches: measured at 145.24s for the first pass against 0.64s for the second. The list ships empty, because the obvious candidate is not one: hf-mirror.com answers 206 but resolves to us.aws.cdn.hf.co, so it is a front end for HuggingFace and adds no redundancy against a HuggingFace outage. The mechanism is here; an independent copy is the operator's to name. The external check grew with the declaration: `bytes` is compared against the size the API publishes, and every declared mirror must still serve the pinned file, or it is redundancy on paper only. 01_manager_ops.yml's rescue rationale claimed that restarting the container clears a wedged transfer. Job 105558224377 shows the restart returning rc 0 without clearing it, so the claim now says what was measured instead. Residual: the mirror path has never run in a deploy, since no mirror is declared. Its loop expression is verified against ansible-core's subelements over four shapes, including the zero-iteration case when nothing is mirrored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-19 06:31:48 +02:00
kevinveenbirkenbachandClaude Opus 5	ee582762b0	feat(lmstudio): fetch the model ourselves instead of trusting one client shot ... The preload is the only artifact this repository pulls raw from the public internet. Everything else, including the LM Studio image itself, arrives through the mirror. `lms get` gets exactly one attempt under async, cannot resume, and wedges its own daemon when a transfer dies, so one dropped socket costs a whole job. XWiki#2 of run 35152776835 died 28 seconds into the transfer at 3 MB/s and rising, with 3600 seconds of budget left. The model is now fetched on the hosting node with get_url_retry, which gives 13 attempts and verifies a pinned sha256, and handed to the container with `lms import`. Both steps are best effort: when the warm-up fails, `lms get` runs exactly as before, so the change adds a faster path without removing the old one. The pattern is web-app-mediawiki's extension install, down to the `container cp` in the script. Verified against the binary the role actually deploys, extracted from lmstudio/llmster-preview:cpu and built 2025-08-28, not against upstream HEAD: `lms import` takes `<file-path>`, `-y, --yes` and `--user-repo <user/repo>`; the destination is join(modelsFolderPath, user, repo, basename(path)) under join(lmstudioHome, "models"); the image ships no settings.json, so downloadsFolder cannot move that target. The already-present branch of `lms get` carries no process.exit, so it returns 0 and the task's existing `until: rc == 0` accepts it. The pin carries file and sha256 next to the source URL, and the digest is quoted because an all-digit hash would otherwise parse as an integer and fail the shape check for the wrong reason. tests/external guards the pin against what HuggingFace publishes, comparing digests through the API without transferring the file, and reports a network error as unverified rather than failing a good pin on a firewalled runner. Two limits worth stating. `lms get` still queries HuggingFace before it reaches the download step, so this removes 531 MB of exposure rather than the network dependency. And the tolerated substring 'already downloaded' in the task's until, failed_when and reap list matches no output this version emits, which says "You already have this model."; that branch is dead today and is left untouched here because the return code carries the contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 23:46:27 +02:00
kevinveenbirkenbachandClaude Opus 5	dac61edbc6	feat(ai): let LM Studio serve the models Ollama serves, under one catalogue ... svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the once-flag and the container network, nothing else. The gateway published lmstudio/default anyway, so the route existed and nothing could answer it. wp-compose-tor.log:48994 shows the route written into the config, :51770 shows the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine spec ran against that. LM Studio was deployed, published and never prompted. The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the hosting node and container id, loops the declared models through `lms get --yes --gguf`, overlaps them with async/poll and reaps before playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start, and `lms get --help` documents -y for scripting and --gguf for the format. Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry: `alias` is what the gateway publishes and what Ollama pulls, `name` is what `lms get` resolves. Both roles now reference that one list, and config.yaml.j2 publishes each backend under the alias, routing Ollama to ollama/<alias> and LM Studio to openai/<name>. The lmstudio loop skips an alias Ollama already publishes, so two backends never produce a duplicate model_name and Ollama keeps precedence. lmstudio/default is gone; the model name is now the same whichever backend answers it. Ollama's preload_models changes shape from a list of strings to the {alias, name} entries. Every consumer moves with it: the pull command, the template loop, and both expressions in 16_ai.yml. An inventory that overrides `applications.svc-ai-ollama.services.ollama.preload_models` with the old string list will break at map(attribute='alias'); this is a deliberate shape change, in the spirit of 7b15c61dfc. The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI lives in inventories/development/default.yml, whose own header scopes that file to the cross-cutting environment knobs. test_variable_definitions only knew group_vars, so it reported the constant as undefined; the shipped inventories are a definition source too, because the deploy CLI bakes them into host_vars, and the guard's own docstring already claimed "defined somewhere in the repository". A latent crash came out of this: `| list | first` raises on an empty sequence, so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama with only embedding models could already hit it; the existing test dodged it by rendering only _SERVED. `| default('')` now yields the empty string that _SERVED already reports as false. Tests: config.yaml.j2 is rendered rather than read, over ollama-only, lmstudio-only, both and neither, asserting the published model_name set, that both backends name the same alias, that a shared alias is published once and routed to Ollama, and that the render is valid YAML. test_litellm_chat_model gains the lmstudio branch, the no-preload case and the same-model assertion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 19:21:47 +02:00
kevinveenbirkenbachandClaude Opus 4.8	ad743bf447	feat(svc-ai): add the LiteLLM gateway and the LM Studio backend ... Consumers of the model plane speak one OpenAI-compatible endpoint and carry one virtual key; which backend answers is the gateway's business. svc-ai-litellm renders its model_list from the round's actual topology, so an alias only exists when its backend role is in group_names, and it emits an explicit empty list when no backend is present at all because LiteLLM rejects a bare model_list: key. Keys reach the gateway through the per-consumer credential rather than being minted there: 01_keys.yml probes /key/info and only POSTs /key/generate on 404, so reruns stay idempotent. The consumer list is built in 00_core.yml, not inside the included file, because 01_keys.yml delegates to the hosting node and delegate_to is templated before when. The gateway declares ollama and lmstudio as consumer flags so the network renderer attaches it to its own backends; without them it cannot reach them. svc-ai-lmstudio runs the headless llmster daemon behind expose only, so it is reachable from the gateway but never bound on the host, matching svc-ai-ollama. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 18:50:49 +02:00