Upstream re-pushed lmstudio/llmster-preview:cpu on 2026-09-18. The old build's daemon-run.sh wrote networkInterface 0.0.0.0 into .internal/http-server-config.json before starting the server. The new one drops that block, and its lms resolves the address as --bind, then LMS_SERVER_HOST, then 127.0.0.1, reading only the port from that file.
The server therefore listened on loopback alone. The image healthcheck curls localhost and every role task drives lms through container exec, so svc-ai-lmstudio stayed green, while litellm's warm-up got Connect call failed on lmstudio:1234 and failed Mattermost#2, Matrix#2, XWiki#2, Moodle#2 and Zammad#2 in fork run 35545771696.
LMS_SERVER_HOST is the documented knob and restores exactly the old bind. The port stays expose-only on the container network, so nothing new is published.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI job 105881436871 on 0b7a8a95d:
A container path declared in meta/volumes.yml is written out again.
roles/svc-ai-lmstudio/tasks/utils/preload/node.yml:7: /root/.lmstudio/models
(declared as 'lmstudio_models')
The disk guard typed the mount target instead of deriving it. That matters more
since the split: the role now carries two nested volumes, and which of them holds
models/ is decided solely by meta/volumes.yml. A second copy of the path drifts
the moment the mount moves, which is the drift 29015311b6 built this lint for.
LMSTUDIO_MODELS_TARGET resolves it through lookup('volume', …).target, next to
LMSTUDIO_SHARED_MODELS_PATH, which already reads .name from the same entry.
The test script held the same path a second time. The lint only reads yaml and
templates, so it would never have reported that copy, and it would have gone
stale just as quietly. It now receives the value through test.env.
Not caught locally, and the reason is worth recording: the preceding `make
quality` was red on three formalities of the new test assets. I fixed those,
re-ran only those three lints, and deployed. This finding sat a few lines below
them in the same output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
c6e8990e25 kept the daemon's auth key off the shared volume and made the red
job green, but it left multi-replica operation as broken as it found it: three
daemons still shared one /root/.lmstudio, where LM Studio rewrites
extensions/plugins with rm -rf at every start, writes settings.json in place and
resets it to defaults on a torn read, and re-copies a 102 MB bin/lms while peers
execute it.
Splitting the tree by access pattern fixes that. /root/.lmstudio is node-local,
because it is per-daemon state the daemon rewrites in place. models/ stays
shared, because a blob is written once and read forever.
That leaves one question, and it is temporal rather than mechanical: the daemon
discovers models by an unconditional scan at startup and by an inotify watcher,
and inotify never sees writes made by another NFS client. So a model seeded
after a daemon started is invisible to it, whichever tool wrote the file. The
previous design answered that with `container service update --force`, which
restarts the whole inference backend svc-ai-litellm points at, on every deploy.
The seed now runs before the stack does. 00_core.yml places the blob on the NFS
export ahead of the compose include that deploys the stack, so every daemon
finds a complete tree at its own startup scan and nothing has to be restarted.
Measured on a three-node swarm with NFS:
Fetch the models into the shared store 136.14s one 531 MB download
Pre-pull on mgr-01 0.69s
Pre-pull on wrk-01 0.50s
Pre-pull on wrk-02 0.49s
Half a second is not a download. All three daemons recognised the pre-placed
model, so the cluster now stores and transfers it once instead of three times.
The nested mount across two devices raises no EXDEV, and the per-node disk guard
correctly skipped itself four times because the pre-seed covers that check.
Vendor coupling is gone from the role. `repo` and `url` are declared fields
rather than pieces cut out of a HuggingFace URL with a regex; a model on any
host needs only those two. `lms get` still knows only HuggingFace, so `source`
stays for it, and a non-HF model simply leaves it unset: the blob is seeded and
the startup scan picks it up. The external check follows, comparing against the
HuggingFace API only for entries whose url actually points there.
The guard arithmetic was wrong and is now derived rather than guessed. Peak
usage is the missing bytes plus the largest single model, not twice the declared
total, so the read-back moved ahead of the check and subtracts what the store
already holds. Before, the requirement grew after every successful seed. Staging
moved beside the tree instead of inside it, so an aborted transfer cannot leave
orphans where the indexer scans.
A deployment test ships with it and states its findings rather than its exit
code: it fails on `authPacket` by naming the shared key, on a missing or
mismatching blob by naming path and digest, and on a model that is on disk but
absent from /v1/models by saying the startup scan did not happen.
Gone with the restart: import.sh and its container-cp suppression, seed.yml,
rescan.yml, and the disk-copy factor of three that only existed because the
import staged through the container.
Residual: compose is untested here. There the pre-seed is off, the node-local
guard is the one that runs, and `lms get` is the only fetch, so a non-HF model
is not loadable in that mode. Seeding node-local volumes is the follow-up that
closes it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
acc9e25c0f moved the volume lookups to context-wired signatures, taking
application_id from the task context instead of an argument:
lookup('container_volumes', application_id, service_name) -> (service_name)
lookup('compose_volumes', application_id) -> ()
Six templates were left on the old form: the three AI services and hermes,
homeassistant and openclaw. One of them also picks up the matching
container_healthcheck lookup.
This is the same failure mode as the php paths: a refactor that leaves
callers behind, and the break only appears when the template renders, not in
the gate. A deploy in this session died exactly that way, on a template still
passing application_id to a lookup that no longer takes terms.
No file in the repository carries the old signature after this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consumers of the model plane speak one OpenAI-compatible endpoint and
carry one virtual key; which backend answers is the gateway's business.
svc-ai-litellm renders its model_list from the round's actual topology,
so an alias only exists when its backend role is in group_names, and it
emits an explicit empty list when no backend is present at all because
LiteLLM rejects a bare model_list: key.
Keys reach the gateway through the per-consumer credential rather than
being minted there: 01_keys.yml probes /key/info and only POSTs
/key/generate on 404, so reruns stay idempotent. The consumer list is
built in 00_core.yml, not inside the included file, because 01_keys.yml
delegates to the hosting node and delegate_to is templated before when.
The gateway declares ollama and lmstudio as consumer flags so the
network renderer attaches it to its own backends; without them it cannot
reach them.
svc-ai-lmstudio runs the headless llmster daemon behind expose only, so
it is reachable from the gateway but never bound on the host, matching
svc-ai-ollama.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5