fix(lmstudio): pin the quantization lms get downloads instead of describing it

The pre-pull ran lms get --yes --gguf against the repository url, which is the collection of every quantization, so LM Studio chose one. The file, sha256 and bytes beside it then guessed that choice rather than stating it, and two shipped checks measure against those fields: files/test/test.sh looks for <models>/<repo>/<file>, and the download budget in pull.yml derives from bytes.

The guess held while qwen2.5:0.5b declared the Q8_0 that lms get picks anyway. llama3.2:1b declared Q4_K_M at 808 MB and lms get fetched Q8_0 at 1.32 GB, which run 36107421422 reported twice over: 'Q4_K_M.gguf is not in the model store' in litellm#2, and 'Download failed: Timed-out' in lmstudio#0 after a transfer 63% larger than the budget was sized for.

lms get --help documents the suffix the entry now carries: 'For a specific model quantization, append its name with @'. Measured against the running daemon with an invented variant so nothing downloaded, the url form resolves the repository, prints the default it would take, and refuses the unknown variant before transferring. The declaration therefore drives the download, and Q4_K_M leaves the store 39% smaller against the wedged-transfer path utils/preload/node.yml already documents.

A lint holds the three fields to one variant, because the disagreement was silent: lms get succeeded, the daemon served the model, and only a later check noticed it had measured against a guess. Each of its cases was run against the defect it names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

group_vars/all/16_ai.yml

@@ -30,6 +30,7 @@ AI_LOCAL_MODELS:
     repo: lmstudio-community/Llama-3.2-1B-Instruct-GGUF
     url: https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf
     source: https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF
+    quant: q4_k_m
     file: Llama-3.2-1B-Instruct-Q4_K_M.gguf
     sha256: "f7ede42862ceca07ad1c88a97b67520019c4ac7e5ced250d2e696fa62ab189af"
     bytes: 807690688

roles/svc-ai-lmstudio/README.md

@@ -35,7 +35,7 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (
 - **OpenAI-compatible endpoint:** The headless server answers `/v1` requests on port 1234 inside the container network.
 - **CPU inference:** The role pins the CPU build of the upstream image and passes no GPU device into the container.
 - **Persistent model store:** A named volume mounted at `/root/.lmstudio` keeps downloaded models and server settings across redeploys.
-- **Preloaded models:** Every entry of `services.lmstudio.preload_models` is downloaded on the hosting node with `lms get --gguf`, overlapped and reaped like the Ollama pre-pull. Each entry carries the `source` repository to download, the `name` LM Studio indexes that repository under and the gateway addresses it by, and the `alias` the gateway publishes it as. `source` is a full Hugging Face URL, never a search term: a search term resolves to whichever model the catalogue lists first.
+- **Preloaded models:** Every entry of `services.lmstudio.preload_models` is downloaded on the hosting node with `lms get --gguf`, overlapped and reaped like the Ollama pre-pull. Each entry carries the `source` repository to download, the `quant` variant to take from it, the `name` LM Studio indexes that repository under and the gateway addresses it by, and the `alias` the gateway publishes it as. `source` is a full Hugging Face URL, never a search term: a search term resolves to whichever model the catalogue lists first. `quant` is appended as `@<variant>` and is required: without it LM Studio picks the variant itself, and the `file`, `sha256` and `bytes` the entry declares then describe a guess that the store check and the download budget are both measured against.
 - **Loading:** The pre-pull downloads without loading. LM Studio serves with just-in-time loading, so the first `/v1/chat/completions` naming a `name` loads that model; the gateway's warm task issues that first request at deploy time.
 - **Gateway backend:** The LiteLLM Gateway publishes those aliases, which are the names Ollama serves too, so the same model name routes to whichever local backend a deployment runs.
 - **Bounded resources:** The container is capped at 4 CPUs, 8 GB of memory and 2048 processes, and the role declares a minimum of 588 MB free storage for the image; downloaded models grow the volume beyond that.

roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml

@@ -1,7 +1,7 @@
 ---
 - name: ⬇️ Pre-pull LM Studio models
   ansible.builtin.shell: > # nocheck: container-exec-resolver  runtime-resolved CID, delegated to the hosting node
-    container exec -i {{ LMSTUDIO_LOCAL_CID | quote }} lms get --yes --gguf {{ model.source | quote }}
+    container exec -i {{ LMSTUDIO_LOCAL_CID | quote }} lms get --yes --gguf {{ (model.source ~ '@' ~ model.quant) | quote }}
   environment:
     CONTAINER_EXEC_TIMEOUT: >-
       {{ [ LMSTUDIO_PRELOAD_FLOOR_SECONDS | int,

tests/lint/ansible/variables/test_local_model_quant_pinned.py

@@ -0,0 +1,110 @@
+"""An ``AI_LOCAL_MODELS`` entry names one quantization in every field.
+
+Rationale
+=========
+``roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml`` downloads with
+``lms get --yes --gguf <source>@<quant>``. Without the ``@<quant>`` suffix LM
+Studio picks the variant itself, and the ``file``, ``sha256`` and ``bytes`` the
+entry declares become a guess about that pick rather than a description of it.
+
+Two shipped checks measure against those fields, so the guess is not harmless.
+``roles/svc-ai-lmstudio/files/test/test.sh`` looks for
+``<models>/<repo>/<file>`` and reports ``is not in the model store`` when the
+store holds a different variant. ``pull.yml`` derives the download budget from
+``bytes``, so a declaration smaller than the real file cuts the transfer off.
+
+Run 36107421422 hit both at once: the entry declared ``Q4_K_M`` at 808 MB while
+``lms get`` fetched ``Q8_0`` at 1.32 GB. The declared quantization is therefore
+pinned here against the other three fields, because an entry whose fields
+disagree fails far from the line that introduced the disagreement.
+
+Per-line opt-out
+================
+Add ``# nocheck: local-model-quant-pinned`` on the offending line or the one
+above it, with a comment naming why the fields may diverge.
+"""
+
+from __future__ import annotations
+
+import unittest
+from typing import ClassVar
+
+import yaml
+
+from utils.annotations.suppress import is_suppressed_at
+
+from . import PROJECT_ROOT
+
+_RULE = "local-model-quant-pinned"
+_VARS_FILE = PROJECT_ROOT / "group_vars" / "all" / "16_ai.yml"
+_KEY = "AI_LOCAL_MODELS"
+
+
+def _entries() -> list[dict]:
+    data = yaml.safe_load(_VARS_FILE.read_text(encoding="utf-8")) or {}
+    declared = data.get(_KEY)
+    return [entry for entry in (declared or []) if isinstance(entry, dict)]
+
+
+def _line_of(alias: str) -> int:
+    for number, line in enumerate(
+        _VARS_FILE.read_text(encoding="utf-8").splitlines(), 1
+    ):
+        if f"alias: {alias}" in line:
+            return number
+    return 0
+
+
+class TestLocalModelQuantPinned(unittest.TestCase):
+    rel: ClassVar[str] = _VARS_FILE.relative_to(PROJECT_ROOT).as_posix()
+
+    def _unsuppressed(self) -> list[dict]:
+        return [
+            entry
+            for entry in _entries()
+            if not is_suppressed_at(
+                str(_VARS_FILE), _line_of(str(entry.get("alias", ""))), _RULE
+            )
+        ]
+
+    def test_every_entry_pins_a_quantization(self) -> None:
+        missing = [
+            entry.get("alias")
+            for entry in self._unsuppressed()
+            if not str(entry.get("quant") or "").strip()
+        ]
+        self.assertFalse(
+            missing,
+            f"{self.rel}: {missing} declare no 'quant', so lms get picks the "
+            "variant and the declared file, sha256 and bytes describe a guess",
+        )
+
+    def test_the_declared_file_carries_the_pinned_quantization(self) -> None:
+        drifted = [
+            (entry.get("alias"), entry.get("quant"), entry.get("file"))
+            for entry in self._unsuppressed()
+            if str(entry.get("quant") or "").strip()
+            and str(entry.get("quant")).lower()
+            not in str(entry.get("file") or "").lower()
+        ]
+        self.assertFalse(
+            drifted,
+            f"{self.rel}: {drifted} name a quantization the file does not "
+            "carry; the store check looks for that file and reports it missing",
+        )
+
+    def test_the_url_resolves_the_declared_file(self) -> None:
+        drifted = [
+            (entry.get("alias"), entry.get("file"))
+            for entry in self._unsuppressed()
+            if not str(entry.get("url") or "").endswith(str(entry.get("file") or ""))
+        ]
+        self.assertFalse(
+            drifted,
+            f"{self.rel}: {drifted} declare a url that does not end in the "
+            "declared file, so the digest is pinned against a different blob",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()