Files
computer-playbook/tests/unit/python
kevinveenbirkenbachandClaude Opus 5 2d886ad19b feat(svc-ai-nllb-engine): add a translation model that covers 136 languages
The gateway had two engine kinds and neither reaches the catalogue.
LibreTranslate covers roughly 48 languages. The chat engine is backed
by the single entry in AI_LOCAL_MODELS, llama3.2:1b, which officially
speaks eight, while web-svc-translate/vars/main.yml declares every
language in meta/languages.yml as a chat target. The gateway therefore
asserted 183-language capability for a model that invents the rest:
Belarusian came back as Polish in 86 of 87 new entries, Abkhaz had no
Cyrillic in any of 83, Amharic 10 of 236 in Ethiopic, and Avaric,
Assamese and Bashkir returned the source verbatim.

NLLB-200 is built for translation rather than generation. A third
engine kind serves it beside the two libretranslate instances, since
it is seq2seq and does not belong as an ollama alias.

The honest ceiling is 136 of 184, not 184. flores.py maps 136 ISO
codes and names the 48 the model has no code for, and the test asserts
mapped plus unserved equals exactly the keys of meta/languages.yml so
neither side can drift. Belarusian, Amharic, Assamese and Bashkir are
covered; Abkhaz and Avaric are not, so two of the worst catalogues
still have no local engine.

The script half of each mapping is derived rather than typed: the test
re-derives every endonym's script from unicodedata and asserts it
matches the suffix of the FLORES code, ja and zh excepted because Han
cannot distinguish Jpan from Hans from Hant. The subtag half cannot be
derived, because where ISO 639-1 names a macrolanguage NLLB ships one
variety (ar->arb_Arab, fa->pes_Arab, ms->zsm_Latn and ten more), so it
is stated once in one table.

Counting languages surfaced a real bug: meta/languages.yml has 184
entries, not 183. 'no' is quoted because YAML reads bare no as false.

Inert until deployed. The engine entry is multiplied by zero while
TRANSLATE_NLLB_ENABLED is false, so TRANSLATE_ENGINES is unchanged;
the two new vars are evaluated unconditionally and every key they read
exists in the new role. No variant round was added, so CI does not
exercise this engine yet, and a first deploy needs an inventory
regeneration for the new service key, volume and subnet.

Assumed facebook/nllb-200-distilled-600M, about 3 GB resident, which
fits the envelope svc-ai-s1 already occupies. 1.3B buys roughly
+2 chrF++ concentrated in low-resource languages for twice the memory;
the README tables the sizes so that choice is visible.

Owed before this is offered beyond private use: the weights are
CC BY-NC 4.0, and the role does not yet say so or credit the NLLB
team. Tracked with the per-role licence index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-20 19:52:21 +02:00
..