A chat engine prefixed the Vietnamese catalog with its own preamble ("...Vietnamese:" followed by a blank line) and wrote Urdu in Devanagari. Neither tripped the damage predicate: vi is Latin script, so the script checks were blind, and untranslated only measures the Latin share, so a non-Latin language answered in another non-Latin script passed.
added_lines refuses a translation that breaks more lines than its source, and the gateway's _mangled carries the same check so such an answer falls through to the next engine instead of being cached. wrong_script refuses a non-Latin language when less than half of the letters it writes in another language's endonym script belong to its own; the script per language is derived from the endonym in meta/languages.yml, and letters are counted over runs of letters plus combining marks so Indic and vocalised Arabic words stay whole.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
meta/languages.yml carried `libretranslate: true|false` per entry, which
answered two different questions with one field: which languages the
repository translates, and which engine renders them. The first is the
table's business; the second is not. The gateway offers every backend
that serves a pair, falls through when one refuses or mangles the
answer, drops an engine whose failures pile up, and settles a request
class on the engine that wins its comparisons - and `signature()` is
"{source}:{target}:{length bucket}:{markup|plain}", so that learning is
already per target language. A static column beside it is a second
opinion that goes stale the moment the router learns something.
So the field is gone, 184 lines, and the table now holds name, endonym
and direction. Every language it names carries a catalog in every
domain.
What one engine can do is declared with that engine instead:
services.libretranslate-engine.serves, 47 codes. Not a detour - both
readers run before any engine answers. The lane prefetches one model per
code, and the readiness probe is itself the thing that would have asked.
`translatable()` split along that seam. Catalog-scoped callers take
`domain_languages()`, engine-scoped ones take the new
`served_languages()`: the CI matrix, extraction and translation the
first, the model prefetch and the tuning sweep the second. Collapsing
the two is what this change is guarding against - `baked_languages()`
renders identically before and after, verified by diffing both outputs.
`translate()` also stops dying on a language whose catalog has not been
extracted yet. It reads a missing catalog as nothing pending rather than
raising, because 136 docs catalogs do not exist until the next
extraction and refusing those languages would quietly shrink the set the
repository promises to translate.
The oracle suite follows: tests/oracle/test_translation_score.py asks
utils/i18n/judge.py about every language instead of routing a laya
checkpoint per language. It cannot run yet - the tools lane carries no
LiteLLM - and its skip message says exactly that.
test_catalogs_complete_on_gpu now measures against the table rather than
one engine's coverage. It is live on a GPU host and will report the 136
missing catalogs as the gap they are, which is the point: narrowing it
to the engine would hide exactly the languages worth watching.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5.5