Files
computer-playbook/utils/i18n/keyed.py
T
kevinveenbirkenbachandClaude Opus 5 4c98624e07 feat(i18n): gettext catalogs with LibreTranslate machine translation
Core and the documentation get gettext catalogs for all 184 ISO 639-1
languages listed in meta/languages.yml, 47 of which LibreTranslate can fill.

`make i18n-extract` merges the current sources into locale/<code>/
LC_MESSAGES/{core,docs}.po: role descriptions, category and menu titles and
the logout strings for the core domain, the Sphinx gettext output for the
docs domain. A changed source keeps its previous translation as a fuzzy
entry, and catalogs are written deterministically so an unchanged merge
leaves the files untouched.

`make i18n-translate domain=<core|docs> [languages=<codes>]` fills only the
empty and fuzzy entries with a short-lived LibreTranslate container (the
image web-svc-libretranslate pins) and removes it afterwards. It starts the
container, and loads models, only for languages that still have pending
entries. Code spans, placeholders, URLs, tags and dotted names such as
Infinito.Nexus are masked for the request and a translation that damages
one is discarded. An unreachable server stops the run instead of
discarding every remaining entry, so a rerun continues where it stopped;
machine translations carry the translator comment `libretranslate`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 18:22:03 +02:00

74 lines
2.5 KiB
Python

"""Catalogues keyed by name, as browser code reads them, built from the ``core`` domain."""
from __future__ import annotations
from typing import TYPE_CHECKING
from utils.i18n.catalog import catalog_path, read_catalog, translations
from utils.i18n.languages import SOURCE_LANGUAGE, domain_languages, load_languages
if TYPE_CHECKING:
from collections.abc import Callable
from pathlib import Path
def keyed_catalogue(
root: Path, context: Callable[[str], str], source: dict[str, str]
) -> dict[str, dict[str, str]]:
"""Return ``source`` in English and every language that translates any of it.
Args:
root: repository root.
context: maps a key to the ``msgctxt`` of its message.
source: key mapped to its English text.
Returns:
ISO 639-1 code mapped to ``{key: text, "dir": direction}``; a key a
language leaves untranslated keeps its English text.
"""
languages = load_languages(root)
catalogue = {
SOURCE_LANGUAGE: {**source, "dir": languages[SOURCE_LANGUAGE]["direction"]}
}
for code in domain_languages(languages, "core"):
path = catalog_path(root, code, "core")
if not path.is_file():
continue
found = translations(read_catalog(path))
texts = {key: found.get((context(key), text)) for key, text in source.items()}
if any(texts.values()):
catalogue[code] = {
**{key: texts[key] or source[key] for key in source},
"dir": languages[code]["direction"],
}
return catalogue
def source_keyed_catalogues(
root: Path, prefixes: tuple[str, ...]
) -> dict[str, dict[str, str]]:
"""Return catalogues keyed by the English source text, as port-ui reads them.
Args:
root: repository root.
prefixes: ``msgctxt`` prefixes of the messages to include, e.g. ``role:``.
Returns:
ISO 639-1 code mapped to ``{english: translation}`` for every language
that translates at least one included message.
"""
catalogues = {}
for code in domain_languages(load_languages(root), "core"):
path = catalog_path(root, code, "core")
if not path.is_file():
continue
entries: dict[str, str] = {}
for (context, msgid), msgstr in sorted(
translations(read_catalog(path)).items()
):
if context and context.startswith(prefixes):
entries.setdefault(msgid, msgstr)
if entries:
catalogues[code] = entries
return catalogues