mirror of
https://github.com/kevinveenbirkenbach/homepage.veen.world.git
synced 2026-08-24 05:04:33 +00:00
feat(i18n): offer every ISO 639-1 language
The table was thirty languages typed by hand. It is now generated: utils/generate_languages.py takes the 184 alpha-2 codes from pycountry, the display names from CLDR through babel, and the writing direction from CLDR character order. 159 languages carry their endonym; the remaining 25 have no CLDR entry and carry their English ISO name. That corrects the right-to-left set, which had four entries and needs ten — dv, ks, ps, sd, ug and yi were simply missed. Only 29 languages ship an interface catalogue, so the other 155 render in English until one is filled. make i18n-ui fills app/i18n/ui/ for them, and make i18n now covers the interface strings as well; neither asks for a string a shipped catalogue already answers, so hand-written entries stay. 184 entries do not fit on a screen, so the language menu scrolls inside itself. overscroll-behavior keeps the page behind it from moving once the list reaches its end. babel and pycountry are dev dependencies: the generator needs them, the application does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
9
Makefile
9
Makefile
@@ -144,6 +144,15 @@ i18n: env config
|
|||||||
--url "$$LIBRETRANSLATE_URL" \
|
--url "$$LIBRETRANSLATE_URL" \
|
||||||
--api-key "$$LIBRETRANSLATE_API_KEY"
|
--api-key "$$LIBRETRANSLATE_API_KEY"
|
||||||
|
|
||||||
|
.PHONY: i18n-ui
|
||||||
|
i18n-ui: env
|
||||||
|
# Fill missing interface translations in app/i18n/ui/ via LibreTranslate.
|
||||||
|
@$(call _require_env,LIBRETRANSLATE_URL); \
|
||||||
|
$(PYTHON) utils/i18n_sync.py \
|
||||||
|
--catalog ui \
|
||||||
|
--url "$$LIBRETRANSLATE_URL" \
|
||||||
|
--api-key "$$LIBRETRANSLATE_API_KEY"
|
||||||
|
|
||||||
.PHONY: lint-actions
|
.PHONY: lint-actions
|
||||||
lint-actions:
|
lint-actions:
|
||||||
# Lint GitHub Actions workflows.
|
# Lint GitHub Actions workflows.
|
||||||
|
|||||||
23
README.md
23
README.md
@@ -22,8 +22,8 @@ A lightweight, Docker-powered portfolio/landing-page generator—fully customiza
|
|||||||
Auto-cache assets for lightning-fast loading.
|
Auto-cache assets for lightning-fast loading.
|
||||||
- **Responsive Design**
|
- **Responsive Design**
|
||||||
Built on Bootstrap; looks great on desktop, tablet & mobile.
|
Built on Bootstrap; looks great on desktop, tablet & mobile.
|
||||||
- **30 Languages**
|
- **184 Languages**
|
||||||
Browser-negotiated, RTL-aware, with machine translation for your own content.
|
Every ISO 639-1 code, browser-negotiated, RTL-aware, with machine translation for your own content.
|
||||||
- **YAML-Driven**
|
- **YAML-Driven**
|
||||||
All content & structure defined in a simple `config.yaml`.
|
All content & structure defined in a simple `config.yaml`.
|
||||||
- **CLI Control**
|
- **CLI Control**
|
||||||
@@ -143,16 +143,19 @@ company:
|
|||||||
|
|
||||||
## 🌍 Languages
|
## 🌍 Languages
|
||||||
|
|
||||||
The interface ships in 30 languages. `/` serves the best match for the visitor's
|
Every ISO 639-1 language — all 184 two-letter codes — has a URL, a display
|
||||||
|
name in its own script and a writing direction. The interface ships translated
|
||||||
|
for 29 of them; the rest fall back to English string by string until a
|
||||||
|
catalogue is filled. `/` serves the best match for the visitor's
|
||||||
`Accept-Language` header, `/<code>/` forces one, and a switcher in the navbar
|
`Accept-Language` header, `/<code>/` forces one, and a switcher in the navbar
|
||||||
lists them all. Right-to-left languages (`ar`, `fa`, `he`, `ur`) get
|
lists them all. The ten right-to-left languages get `dir="rtl"` and Bootstrap's RTL
|
||||||
`dir="rtl"` and Bootstrap's RTL stylesheet automatically.
|
stylesheet automatically.
|
||||||
|
|
||||||
Translations live in two catalogues, both keyed by the English source string:
|
Translations live in two catalogues, both keyed by the English source string:
|
||||||
|
|
||||||
| Path | Tracked | Holds |
|
| Path | Tracked | Holds |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `app/i18n/ui/<code>.yaml` | yes | Interface strings, shipped complete for all 29 non-English languages. English is the source and has no file. |
|
| `app/i18n/ui/<code>.yaml` | yes | Interface strings. Shipped for 29 languages; English is the source and has no file. |
|
||||||
| `app/i18n/content/<code>.yaml` | no | Your `config.yaml` prose, generated per deployment. |
|
| `app/i18n/content/<code>.yaml` | no | Your `config.yaml` prose, generated per deployment. |
|
||||||
|
|
||||||
A string with no catalogue entry falls back to English, so a half-filled
|
A string with no catalogue entry falls back to English, so a half-filled
|
||||||
@@ -165,8 +168,10 @@ instance — set `LIBRETRANSLATE_URL` in `.env`, then:
|
|||||||
make i18n
|
make i18n
|
||||||
```
|
```
|
||||||
|
|
||||||
Existing entries are never overwritten, so corrections you make by hand survive
|
This fills the interface strings of the languages that ship no catalogue as
|
||||||
later runs. Only prose (`description`, `text`, `warning`, `info`, `subtitel`) is
|
well. Existing entries are never overwritten, and a string the shipped
|
||||||
|
catalogue already covers is never requested, so corrections you make by hand
|
||||||
|
survive later runs. Only prose (`description`, `text`, `warning`, `info`, `subtitel`) is
|
||||||
filled automatically; `name` and `title` are left to you, because a machine
|
filled automatically; `name` and `title` are left to you, because a machine
|
||||||
cannot tell the menu label "Pictures" from the brand "Mastodon". Write those
|
cannot tell the menu label "Pictures" from the brand "Mastodon". Write those
|
||||||
into the content catalogue yourself when you want them translated.
|
into the content catalogue yourself when you want them translated.
|
||||||
@@ -179,7 +184,7 @@ into the content catalogue yourself when you want them translated.
|
|||||||
* Secure with SSL/TLS.
|
* Secure with SSL/TLS.
|
||||||
* Swap to a production database if needed.
|
* Swap to a production database if needed.
|
||||||
|
|
||||||
Because every page carries a canonical URL and 30 `hreflang` alternates, two
|
Because every page carries a canonical URL and 184 `hreflang` alternates, two
|
||||||
details of the proxy setup now matter:
|
details of the proxy setup now matter:
|
||||||
|
|
||||||
* **Set `TRUSTED_HOSTS`** in `.env` to your public hostname(s), comma-separated.
|
* **Set `TRUSTED_HOSTS`** in `.env` to your public hostname(s), comma-separated.
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ describe('Language switcher', () => {
|
|||||||
cy.visit('/en/');
|
cy.visit('/en/');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('names the active language and offers all thirty', () => {
|
it('names the active language and offers every ISO 639-1 code', () => {
|
||||||
cy.get('#navbarDropdownLanguage')
|
cy.get('#navbarDropdownLanguage')
|
||||||
.should('have.attr', 'data-bs-toggle', 'dropdown')
|
.should('have.attr', 'data-bs-toggle', 'dropdown')
|
||||||
.and('contain.text', 'English');
|
.and('contain.text', 'English');
|
||||||
@@ -61,7 +61,7 @@ describe('Language switcher', () => {
|
|||||||
cy.get('#navbarDropdownLanguage')
|
cy.get('#navbarDropdownLanguage')
|
||||||
.parent('.nav-item')
|
.parent('.nav-item')
|
||||||
.find('> .dropdown-menu a.dropdown-item')
|
.find('> .dropdown-menu a.dropdown-item')
|
||||||
.should('have.length', 30);
|
.should('have.length', 184);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('marks the active language', () => {
|
it('marks the active language', () => {
|
||||||
@@ -200,6 +200,21 @@ describe('Translated interface details', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('scrolls inside the language menu instead of past the page', () => {
|
||||||
|
cy.viewport(1280, 720);
|
||||||
|
cy.visit('/en/');
|
||||||
|
cy.get('#navbarDropdownLanguage').click();
|
||||||
|
|
||||||
|
cy.get('.dropdown-menu.language-menu').should($menu => {
|
||||||
|
const menu = $menu[0];
|
||||||
|
expect(menu.scrollHeight, 'taller than it shows').to.be.greaterThan(
|
||||||
|
menu.clientHeight,
|
||||||
|
);
|
||||||
|
expect(menu.getBoundingClientRect().height).to.be.lessThan(720);
|
||||||
|
expect(getComputedStyle(menu).overflowY).to.eq('auto');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it('offers the switcher in the header only', () => {
|
it('offers the switcher in the header only', () => {
|
||||||
cy.viewport(1280, 720);
|
cy.viewport(1280, 720);
|
||||||
cy.visit('/en/');
|
cy.visit('/en/');
|
||||||
@@ -215,7 +230,7 @@ describe('Search engine metadata', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('declares an alternate for every language plus a default', () => {
|
it('declares an alternate for every language plus a default', () => {
|
||||||
cy.get('link[rel="alternate"][hreflang]').should('have.length', 31);
|
cy.get('link[rel="alternate"][hreflang]').should('have.length', 185);
|
||||||
cy.get('link[rel="alternate"][hreflang="x-default"]').should('exist');
|
cy.get('link[rel="alternate"][hreflang="x-default"]').should('exist');
|
||||||
cy.get('link[rel="alternate"][hreflang="ja"]')
|
cy.get('link[rel="alternate"][hreflang="ja"]')
|
||||||
.should('have.attr', 'href')
|
.should('have.attr', 'href')
|
||||||
|
|||||||
@@ -120,6 +120,12 @@ div#navbarNavfooter .navbar-nav {
|
|||||||
flex-wrap: nowrap;
|
flex-wrap: nowrap;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.dropdown-menu.language-menu {
|
||||||
|
max-height: 60vh;
|
||||||
|
overflow-y: auto;
|
||||||
|
overscroll-behavior: contain;
|
||||||
|
}
|
||||||
|
|
||||||
main, footer, header, nav {
|
main, footer, header, nav {
|
||||||
position: relative;
|
position: relative;
|
||||||
box-shadow:
|
box-shadow:
|
||||||
|
|||||||
@@ -103,7 +103,7 @@
|
|||||||
<a class="nav-link dropdown-toggle btn btn-light" id="navbarDropdownLanguage" role="button" data-bs-toggle="dropdown" data-bs-display="dynamic" aria-expanded="false" title="{{ t('Language') }}">
|
<a class="nav-link dropdown-toggle btn btn-light" id="navbarDropdownLanguage" role="button" data-bs-toggle="dropdown" data-bs-display="dynamic" aria-expanded="false" title="{{ t('Language') }}">
|
||||||
<i class="fa-solid fa-language"></i> {{ languages[lang] }}
|
<i class="fa-solid fa-language"></i> {{ languages[lang] }}
|
||||||
</a>
|
</a>
|
||||||
<ul class="dropdown-menu">
|
<ul class="dropdown-menu language-menu">
|
||||||
{% for code, endonym in languages.items() %}
|
{% for code, endonym in languages.items() %}
|
||||||
<li>
|
<li>
|
||||||
<a class="dropdown-item {% if code == lang %}active{% endif %}"
|
<a class="dropdown-item {% if code == lang %}active{% endif %}"
|
||||||
|
|||||||
@@ -11,46 +11,17 @@ from pathlib import Path
|
|||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
try:
|
||||||
|
from app.utils.languages import LANGUAGES, RTL_LANGUAGES
|
||||||
|
except ImportError: # pragma: no cover - supports running from the app/ directory.
|
||||||
|
from utils.languages import LANGUAGES, RTL_LANGUAGES
|
||||||
|
|
||||||
I18N_DIR = Path(__file__).resolve().parent.parent / "i18n"
|
I18N_DIR = Path(__file__).resolve().parent.parent / "i18n"
|
||||||
UI_DIR = I18N_DIR / "ui"
|
UI_DIR = I18N_DIR / "ui"
|
||||||
CONTENT_DIR = I18N_DIR / "content"
|
CONTENT_DIR = I18N_DIR / "content"
|
||||||
|
|
||||||
SOURCE_LANGUAGE = "en"
|
SOURCE_LANGUAGE = "en"
|
||||||
|
|
||||||
LANGUAGES = {
|
|
||||||
"en": "English",
|
|
||||||
"zh": "中文",
|
|
||||||
"hi": "हिन्दी",
|
|
||||||
"es": "Español",
|
|
||||||
"fr": "Français",
|
|
||||||
"ar": "العربية",
|
|
||||||
"bn": "বাংলা",
|
|
||||||
"pt": "Português",
|
|
||||||
"ru": "Русский",
|
|
||||||
"ur": "اردو",
|
|
||||||
"id": "Bahasa Indonesia",
|
|
||||||
"de": "Deutsch",
|
|
||||||
"ja": "日本語",
|
|
||||||
"tr": "Türkçe",
|
|
||||||
"ko": "한국어",
|
|
||||||
"vi": "Tiếng Việt",
|
|
||||||
"it": "Italiano",
|
|
||||||
"th": "ไทย",
|
|
||||||
"pl": "Polski",
|
|
||||||
"nl": "Nederlands",
|
|
||||||
"uk": "Українська",
|
|
||||||
"fa": "فارسی",
|
|
||||||
"ro": "Română",
|
|
||||||
"el": "Ελληνικά",
|
|
||||||
"cs": "Čeština",
|
|
||||||
"sv": "Svenska",
|
|
||||||
"hu": "Magyar",
|
|
||||||
"he": "עברית",
|
|
||||||
"da": "Dansk",
|
|
||||||
"fi": "Suomi",
|
|
||||||
}
|
|
||||||
|
|
||||||
RTL_LANGUAGES = frozenset({"ar", "fa", "he", "ur"})
|
|
||||||
|
|
||||||
AUTOFILL_KEYS = frozenset({"description", "text", "warning", "info", "subtitel"})
|
AUTOFILL_KEYS = frozenset({"description", "text", "warning", "info", "subtitel"})
|
||||||
|
|
||||||
|
|||||||
196
app/utils/languages.py
Normal file
196
app/utils/languages.py
Normal file
@@ -0,0 +1,196 @@
|
|||||||
|
"""ISO 639-1 languages, their display names and writing direction.
|
||||||
|
|
||||||
|
Generated by utils/generate_languages.py — edit that script, not this file.
|
||||||
|
|
||||||
|
Display names are CLDR endonyms where CLDR covers the language and the English
|
||||||
|
ISO 639-1 name for the 25 codes it does not.
|
||||||
|
"""
|
||||||
|
|
||||||
|
LANGUAGES = {
|
||||||
|
"en": "English",
|
||||||
|
"aa": "Qafar",
|
||||||
|
"ab": "Аԥсшәа",
|
||||||
|
"ae": "Avestan",
|
||||||
|
"af": "Afrikaans",
|
||||||
|
"ak": "Akan",
|
||||||
|
"am": "አማርኛ",
|
||||||
|
"an": "aragonés",
|
||||||
|
"ar": "العربية",
|
||||||
|
"as": "অসমীয়া",
|
||||||
|
"av": "Avaric",
|
||||||
|
"ay": "Aymara",
|
||||||
|
"az": "azərbaycan",
|
||||||
|
"ba": "башҡорт теле",
|
||||||
|
"be": "беларуская",
|
||||||
|
"bg": "български",
|
||||||
|
"bi": "Bislama",
|
||||||
|
"bm": "bamanakan",
|
||||||
|
"bn": "বাংলা",
|
||||||
|
"bo": "བོད་སྐད་",
|
||||||
|
"br": "brezhoneg",
|
||||||
|
"bs": "bosanski",
|
||||||
|
"ca": "català",
|
||||||
|
"ce": "нохчийн",
|
||||||
|
"ch": "Chamorro",
|
||||||
|
"co": "corsu",
|
||||||
|
"cr": "Cree",
|
||||||
|
"cs": "čeština",
|
||||||
|
"cu": "церковнослове́нскїй",
|
||||||
|
"cv": "чӑваш",
|
||||||
|
"cy": "Cymraeg",
|
||||||
|
"da": "dansk",
|
||||||
|
"de": "Deutsch",
|
||||||
|
"dv": "ދިވެހިބަސް",
|
||||||
|
"dz": "རྫོང་ཁ",
|
||||||
|
"ee": "eʋegbe",
|
||||||
|
"el": "Ελληνικά",
|
||||||
|
"eo": "Esperanto",
|
||||||
|
"es": "español",
|
||||||
|
"et": "eesti",
|
||||||
|
"eu": "euskara",
|
||||||
|
"fa": "فارسی",
|
||||||
|
"ff": "Pulaar",
|
||||||
|
"fi": "suomi",
|
||||||
|
"fj": "Fijian",
|
||||||
|
"fo": "føroyskt",
|
||||||
|
"fr": "français",
|
||||||
|
"fy": "Frysk",
|
||||||
|
"ga": "Gaeilge",
|
||||||
|
"gd": "Gàidhlig",
|
||||||
|
"gl": "galego",
|
||||||
|
"gn": "avañe’ẽ",
|
||||||
|
"gu": "ગુજરાતી",
|
||||||
|
"gv": "Gaelg",
|
||||||
|
"ha": "Hausa",
|
||||||
|
"he": "עברית",
|
||||||
|
"hi": "हिन्दी",
|
||||||
|
"ho": "Hiri Motu",
|
||||||
|
"hr": "hrvatski",
|
||||||
|
"ht": "Kreyòl Ayisyen",
|
||||||
|
"hu": "magyar",
|
||||||
|
"hy": "հայերեն",
|
||||||
|
"hz": "Herero",
|
||||||
|
"ia": "interlingua",
|
||||||
|
"id": "Indonesia",
|
||||||
|
"ie": "Interlingue",
|
||||||
|
"ig": "Igbo",
|
||||||
|
"ii": "ꆈꌠꉙ",
|
||||||
|
"ik": "Inupiaq",
|
||||||
|
"io": "Ido",
|
||||||
|
"is": "íslenska",
|
||||||
|
"it": "italiano",
|
||||||
|
"iu": "ᐃᓄᒃᑎᑐᑦ",
|
||||||
|
"ja": "日本語",
|
||||||
|
"jv": "Jawa",
|
||||||
|
"ka": "ქართული",
|
||||||
|
"kg": "Kongo",
|
||||||
|
"ki": "Gikuyu",
|
||||||
|
"kj": "Kuanyama",
|
||||||
|
"kk": "қазақ тілі",
|
||||||
|
"kl": "kalaallisut",
|
||||||
|
"km": "ខ្មែរ",
|
||||||
|
"kn": "ಕನ್ನಡ",
|
||||||
|
"ko": "한국어",
|
||||||
|
"kr": "Kanuri",
|
||||||
|
"ks": "کٲشُر",
|
||||||
|
"ku": "kurdî (kurmancî)",
|
||||||
|
"kv": "Komi",
|
||||||
|
"kw": "kernewek",
|
||||||
|
"ky": "кыргызча",
|
||||||
|
"la": "Latina",
|
||||||
|
"lb": "Lëtzebuergesch",
|
||||||
|
"lg": "Luganda",
|
||||||
|
"li": "Limburgan",
|
||||||
|
"ln": "lingála",
|
||||||
|
"lo": "ລາວ",
|
||||||
|
"lt": "lietuvių",
|
||||||
|
"lu": "Tshiluba",
|
||||||
|
"lv": "latviešu",
|
||||||
|
"mg": "Malagasy",
|
||||||
|
"mh": "Marshallese",
|
||||||
|
"mi": "Māori",
|
||||||
|
"mk": "македонски",
|
||||||
|
"ml": "മലയാളം",
|
||||||
|
"mn": "монгол",
|
||||||
|
"mr": "मराठी",
|
||||||
|
"ms": "Melayu",
|
||||||
|
"mt": "Malti",
|
||||||
|
"my": "မြန်မာ",
|
||||||
|
"na": "Nauru",
|
||||||
|
"nb": "norsk bokmål",
|
||||||
|
"nd": "isiNdebele",
|
||||||
|
"ne": "नेपाली",
|
||||||
|
"ng": "Ndonga",
|
||||||
|
"nl": "Nederlands",
|
||||||
|
"nn": "norsk nynorsk",
|
||||||
|
"no": "norsk",
|
||||||
|
"nr": "isiNdebele",
|
||||||
|
"nv": "Diné Bizaad",
|
||||||
|
"ny": "Nyanja",
|
||||||
|
"oc": "occitan",
|
||||||
|
"oj": "Ojibwa",
|
||||||
|
"om": "Oromoo",
|
||||||
|
"or": "ଓଡ଼ିଆ",
|
||||||
|
"os": "ирон",
|
||||||
|
"pa": "ਪੰਜਾਬੀ",
|
||||||
|
"pi": "Pali",
|
||||||
|
"pl": "polski",
|
||||||
|
"ps": "پښتو",
|
||||||
|
"pt": "português",
|
||||||
|
"qu": "Runasimi",
|
||||||
|
"rm": "rumantsch",
|
||||||
|
"rn": "Ikirundi",
|
||||||
|
"ro": "română",
|
||||||
|
"ru": "русский",
|
||||||
|
"rw": "Ikinyarwanda",
|
||||||
|
"sa": "संस्कृत भाषा",
|
||||||
|
"sc": "sardu",
|
||||||
|
"sd": "سنڌي",
|
||||||
|
"se": "davvisámegiella",
|
||||||
|
"sg": "Sängö",
|
||||||
|
"sh": "Serbo-Croatian",
|
||||||
|
"si": "සිංහල",
|
||||||
|
"sk": "slovenčina",
|
||||||
|
"sl": "slovenščina",
|
||||||
|
"sm": "Samoan",
|
||||||
|
"sn": "chiShona",
|
||||||
|
"so": "Soomaali",
|
||||||
|
"sq": "shqip",
|
||||||
|
"sr": "српски",
|
||||||
|
"ss": "siSwati",
|
||||||
|
"st": "Sesotho",
|
||||||
|
"su": "Basa Sunda",
|
||||||
|
"sv": "svenska",
|
||||||
|
"sw": "Kiswahili",
|
||||||
|
"ta": "தமிழ்",
|
||||||
|
"te": "తెలుగు",
|
||||||
|
"tg": "тоҷикӣ",
|
||||||
|
"th": "ไทย",
|
||||||
|
"ti": "ትግርኛ",
|
||||||
|
"tk": "türkmen dili",
|
||||||
|
"tl": "Tagalog",
|
||||||
|
"tn": "Setswana",
|
||||||
|
"to": "lea fakatonga",
|
||||||
|
"tr": "Türkçe",
|
||||||
|
"ts": "Xitsonga",
|
||||||
|
"tt": "татар",
|
||||||
|
"tw": "Twi",
|
||||||
|
"ty": "Tahitian",
|
||||||
|
"ug": "ئۇيغۇرچە",
|
||||||
|
"uk": "українська",
|
||||||
|
"ur": "اردو",
|
||||||
|
"uz": "o‘zbek",
|
||||||
|
"ve": "Tshivenḓa",
|
||||||
|
"vi": "Tiếng Việt",
|
||||||
|
"vo": "Volapük",
|
||||||
|
"wa": "walon",
|
||||||
|
"wo": "Wolof",
|
||||||
|
"xh": "IsiXhosa",
|
||||||
|
"yi": "ייִדיש",
|
||||||
|
"yo": "Èdè Yorùbá",
|
||||||
|
"za": "Vahcuengh",
|
||||||
|
"zh": "中文",
|
||||||
|
"zu": "isiZulu",
|
||||||
|
}
|
||||||
|
|
||||||
|
RTL_LANGUAGES = frozenset({"ar", "dv", "fa", "he", "ks", "ps", "sd", "ug", "ur", "yi"})
|
||||||
@@ -18,8 +18,10 @@ dependencies = [
|
|||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
dev = [
|
dev = [
|
||||||
|
"babel",
|
||||||
"bandit",
|
"bandit",
|
||||||
"pip-audit",
|
"pip-audit",
|
||||||
|
"pycountry",
|
||||||
"ruff",
|
"ruff",
|
||||||
"yamllint",
|
"yamllint",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -35,12 +35,12 @@ class TestNegotiate(unittest.TestCase):
|
|||||||
|
|
||||||
class TestDirection(unittest.TestCase):
|
class TestDirection(unittest.TestCase):
|
||||||
def test_every_right_to_left_language_is_marked(self):
|
def test_every_right_to_left_language_is_marked(self):
|
||||||
for code in ("ar", "fa", "he", "ur"):
|
for code in ("ar", "dv", "fa", "he", "ks", "ps", "sd", "ug", "ur", "yi"):
|
||||||
with self.subTest(code=code):
|
with self.subTest(code=code):
|
||||||
self.assertEqual(i18n.direction(code), "rtl")
|
self.assertEqual(i18n.direction(code), "rtl")
|
||||||
|
|
||||||
def test_other_languages_are_left_to_right(self):
|
def test_other_languages_are_left_to_right(self):
|
||||||
for code in set(i18n.LANGUAGES) - {"ar", "fa", "he", "ur"}:
|
for code in set(i18n.LANGUAGES) - set(i18n.RTL_LANGUAGES):
|
||||||
with self.subTest(code=code):
|
with self.subTest(code=code):
|
||||||
self.assertEqual(i18n.direction(code), "ltr")
|
self.assertEqual(i18n.direction(code), "ltr")
|
||||||
|
|
||||||
@@ -157,9 +157,17 @@ class TestCatalogMerge(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestShippedCatalogs(unittest.TestCase):
|
class TestShippedCatalogs(unittest.TestCase):
|
||||||
def test_thirty_languages_are_offered(self):
|
def test_every_iso_639_1_language_is_offered(self):
|
||||||
self.assertEqual(len(i18n.LANGUAGES), 30)
|
self.assertEqual(len(i18n.LANGUAGES), 184)
|
||||||
self.assertIn(i18n.SOURCE_LANGUAGE, i18n.LANGUAGES)
|
self.assertEqual(i18n.SOURCE_LANGUAGE, next(iter(i18n.LANGUAGES)))
|
||||||
|
self.assertGreaterEqual(
|
||||||
|
set(i18n.LANGUAGES), {"en", "de", "zh", "ar", "he", "vo", "za", "cu"}
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_every_display_name_is_filled_in(self):
|
||||||
|
blank = [code for code, name in i18n.LANGUAGES.items() if not name.strip()]
|
||||||
|
|
||||||
|
self.assertFalse(blank, f"No display name for: {blank}")
|
||||||
|
|
||||||
def test_every_code_is_usable_in_the_route_converter(self):
|
def test_every_code_is_usable_in_the_route_converter(self):
|
||||||
offenders = [
|
offenders = [
|
||||||
@@ -168,24 +176,21 @@ class TestShippedCatalogs(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertFalse(offenders, f"Not usable in the route converter: {offenders}")
|
self.assertFalse(offenders, f"Not usable in the route converter: {offenders}")
|
||||||
|
|
||||||
def test_every_language_but_the_source_ships_a_ui_catalog(self):
|
def test_every_shipped_catalog_names_a_known_language(self):
|
||||||
missing = [
|
stray = [
|
||||||
code
|
path.stem
|
||||||
for code in i18n.LANGUAGES
|
for path in i18n.UI_DIR.glob("*.yaml")
|
||||||
if code != i18n.SOURCE_LANGUAGE
|
if path.stem not in i18n.LANGUAGES or path.stem == i18n.SOURCE_LANGUAGE
|
||||||
and not (i18n.UI_DIR / f"{code}.yaml").is_file()
|
|
||||||
]
|
]
|
||||||
|
|
||||||
self.assertFalse(missing, f"No UI catalogue for: {missing}")
|
self.assertFalse(stray, f"Catalogue for an unknown language: {stray}")
|
||||||
|
|
||||||
def test_ui_catalogs_cover_exactly_the_interface_strings(self):
|
def test_ui_catalogs_cover_exactly_the_interface_strings(self):
|
||||||
expected = set(i18n.UI_STRINGS)
|
expected = set(i18n.UI_STRINGS)
|
||||||
mismatched = {}
|
mismatched = {}
|
||||||
|
|
||||||
for code in i18n.LANGUAGES:
|
for path in sorted(i18n.UI_DIR.glob("*.yaml")):
|
||||||
if code == i18n.SOURCE_LANGUAGE:
|
code = path.stem
|
||||||
continue
|
|
||||||
path = i18n.UI_DIR / f"{code}.yaml"
|
|
||||||
entries = yaml.safe_load(path.read_text(encoding="utf-8"))
|
entries = yaml.safe_load(path.read_text(encoding="utf-8"))
|
||||||
if set(entries) != expected:
|
if set(entries) != expected:
|
||||||
mismatched[code] = {
|
mismatched[code] = {
|
||||||
|
|||||||
@@ -224,7 +224,7 @@ class TestSync(unittest.TestCase):
|
|||||||
session = Mock(get=Mock(return_value=listing), post=Mock(return_value=answer))
|
session = Mock(get=Mock(return_value=listing), post=Mock(return_value=answer))
|
||||||
|
|
||||||
with patch.object(i18n_sync.requests, "Session", return_value=session):
|
with patch.object(i18n_sync.requests, "Session", return_value=session):
|
||||||
i18n_sync.sync("http://lt", "", set(sources), ["de"])
|
i18n_sync.sync("http://lt", "", set(sources), ["de"], self.directory)
|
||||||
return session
|
return session
|
||||||
|
|
||||||
def test_missing_entries_are_written(self):
|
def test_missing_entries_are_written(self):
|
||||||
@@ -267,13 +267,26 @@ class TestSync(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertEqual(self.path.read_text(encoding="utf-8"), original)
|
self.assertEqual(self.path.read_text(encoding="utf-8"), original)
|
||||||
|
|
||||||
def test_the_content_directory_is_created(self):
|
def test_a_string_the_shipped_catalog_covers_is_not_requested(self):
|
||||||
|
ui = Path(tempfile.mkdtemp())
|
||||||
|
self.addCleanup(shutil.rmtree, ui, True)
|
||||||
|
self.addCleanup(setattr, i18n, "UI_DIR", i18n.UI_DIR)
|
||||||
|
i18n.UI_DIR = ui
|
||||||
|
(ui / "de.yaml").write_text("Close: Schliessen\n", encoding="utf-8")
|
||||||
|
|
||||||
|
session = self._run(["Close", "Hello"])
|
||||||
|
|
||||||
|
self.assertEqual(session.post.call_count, 1)
|
||||||
|
self.assertNotIn("Close", yaml.safe_load(self.path.read_text(encoding="utf-8")))
|
||||||
|
|
||||||
|
def test_the_catalog_directory_is_created(self):
|
||||||
nested = self.directory / "content"
|
nested = self.directory / "content"
|
||||||
i18n.CONTENT_DIR = nested
|
self.directory = nested
|
||||||
|
self.path = nested / "de.yaml"
|
||||||
|
|
||||||
self._run(["Hello"])
|
self._run(["Hello"])
|
||||||
|
|
||||||
self.assertTrue((nested / "de.yaml").is_file())
|
self.assertTrue(self.path.is_file())
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
83
utils/generate_languages.py
Normal file
83
utils/generate_languages.py
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Regenerate app/utils/languages.py from ISO 639-1 and CLDR.
|
||||||
|
|
||||||
|
Every ISO 639-1 alpha-2 code becomes an entry. The display name is the CLDR
|
||||||
|
endonym where CLDR knows the language, and the English ISO name for the
|
||||||
|
remainder. Writing direction comes from CLDR's character order.
|
||||||
|
|
||||||
|
Needs the dev extra: pip install -e ".[dev]"
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pycountry
|
||||||
|
from babel import Locale
|
||||||
|
from babel.localedata import locale_identifiers
|
||||||
|
|
||||||
|
TARGET = Path(__file__).resolve().parents[1] / "app" / "utils" / "languages.py"
|
||||||
|
|
||||||
|
HEADER = '''"""ISO 639-1 languages, their display names and writing direction.
|
||||||
|
|
||||||
|
Generated by utils/generate_languages.py — edit that script, not this file.
|
||||||
|
|
||||||
|
Display names are CLDR endonyms where CLDR covers the language and the English
|
||||||
|
ISO 639-1 name for the {fallback} codes it does not.
|
||||||
|
"""
|
||||||
|
|
||||||
|
LANGUAGES = {{
|
||||||
|
'''
|
||||||
|
|
||||||
|
FOOTER = """}}
|
||||||
|
|
||||||
|
RTL_LANGUAGES = frozenset({rtl})
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def collect():
|
||||||
|
"""Return (code -> display name) and the set of right-to-left codes."""
|
||||||
|
cldr = {code for code in locale_identifiers() if len(code) == 2 and code.isalpha()}
|
||||||
|
names, rtl, fallbacks = {}, set(), 0
|
||||||
|
|
||||||
|
for language in sorted(
|
||||||
|
pycountry.languages, key=lambda item: getattr(item, "alpha_2", "")
|
||||||
|
):
|
||||||
|
code = getattr(language, "alpha_2", None)
|
||||||
|
if not code:
|
||||||
|
continue
|
||||||
|
if code in cldr:
|
||||||
|
locale = Locale.parse(code)
|
||||||
|
names[code] = locale.get_display_name(code)
|
||||||
|
if locale.character_order == "right-to-left":
|
||||||
|
rtl.add(code)
|
||||||
|
else:
|
||||||
|
names[code] = language.name
|
||||||
|
fallbacks += 1
|
||||||
|
|
||||||
|
return names, rtl, fallbacks
|
||||||
|
|
||||||
|
|
||||||
|
def render(names, rtl, fallbacks):
|
||||||
|
"""Return the module source, English first and the rest by code."""
|
||||||
|
ordered = ["en"] + [code for code in sorted(names) if code != "en"]
|
||||||
|
body = "".join(f' "{code}": "{names[code]}",\n' for code in ordered)
|
||||||
|
listed = ", ".join(f'"{code}"' for code in sorted(rtl))
|
||||||
|
return (
|
||||||
|
HEADER.format(fallback=fallbacks) + body + FOOTER.format(rtl="{" + listed + "}")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
names, rtl, fallbacks = collect()
|
||||||
|
TARGET.write_text(render(names, rtl, fallbacks), encoding="utf-8")
|
||||||
|
subprocess.run([sys.executable, "-m", "ruff", "format", str(TARGET)], check=False)
|
||||||
|
print(
|
||||||
|
f"{len(names)} languages, {len(rtl)} right-to-left, "
|
||||||
|
f"{fallbacks} without a CLDR endonym"
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -144,9 +144,17 @@ def translate(session, url, api_key, text, target):
|
|||||||
return translated
|
return translated
|
||||||
|
|
||||||
|
|
||||||
def sync(url, api_key, sources, targets):
|
def sync(url, api_key, sources, targets, directory):
|
||||||
"""Fill and write the content catalogue of every language in ``targets``."""
|
"""Fill and write the catalogue of every language in ``targets``.
|
||||||
i18n.CONTENT_DIR.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
Args:
|
||||||
|
url: base URL of the LibreTranslate instance.
|
||||||
|
api_key: API key, or an empty string.
|
||||||
|
sources: English strings to translate.
|
||||||
|
targets: language codes to fill.
|
||||||
|
directory: catalogue directory to write into.
|
||||||
|
"""
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
session = requests.Session()
|
session = requests.Session()
|
||||||
|
|
||||||
available = supported_languages(url, session)
|
available = supported_languages(url, session)
|
||||||
@@ -155,12 +163,21 @@ def sync(url, api_key, sources, targets):
|
|||||||
print(f"- {target}: not offered by {url}, skipped")
|
print(f"- {target}: not offered by {url}, skipped")
|
||||||
continue
|
continue
|
||||||
|
|
||||||
path = i18n.CONTENT_DIR / f"{target}.yaml"
|
path = directory / f"{target}.yaml"
|
||||||
catalog = load_existing(path)
|
catalog = load_existing(path)
|
||||||
if catalog is None:
|
if catalog is None:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
missing = sorted(source for source in sources if source not in catalog)
|
shipped = (
|
||||||
|
{}
|
||||||
|
if directory == i18n.UI_DIR
|
||||||
|
else i18n.read_catalog(i18n.UI_DIR / f"{target}.yaml")
|
||||||
|
)
|
||||||
|
missing = sorted(
|
||||||
|
source
|
||||||
|
for source in sources
|
||||||
|
if source not in catalog and source not in shipped
|
||||||
|
)
|
||||||
if not missing:
|
if not missing:
|
||||||
print(f"- {target}: complete")
|
print(f"- {target}: complete")
|
||||||
continue
|
continue
|
||||||
@@ -201,6 +218,15 @@ def main(argv=None):
|
|||||||
default=DEFAULT_CONFIG_PATH,
|
default=DEFAULT_CONFIG_PATH,
|
||||||
help=f"Configuration to read the prose from (default: {DEFAULT_CONFIG_PATH}).",
|
help=f"Configuration to read the prose from (default: {DEFAULT_CONFIG_PATH}).",
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--catalog",
|
||||||
|
choices=("content", "ui"),
|
||||||
|
default="content",
|
||||||
|
help=(
|
||||||
|
"content: your configuration's prose, generated per deployment. "
|
||||||
|
"ui: the interface strings that ship with the project."
|
||||||
|
),
|
||||||
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--languages",
|
"--languages",
|
||||||
nargs="+",
|
nargs="+",
|
||||||
@@ -209,12 +235,18 @@ def main(argv=None):
|
|||||||
)
|
)
|
||||||
args = parser.parse_args(argv)
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
if args.catalog == "ui":
|
||||||
|
directory = i18n.UI_DIR
|
||||||
|
sources = set(i18n.UI_STRINGS)
|
||||||
|
print(f"{len(sources)} interface string(s)")
|
||||||
|
else:
|
||||||
|
directory = i18n.CONTENT_DIR
|
||||||
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))
|
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))
|
||||||
sources = collect_sources(config)
|
sources = collect_sources(config) | set(i18n.UI_STRINGS)
|
||||||
print(f"{len(sources)} translatable string(s) in {args.config}")
|
print(f"{len(sources)} string(s) in {args.config} and the interface")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
sync(args.url.rstrip("/"), args.api_key, sources, args.languages)
|
sync(args.url.rstrip("/"), args.api_key, sources, args.languages, directory)
|
||||||
except BackendError as error:
|
except BackendError as error:
|
||||||
print(f"ERROR: {error}", file=sys.stderr)
|
print(f"ERROR: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
|
|||||||
Reference in New Issue
Block a user