feat(i18n): serve every page in 30 languages

The interface ships translated; page content stays English until a
LibreTranslate instance fills app/i18n/content/ through make i18n. A string
without a catalogue entry falls back to its English source, so a half-filled
catalogue degrades instead of breaking.

Translation runs after ConfigurationResolver.resolve_links(), on a copy.
resolve_links matches by the `name` field, so translating it beforehand
would break every `link:` reference in the configuration.

negotiate() normalises to the primary subtag itself. Werkzeug's best_match
returns an exact match before it considers a primary-tag fallback, so the
Chrome default `de-DE,en;q=0.8` resolves to English there. "/" carries
Vary: Accept-Language, without which a shared cache pins the first
visitor's language for everyone.

The route rule lists the known codes as a converter argument. A bare
"/<lang>/" answers /robots.txt and /favicon.ico with a permanently
cacheable 308 to their trailing-slash form.

Templates gain lang, dir, the RTL stylesheet, a canonical URL and 30
hreflang alternates. Those are the first external URLs in this app:
ProxyFix takes the scheme from X-Forwarded-Proto so they do not claim
http:// behind a TLS-terminating proxy, X-Forwarded-Host stays untrusted
because nginx passes a client-supplied one through, and TRUSTED_HOSTS lets
Flask reject a forged Host outright.

Flask only autoescapes .html/.htm/.xml/.xhtml/.svg, so every *.html.j2
template interpolated configuration raw. Enabling it changes two lines of
the shipped page, both an apostrophe.

read_catalog degrades an unreadable catalogue to English rather than
serving a 500, and drops non-string entries that would otherwise render as
"42". i18n_sync writes atomically, never overwrites an existing entry,
refuses to touch a catalogue it could not parse, and leaves the file alone
when a run translated nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-22 00:02:35 +02:00
parent a42930a699
commit 2a35b2910a
46 changed files with 1915 additions and 37 deletions

1
.gitignore vendored
View File

@@ -1,4 +1,5 @@
app/config.yaml app/config.yaml
app/i18n/content/
*__pycache__* *__pycache__*
app/static/cache/* app/static/cache/*
.env .env

View File

@@ -22,6 +22,8 @@ A lightweight, Docker-powered portfolio/landing-page generator—fully customiza
Auto-cache assets for lightning-fast loading. Auto-cache assets for lightning-fast loading.
- **Responsive Design** - **Responsive Design**
Built on Bootstrap; looks great on desktop, tablet & mobile. Built on Bootstrap; looks great on desktop, tablet & mobile.
- **30 Languages**
Browser-negotiated, RTL-aware, with machine translation for your own content.
- **YAML-Driven** - **YAML-Driven**
All content & structure defined in a simple `config.yaml`. All content & structure defined in a simple `config.yaml`.
- **CLI Control** - **CLI Control**
@@ -139,12 +141,55 @@ company:
--- ---
## 🌍 Languages
The interface ships in 30 languages. `/` serves the best match for the visitor's
`Accept-Language` header, `/<code>/` forces one, and a switcher in the navbar
lists them all. Right-to-left languages (`ar`, `fa`, `he`, `ur`) get
`dir="rtl"` and Bootstrap's RTL stylesheet automatically.
Translations live in two catalogues, both keyed by the English source string:
| Path | Tracked | Holds |
| --- | --- | --- |
| `app/i18n/ui/<code>.yaml` | yes | Interface strings, shipped complete for all 29 non-English languages. English is the source and has no file. |
| `app/i18n/content/<code>.yaml` | no | Your `config.yaml` prose, generated per deployment. |
A string with no catalogue entry falls back to English, so a half-filled
catalogue degrades instead of breaking.
Fill the content catalogues from a [LibreTranslate](https://libretranslate.com/)
instance — set `LIBRETRANSLATE_URL` in `.env`, then:
```bash
make i18n
```
Existing entries are never overwritten, so corrections you make by hand survive
later runs. Only prose (`description`, `text`, `warning`, `info`, `subtitel`) is
filled automatically; `name` and `title` are left to you, because a machine
cannot tell the menu label "Pictures" from the brand "Mastodon". Write those
into the content catalogue yourself when you want them translated.
---
## 🚢 Production Deployment ## 🚢 Production Deployment
* Use a reverse proxy (NGINX/Apache). * Use a reverse proxy (NGINX/Apache).
* Secure with SSL/TLS. * Secure with SSL/TLS.
* Swap to a production database if needed. * Swap to a production database if needed.
Because every page carries a canonical URL and 30 `hreflang` alternates, two
details of the proxy setup now matter:
* **Set `TRUSTED_HOSTS`** in `.env` to your public hostname(s), comma-separated.
Left empty, the app reflects whatever `Host` header arrives into its canonical,
`hreflang` and redirect URLs — so a shared cache in front of it can be made to
store a redirect pointing somewhere else.
* **Have the proxy send `X-Forwarded-Proto`.** Without it the app cannot know TLS
terminated upstream and every canonical URL claims `http://`. `X-Forwarded-Host`
is deliberately *not* trusted; set `Host` to the public name instead.
--- ---
## 📜 License ## 📜 License

View File

@@ -3,10 +3,12 @@ import os
import requests import requests
import yaml import yaml
from flask import Flask, current_app, render_template, url_for from flask import Flask, current_app, make_response, render_template, request, url_for
from markupsafe import Markup from markupsafe import Markup
from werkzeug.middleware.proxy_fix import ProxyFix
try: try:
from app.utils import i18n
from app.utils.asset_resolver import asset_src, resolve_asset_cache from app.utils.asset_resolver import asset_src, resolve_asset_cache
from app.utils.cache_manager import CacheManager from app.utils.cache_manager import CacheManager
from app.utils.compute_card_classes import compute_card_classes from app.utils.compute_card_classes import compute_card_classes
@@ -17,6 +19,10 @@ except ImportError: # pragma: no cover - supports running from the app/ directo
from utils.compute_card_classes import compute_card_classes from utils.compute_card_classes import compute_card_classes
from utils.configuration_resolver import ConfigurationResolver from utils.configuration_resolver import ConfigurationResolver
from utils import i18n
TRANSLATED_SECTIONS = ("cards", "company", "navigation", "platform")
logging.basicConfig(level=logging.DEBUG) logging.basicConfig(level=logging.DEBUG)
FLASK_ENV = os.getenv("FLASK_ENV", "production") FLASK_ENV = os.getenv("FLASK_ENV", "production")
@@ -42,6 +48,8 @@ def load_config(app):
resolver = ConfigurationResolver(config) resolver = ConfigurationResolver(config)
resolver.resolve_links() resolver.resolve_links()
app.config.update(resolver.get_config()) app.config.update(resolver.get_config())
app.config["TRANSLATED_CONFIG"] = {}
i18n.clear_catalogs()
def cache_icons_and_logos(app): def cache_icons_and_logos(app):
@@ -60,6 +68,23 @@ def cache_icons_and_logos(app):
# Initialize Flask app # Initialize Flask app
app = Flask(__name__) app = Flask(__name__)
app.jinja_options = {**app.jinja_options, "autoescape": True}
app.wsgi_app = ProxyFix(app.wsgi_app, x_proto=1)
def trusted_hosts(raw):
"""Parse a comma-separated host list, or None when nothing is configured.
Args:
raw: the ``TRUSTED_HOSTS`` value, possibly empty.
"""
hosts = [host.strip() for host in raw.split(",") if host.strip()]
return hosts or None
app.config["TRUSTED_HOSTS"] = trusted_hosts(os.getenv("TRUSTED_HOSTS", ""))
# Load configuration and cache assets on startup # Load configuration and cache assets on startup
load_config(app) load_config(app)
cache_icons_and_logos(app) cache_icons_and_logos(app)
@@ -91,36 +116,79 @@ def reload_config_in_dev():
cache_icons_and_logos(app) cache_icons_and_logos(app)
@app.route("/") def translated_config(lang):
def index(): """Return the configuration sections translated into ``lang``, memoized.
"""Render the main index page."""
cards = app.config["cards"] The memo is dropped by ``load_config``, so a development reload picks up
lg_classes, md_classes = compute_card_classes(cards) edited content on the next request.
apod_bg = None """
memo = app.config["TRANSLATED_CONFIG"]
if lang not in memo:
source = {section: app.config[section] for section in TRANSLATED_SECTIONS}
memo[lang] = i18n.translate_tree(source, lang)
return memo[lang]
def apod_background():
"""Return today's NASA APOD image URL, or None when unavailable."""
api_key = app.config.get("NASA_API_KEY") api_key = app.config.get("NASA_API_KEY")
if api_key: if not api_key:
return None
try:
resp = requests.get( resp = requests.get(
"https://api.nasa.gov/planetary/apod", "https://api.nasa.gov/planetary/apod",
params={"api_key": api_key}, params={"api_key": api_key},
timeout=10, timeout=10,
) )
if resp.ok: except requests.RequestException:
logging.warning("APOD lookup failed", exc_info=True)
return None
if not resp.ok:
return None
data = resp.json() data = resp.json()
if data.get("media_type") == "image": return data.get("url") if data.get("media_type") == "image" else None
apod_bg = data.get("url")
def render_index(lang):
"""Render the index page in ``lang``."""
config = translated_config(lang)
cards = config["cards"]
lg_classes, md_classes = compute_card_classes(cards)
return render_template( return render_template(
"pages/index.html.j2", "pages/index.html.j2",
cards=cards, cards=cards,
company=app.config["company"], company=config["company"],
navigation=app.config["navigation"], navigation=config["navigation"],
platform=app.config["platform"], platform=config["platform"],
lg_classes=lg_classes, lg_classes=lg_classes,
md_classes=md_classes, md_classes=md_classes,
apod_bg=apod_bg, apod_bg=apod_background(),
lang=lang,
lang_dir=i18n.direction(lang),
languages=i18n.LANGUAGES,
ui_strings=i18n.ui_strings(lang),
t=lambda source: i18n.catalog(lang).get(source, source),
) )
@app.route("/")
def index():
"""Render the index page in the language the browser asks for."""
response = make_response(render_index(i18n.negotiate(request.accept_languages)))
response.headers["Vary"] = "Accept-Language"
return response
@app.route(f"/<any({','.join(i18n.LANGUAGES)}):lang>/")
def localized_index(lang):
"""Render the index page in an explicitly requested language."""
return render_index(lang)
if __name__ == "__main__": if __name__ == "__main__":
app.run( app.run(
debug=(FLASK_ENV == "development"), debug=(FLASK_ENV == "development"),

View File

@@ -0,0 +1,228 @@
// cypress/e2e/i18n.spec.js
const GERMAN_BROWSER = { headers: { 'Accept-Language': 'de-DE,de;q=0.9,en;q=0.8' } };
describe('Language negotiation', () => {
it('serves English to an English browser', () => {
cy.visit('/', { headers: { 'Accept-Language': 'en-US,en;q=0.9' } });
cy.get('html').should('have.attr', 'lang', 'en');
cy.get('footer.footer a.iframe-link').should('contain.text', 'Imprint');
});
it('serves German to a German browser', () => {
cy.visit('/', GERMAN_BROWSER);
cy.get('html').should('have.attr', 'lang', 'de');
cy.get('footer.footer a.iframe-link').should('contain.text', 'Impressum');
});
it('falls back to English for an unsupported browser language', () => {
cy.visit('/', { headers: { 'Accept-Language': 'xx-XX' } });
cy.get('html').should('have.attr', 'lang', 'en');
});
it('lets the URL prefix override the browser language', () => {
cy.visit('/fr/', GERMAN_BROWSER);
cy.get('html').should('have.attr', 'lang', 'fr');
cy.get('footer.footer a.iframe-link').should('contain.text', 'Mentions légales');
});
it('rejects an unsupported language code', () => {
cy.request({ url: '/xx/', failOnStatusCode: false })
.its('status')
.should('eq', 404);
});
it('does not turn unrelated single-segment paths into redirects', () => {
['/robots.txt', '/favicon.ico', '/sitemap.xml'].forEach(path => {
cy.request({ url: path, failOnStatusCode: false, followRedirect: false })
.its('status')
.should('eq', 404);
});
});
it('marks the negotiated route as varying by language', () => {
cy.request('/')
.its('headers.vary')
.should('contain', 'Accept-Language');
});
});
describe('Language switcher', () => {
beforeEach(() => {
cy.viewport(1280, 720);
cy.visit('/en/');
});
it('names the active language and offers all thirty', () => {
cy.get('#navbarDropdownLanguage')
.should('have.attr', 'data-bs-toggle', 'dropdown')
.and('contain.text', 'English');
cy.get('#navbarDropdownLanguage')
.parent('.nav-item')
.find('> .dropdown-menu a.dropdown-item')
.should('have.length', 30);
});
it('marks the active language', () => {
cy.get('#navbarDropdownLanguage').click();
cy.get('.dropdown-menu a.dropdown-item.active[hreflang="en"]').should('exist');
});
it('navigates to the chosen language', () => {
cy.get('#navbarDropdownLanguage').click();
cy.get('.dropdown-menu a.dropdown-item[hreflang="de"]')
.should('have.text', 'Deutsch')
.click();
cy.url().should('match', /\/de\/$/);
cy.get('html').should('have.attr', 'lang', 'de');
});
});
describe('Translated interface strings', () => {
it('translates the strings rendered by the templates', () => {
cy.visit('/de/');
cy.get('#dynamicCopyButton').should('have.text', 'Kopieren');
cy.get('#dynamicChildrenSection h6').should('have.text', 'Optionen:');
cy.get('#dynamicAlternativesSection h6').should('have.text', 'Alternativen:');
cy.get('.modal-footer button').should('have.text', 'Schließen');
});
it('exposes the catalogue to client-side code', () => {
cy.visit('/de/');
cy.window().its('I18N').should('deep.include', {
Open: 'Öffnen',
'Open Link': 'Link öffnen',
'Identifier copied to clipboard!': 'Kennung in die Zwischenablage kopiert!',
});
});
it('leaves the catalogue untranslated in the source language', () => {
cy.visit('/en/');
cy.window().its('I18N').should('deep.include', { Open: 'Open' });
});
});
describe('Strings translated by modal.js', () => {
const item = {
name: 'Test Item',
identifier: 'ABC123',
icon: { class: 'fa fa-test' },
alternatives: [
{ name: 'Alt One', identifier: 'ALT1', icon: { class: 'fa fa-alt1' } },
],
};
beforeEach(() => {
cy.visit('/de/');
cy.window().then(win => {
cy.stub(win.navigator.clipboard, 'writeText').resolves();
cy.stub(win, 'alert');
});
});
it('translates the button of a list entry', () => {
cy.window().invoke('openDynamicPopup', item);
cy.get('#dynamicAlternativesList button').should('have.text', 'Öffnen');
});
it('translates the link label when the entry has no description', () => {
cy.window().invoke('openDynamicPopup', {
...item,
url: 'https://example.com',
description: null,
});
cy.get('#dynamicModalLinkHref').should('have.text', 'Link öffnen');
});
it('translates the clipboard confirmation', () => {
cy.window().invoke('openDynamicPopup', item);
cy.get('#dynamicCopyButton').click();
cy.window()
.its('alert')
.should('have.been.calledWith', 'Kennung in die Zwischenablage kopiert!');
});
});
describe('Right-to-left languages', () => {
it('flips the document and loads the RTL stylesheet', () => {
cy.visit('/ar/');
cy.get('html').should('have.attr', 'dir', 'rtl');
cy.get('link[href*="bootstrap.rtl.min.css"]').should('exist');
cy.get('link[href*="vendor/bootstrap/css/bootstrap.min.css"]').should('not.exist');
cy.get('body').should('have.css', 'direction', 'rtl');
});
it('keeps left-to-right languages on the default stylesheet', () => {
cy.visit('/en/');
cy.get('html').should('have.attr', 'dir', 'ltr');
cy.get('link[href*="bootstrap.rtl.min.css"]').should('not.exist');
});
it('actually serves the RTL stylesheet it links to', () => {
cy.visit('/ar/');
cy.get('link[href*="bootstrap.rtl.min.css"]')
.should('have.attr', 'href')
.then(href => {
cy.request(href).its('status').should('eq', 200);
});
});
});
describe('Translated interface details', () => {
it('translates the strings only a screen reader sees', () => {
cy.visit('/de/');
cy.get('#dynamicModal .btn-close').should('have.attr', 'aria-label', 'Schließen');
});
it('translates the alert headings', () => {
cy.visit('/fr/');
cy.get('#dynamicModalWarning h5').should('contain.text', 'Avertissement');
cy.get('#dynamicModalInfo h5').should('contain.text', 'Informations');
});
it('translates the language switcher tooltip', () => {
cy.visit('/de/');
cy.get('#navbarDropdownLanguage').should('have.attr', 'title', 'Sprache');
});
it('tags each switcher entry with its own language', () => {
cy.visit('/en/');
cy.get('.dropdown-menu a.dropdown-item[hreflang="ja"]').should(
'have.attr',
'lang',
'ja',
);
});
it('offers the switcher in the header only', () => {
cy.viewport(1280, 720);
cy.visit('/en/');
cy.get('#navbarNavheader #navbarDropdownLanguage').should('exist');
cy.get('#navbarNavfooter #navbarDropdownLanguage').should('not.exist');
});
});
describe('Search engine metadata', () => {
beforeEach(() => {
cy.visit('/de/');
});
it('declares an alternate for every language plus a default', () => {
cy.get('link[rel="alternate"][hreflang]').should('have.length', 31);
cy.get('link[rel="alternate"][hreflang="x-default"]').should('exist');
cy.get('link[rel="alternate"][hreflang="ja"]')
.should('have.attr', 'href')
.and('match', /\/ja\/$/);
});
it('points the canonical URL at the language actually served', () => {
cy.get('link[rel="canonical"]').should('have.attr', 'href').and('match', /\/de\/$/);
});
});

11
app/i18n/ui/ar.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: بدائل
Close: إغلاق
Copy: نسخ
Identifier copied to clipboard!: تم نسخ المعرّف إلى الحافظة!
Imprint: معلومات قانونية
Information: معلومات
Language: اللغة
Open: فتح
Open Link: فتح الرابط
Options: خيارات
Warning: تحذير

11
app/i18n/ui/bn.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: অন্যান্য বিকল্প
Close: বন্ধ করুন
Copy: অনুলিপি
Identifier copied to clipboard!: শনাক্তকারী ক্লিপবোর্ডে অনুলিপি করা হয়েছে!
Imprint: আইনি তথ্য
Information: তথ্য
Language: ভাষা
Open: খুলুন
Open Link: লিঙ্ক খুলুন
Options: বিকল্প
Warning: সতর্কতা

11
app/i18n/ui/cs.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativy
Close: Zavřít
Copy: Kopírovat
Identifier copied to clipboard!: Identifikátor zkopírován do schránky!
Imprint: Tiráž
Information: Informace
Language: Jazyk
Open: Otevřít
Open Link: Otevřít odkaz
Options: Možnosti
Warning: Upozornění

11
app/i18n/ui/da.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativer
Close: Luk
Copy: Kopiér
Identifier copied to clipboard!: Identifikator kopieret til udklipsholder!
Imprint: Juridisk information
Information: Information
Language: Sprog
Open: Åbn
Open Link: Åbn link
Options: Valgmuligheder
Warning: Advarsel

11
app/i18n/ui/de.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativen
Close: Schließen
Copy: Kopieren
Identifier copied to clipboard!: Kennung in die Zwischenablage kopiert!
Imprint: Impressum
Information: Information
Language: Sprache
Open: Öffnen
Open Link: Link öffnen
Options: Optionen
Warning: Warnung

11
app/i18n/ui/el.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Εναλλακτικές
Close: Κλείσιμο
Copy: Αντιγραφή
Identifier copied to clipboard!: Το αναγνωριστικό αντιγράφηκε στο πρόχειρο!
Imprint: Νομικές πληροφορίες
Information: Πληροφορίες
Language: Γλώσσα
Open: Άνοιγμα
Open Link: Άνοιγμα συνδέσμου
Options: Επιλογές
Warning: Προειδοποίηση

11
app/i18n/ui/es.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativas
Close: Cerrar
Copy: Copiar
Identifier copied to clipboard!: ¡Identificador copiado al portapapeles!
Imprint: Aviso legal
Information: Información
Language: Idioma
Open: Abrir
Open Link: Abrir enlace
Options: Opciones
Warning: Advertencia

11
app/i18n/ui/fa.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: جایگزین‌ها
Close: بستن
Copy: کپی
Identifier copied to clipboard!: شناسه در کلیپ‌بورد کپی شد!
Imprint: اطلاعات حقوقی
Information: اطلاعات
Language: زبان
Open: باز کردن
Open Link: باز کردن پیوند
Options: گزینه‌ها
Warning: هشدار

11
app/i18n/ui/fi.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Vaihtoehdot
Close: Sulje
Copy: Kopioi
Identifier copied to clipboard!: Tunniste kopioitu leikepöydälle!
Imprint: Oikeudelliset tiedot
Information: Tiedot
Language: Kieli
Open: Avaa
Open Link: Avaa linkki
Options: Valinnat
Warning: Varoitus

11
app/i18n/ui/fr.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatives
Close: Fermer
Copy: Copier
Identifier copied to clipboard!: Identifiant copié dans le presse-papiers !
Imprint: Mentions légales
Information: Informations
Language: Langue
Open: Ouvrir
Open Link: Ouvrir le lien
Options: Options
Warning: Avertissement

11
app/i18n/ui/he.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: חלופות
Close: סגור
Copy: העתק
Identifier copied to clipboard!: המזהה הועתק ללוח!
Imprint: מידע משפטי
Information: מידע
Language: שפה
Open: פתח
Open Link: פתח קישור
Options: אפשרויות
Warning: אזהרה

11
app/i18n/ui/hi.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: अन्य विकल्प
Close: बंद करें
Copy: कॉपी करें
Identifier copied to clipboard!: पहचानकर्ता क्लिपबोर्ड पर कॉपी हो गया!
Imprint: कानूनी सूचना
Information: जानकारी
Language: भाषा
Open: खोलें
Open Link: लिंक खोलें
Options: विकल्प
Warning: चेतावनी

11
app/i18n/ui/hu.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatívák
Close: Bezárás
Copy: Másolás
Identifier copied to clipboard!: Az azonosító a vágólapra másolva!
Imprint: Impresszum
Information: Információ
Language: Nyelv
Open: Megnyitás
Open Link: Hivatkozás megnyitása
Options: Lehetőségek
Warning: Figyelmeztetés

11
app/i18n/ui/id.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatif
Close: Tutup
Copy: Salin
Identifier copied to clipboard!: Pengenal disalin ke papan klip!
Imprint: Informasi hukum
Information: Informasi
Language: Bahasa
Open: Buka
Open Link: Buka tautan
Options: Opsi
Warning: Peringatan

11
app/i18n/ui/it.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternative
Close: Chiudi
Copy: Copia
Identifier copied to clipboard!: Identificatore copiato negli appunti!
Imprint: Note legali
Information: Informazioni
Language: Lingua
Open: Apri
Open Link: Apri collegamento
Options: Opzioni
Warning: Avviso

11
app/i18n/ui/ja.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: 代替
Close: 閉じる
Copy: コピー
Identifier copied to clipboard!: 識別子をクリップボードにコピーしました!
Imprint: 法的情報
Information: 情報
Language: 言語
Open: 開く
Open Link: リンクを開く
Options: オプション
Warning: 警告

11
app/i18n/ui/ko.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: 대안
Close: 닫기
Copy: 복사
Identifier copied to clipboard!: 식별자를 클립보드에 복사했습니다!
Imprint: 법적 고지
Information: 정보
Language: 언어
Open: 열기
Open Link: 링크 열기
Options: 옵션
Warning: 경고

11
app/i18n/ui/nl.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatieven
Close: Sluiten
Copy: Kopiëren
Identifier copied to clipboard!: Identificatie gekopieerd naar klembord!
Imprint: Colofon
Information: Informatie
Language: Taal
Open: Openen
Open Link: Link openen
Options: Opties
Warning: Waarschuwing

11
app/i18n/ui/pl.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatywy
Close: Zamknij
Copy: Kopiuj
Identifier copied to clipboard!: Identyfikator skopiowany do schowka!
Imprint: Nota prawna
Information: Informacja
Language: Język
Open: Otwórz
Open Link: Otwórz link
Options: Opcje
Warning: Ostrzeżenie

11
app/i18n/ui/pt.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativas
Close: Fechar
Copy: Copiar
Identifier copied to clipboard!: Identificador copiado para a área de transferência!
Imprint: Aviso legal
Information: Informação
Language: Idioma
Open: Abrir
Open Link: Abrir link
Options: Opções
Warning: Aviso

11
app/i18n/ui/ro.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternative
Close: Închide
Copy: Copiază
Identifier copied to clipboard!: Identificator copiat în clipboard!
Imprint: Notă legală
Information: Informații
Language: Limbă
Open: Deschide
Open Link: Deschide linkul
Options: Opțiuni
Warning: Avertisment

11
app/i18n/ui/ru.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Альтернативы
Close: Закрыть
Copy: Копировать
Identifier copied to clipboard!: Идентификатор скопирован в буфер обмена!
Imprint: Выходные данные
Information: Информация
Language: Язык
Open: Открыть
Open Link: Открыть ссылку
Options: Варианты
Warning: Предупреждение

11
app/i18n/ui/sv.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternativ
Close: Stäng
Copy: Kopiera
Identifier copied to clipboard!: Identifierare kopierad till urklipp!
Imprint: Juridisk information
Information: Information
Language: Språk
Open: Öppna
Open Link: Öppna länk
Options: Val
Warning: Varning

11
app/i18n/ui/th.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: ทางเลือกอื่น
Close: ปิด
Copy: คัดลอก
Identifier copied to clipboard!: คัดลอกตัวระบุไปยังคลิปบอร์ดแล้ว!
Imprint: ข้อมูลทางกฎหมาย
Information: ข้อมูล
Language: ภาษา
Open: เปิด
Open Link: เปิดลิงก์
Options: ตัวเลือก
Warning: คำเตือน

11
app/i18n/ui/tr.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Alternatifler
Close: Kapat
Copy: Kopyala
Identifier copied to clipboard!: Tanımlayıcı panoya kopyalandı!
Imprint: Künye
Information: Bilgi
Language: Dil
Open:
Open Link: Bağlantıyı
Options: Seçenekler
Warning: Uyarı

11
app/i18n/ui/uk.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Альтернативи
Close: Закрити
Copy: Копіювати
Identifier copied to clipboard!: Ідентифікатор скопійовано в буфер обміну!
Imprint: Вихідні дані
Information: Інформація
Language: Мова
Open: Відкрити
Open Link: Відкрити посилання
Options: Варіанти
Warning: Попередження

11
app/i18n/ui/ur.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: متبادل
Close: بند کریں
Copy: نقل کریں
Identifier copied to clipboard!: شناخت کنندہ کلپ بورڈ پر نقل ہو گیا!
Imprint: قانونی معلومات
Information: معلومات
Language: زبان
Open: کھولیں
Open Link: لنک کھولیں
Options: اختیارات
Warning: انتباہ

11
app/i18n/ui/vi.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: Lựa chọn thay thế
Close: Đóng
Copy: Sao chép
Identifier copied to clipboard!: Đã sao chép mã định danh vào bộ nhớ tạm!
Imprint: Thông tin pháp lý
Information: Thông tin
Language: Ngôn ngữ
Open: Mở
Open Link: Mở liên kết
Options: Tùy chọn
Warning: Cảnh báo

11
app/i18n/ui/zh.yaml Normal file
View File

@@ -0,0 +1,11 @@
Alternatives: 替代方案
Close: 关闭
Copy: 复制
Identifier copied to clipboard!: 标识符已复制到剪贴板!
Imprint: 法律声明
Information: 信息
Language: 语言
Open: 打开
Open Link: 打开链接
Options: 选项
Warning: 警告

View File

@@ -29,6 +29,10 @@ copyFile(
path.join(NM, 'bootstrap', 'dist', 'css', 'bootstrap.min.css'), path.join(NM, 'bootstrap', 'dist', 'css', 'bootstrap.min.css'),
path.join(VENDOR, 'bootstrap', 'css', 'bootstrap.min.css') path.join(VENDOR, 'bootstrap', 'css', 'bootstrap.min.css')
); );
copyFile(
path.join(NM, 'bootstrap', 'dist', 'css', 'bootstrap.rtl.min.css'),
path.join(VENDOR, 'bootstrap', 'css', 'bootstrap.rtl.min.css')
);
copyFile( copyFile(
path.join(NM, 'bootstrap', 'dist', 'js', 'bootstrap.bundle.min.js'), path.join(NM, 'bootstrap', 'dist', 'js', 'bootstrap.bundle.min.js'),
path.join(VENDOR, 'bootstrap', 'js', 'bootstrap.bundle.min.js') path.join(VENDOR, 'bootstrap', 'js', 'bootstrap.bundle.min.js')

View File

@@ -1,5 +1,5 @@
<!DOCTYPE html> <!DOCTYPE html>
<html lang="en"> <html lang="{{ lang }}" dir="{{ lang_dir }}">
<head> <head>
<title>{{platform.titel}}</title> <title>{{platform.titel}}</title>
<meta charset="utf-8" > <meta charset="utf-8" >
@@ -8,8 +8,13 @@
type="image/x-icon" type="image/x-icon"
href="{{ asset_src(platform.favicon) }}" href="{{ asset_src(platform.favicon) }}"
> >
<link rel="canonical" href="{{ url_for('localized_index', lang=lang, _external=True) }}">
{% for code in languages %}
<link rel="alternate" hreflang="{{ code }}" href="{{ url_for('localized_index', lang=code, _external=True) }}">
{% endfor %}
<link rel="alternate" hreflang="x-default" href="{{ url_for('index', _external=True) }}">
<!-- Bootstrap CSS only --> <!-- Bootstrap CSS only -->
<link href="{{ url_for('static', filename='vendor/bootstrap/css/bootstrap.min.css') }}" rel="stylesheet"> <link href="{{ url_for('static', filename='vendor/bootstrap/css/bootstrap' ~ ('.rtl' if lang_dir == 'rtl' else '') ~ '.min.css') }}" rel="stylesheet">
<!-- Bootstrap JavaScript Bundle with Popper --> <!-- Bootstrap JavaScript Bundle with Popper -->
<script src="{{ url_for('static', filename='vendor/bootstrap/js/bootstrap.bundle.min.js') }}"></script> <script src="{{ url_for('static', filename='vendor/bootstrap/js/bootstrap.bundle.min.js') }}"></script>
<!-- Bootstrap Icons --> <!-- Bootstrap Icons -->
@@ -60,12 +65,13 @@
<p itemprop="name">{{ company.titel }} <br /> <p itemprop="name">{{ company.titel }} <br />
{{ company.subtitel }}</p> {{ company.subtitel }}</p>
<span><i class="fa-solid fa-location-dot"></i> {{ company.address.values() | join(", ") }}</span> <span><i class="fa-solid fa-location-dot"></i> {{ company.address.values() | join(", ") }}</span>
<p><a href="{{company.imprint_url}}" class="iframe-link"><i class="fa-solid fa-scale-balanced"></i> Imprint</a></p> <p><a href="{{company.imprint_url}}" class="iframe-link"><i class="fa-solid fa-scale-balanced"></i> {{ t('Imprint') }}</a></p>
</div> </div>
</footer> </footer>
</div> </div>
<!-- Include modal --> <!-- Include modal -->
{% include "moduls/modal.html.j2" %} {% include "moduls/modal.html.j2" %}
<script>window.I18N = {{ ui_strings | tojson }};</script>
{% for name in [ {% for name in [
'modal', 'modal',
'navigation', 'navigation',

View File

@@ -16,14 +16,14 @@
<div class="modal-content"> <div class="modal-content">
<div class="modal-header"> <div class="modal-header">
<h5 class="modal-title" id="dynamicModalLabel"></h5> <h5 class="modal-title" id="dynamicModalLabel"></h5>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button> <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="{{ t('Close') }}"></button>
</div> </div>
<div class="modal-body"> <div class="modal-body">
<!-- Warning box with Markdown --> <!-- Warning box with Markdown -->
{{ alert_box('dynamicModalWarning', 'alert-warning', 'fa-solid fa-triangle-exclamation', 'Warning', 'dynamicModalWarningText') }} {{ alert_box('dynamicModalWarning', 'alert-warning', 'fa-solid fa-triangle-exclamation', t('Warning'), 'dynamicModalWarningText') }}
<!-- Info box with Markdown --> <!-- Info box with Markdown -->
{{ alert_box('dynamicModalInfo', 'alert-info', 'fa-solid fa-circle-info', 'Information', 'dynamicModalInfoText') }} {{ alert_box('dynamicModalInfo', 'alert-info', 'fa-solid fa-circle-info', t('Information'), 'dynamicModalInfoText') }}
<!-- Description text --> <!-- Description text -->
<div id="dynamicDescriptionText" class="mt-2 d-none"></div> <div id="dynamicDescriptionText" class="mt-2 d-none"></div>
@@ -31,7 +31,7 @@
<!-- Input box for Identifier --> <!-- Input box for Identifier -->
<div id="dynamicIdentifierBox" class="input-group mt-2 d-none"> <div id="dynamicIdentifierBox" class="input-group mt-2 d-none">
<input type="text" id="dynamicModalContent" class="form-control" readonly> <input type="text" id="dynamicModalContent" class="form-control" readonly>
<button class="btn btn-outline-secondary" type="button" id="dynamicCopyButton">Copy</button> <button class="btn btn-outline-secondary" type="button" id="dynamicCopyButton">{{ t('Copy') }}</button>
</div> </div>
<!-- Link --> <!-- Link -->
@@ -40,13 +40,13 @@
</div> </div>
<!-- Options --> <!-- Options -->
{{ list_section('dynamicChildrenSection', 'Options', 'dynamicChildrenList') }} {{ list_section('dynamicChildrenSection', t('Options'), 'dynamicChildrenList') }}
<!-- Alternatives --> <!-- Alternatives -->
{{ list_section('dynamicAlternativesSection', 'Alternatives', 'dynamicAlternativesList') }} {{ list_section('dynamicAlternativesSection', t('Alternatives'), 'dynamicAlternativesList') }}
</div> </div>
<div class="modal-footer"> <div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal" onclick="closeAllModals()">Close</button> <button type="button" class="btn btn-secondary" data-bs-dismiss="modal" onclick="closeAllModals()">{{ t('Close') }}</button>
</div> </div>
</div> </div>
</div> </div>

View File

@@ -98,6 +98,23 @@
</li> </li>
{% endif %} {% endif %}
{% endfor %} {% endfor %}
{% if menu_type == "header" %}
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle btn btn-light" id="navbarDropdownLanguage" role="button" data-bs-toggle="dropdown" data-bs-display="dynamic" aria-expanded="false" title="{{ t('Language') }}">
<i class="fa-solid fa-language"></i> {{ languages[lang] }}
</a>
<ul class="dropdown-menu">
{% for code, endonym in languages.items() %}
<li>
<a class="dropdown-item {% if code == lang %}active{% endif %}"
href="{{ url_for('localized_index', lang=code) }}"
hreflang="{{ code }}"
lang="{{ code }}">{{ endonym }}</a>
</li>
{% endfor %}
</ul>
</li>
{% endif %}
</ul> </ul>
</div> </div>
</nav> </nav>

163
app/utils/i18n.py Normal file
View File

@@ -0,0 +1,163 @@
"""Language negotiation and translation of the resolved configuration tree.
Translation is catalogue-driven: a string is replaced only when the target
language's catalogue holds an entry for the exact English source string.
Anything unknown falls through to English, so a partially filled catalogue
degrades instead of breaking.
"""
import logging
from pathlib import Path
import yaml
I18N_DIR = Path(__file__).resolve().parent.parent / "i18n"
UI_DIR = I18N_DIR / "ui"
CONTENT_DIR = I18N_DIR / "content"
SOURCE_LANGUAGE = "en"
LANGUAGES = {
"en": "English",
"zh": "中文",
"hi": "हिन्दी",
"es": "Español",
"fr": "Français",
"ar": "العربية",
"bn": "বাংলা",
"pt": "Português",
"ru": "Русский",
"ur": "اردو",
"id": "Bahasa Indonesia",
"de": "Deutsch",
"ja": "日本語",
"tr": "Türkçe",
"ko": "한국어",
"vi": "Tiếng Việt",
"it": "Italiano",
"th": "ไทย",
"pl": "Polski",
"nl": "Nederlands",
"uk": "Українська",
"fa": "فارسی",
"ro": "Română",
"el": "Ελληνικά",
"cs": "Čeština",
"sv": "Svenska",
"hu": "Magyar",
"he": "עברית",
"da": "Dansk",
"fi": "Suomi",
}
RTL_LANGUAGES = frozenset({"ar", "fa", "he", "ur"})
AUTOFILL_KEYS = frozenset({"description", "text", "warning", "info", "subtitel"})
TRANSLATABLE_KEYS = AUTOFILL_KEYS | frozenset({"name", "title"})
UI_STRINGS = (
"Alternatives",
"Close",
"Copy",
"Identifier copied to clipboard!",
"Imprint",
"Information",
"Language",
"Open",
"Open Link",
"Options",
"Warning",
)
_catalogs: dict[str, dict[str, str]] = {}
def direction(code):
"""Return the writing direction of ``code`` as an HTML ``dir`` value."""
return "rtl" if code in RTL_LANGUAGES else "ltr"
def read_catalog(path):
"""Return the catalogue at ``path``, or an empty one if it is unusable.
Catalogues are hand-edited and machine-written, so a stray character must
degrade that language to English rather than take every page down with a
parse error. Non-string entries are dropped for the same reason: they would
otherwise reach the templates and render as ``42`` or ``null``.
"""
if not path.exists():
return {}
try:
loaded = yaml.safe_load(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, yaml.YAMLError):
logging.warning("Ignoring unreadable translation catalogue: %s", path)
return {}
if not isinstance(loaded, dict):
logging.warning(
"Ignoring translation catalogue that is not a mapping: %s", path
)
return {}
return {
key: value
for key, value in loaded.items()
if isinstance(key, str) and isinstance(value, str)
}
def clear_catalogs():
"""Drop the memoized catalogues so edited files are picked up."""
_catalogs.clear()
def catalog(code):
"""Return the merged UI and content catalogue for ``code``."""
if code not in _catalogs:
_catalogs[code] = {
**read_catalog(UI_DIR / f"{code}.yaml"),
**read_catalog(CONTENT_DIR / f"{code}.yaml"),
}
return _catalogs[code]
def negotiate(accepted, default=SOURCE_LANGUAGE):
"""Pick the best supported language from ``Accept-Language`` pairs.
Args:
accepted: iterable of ``(tag, quality)`` as produced by
``flask.request.accept_languages``.
default: language returned when no tag is supported.
Werkzeug's own ``best_match`` returns an exact match before it considers
primary-tag fallbacks, so ``de-DE,en;q=0.8`` resolves to English. Matching
on the primary subtag up front avoids that.
"""
best, best_quality = default, 0.0
for tag, quality in accepted:
code = tag.replace("_", "-").split("-")[0].lower()
if code in LANGUAGES and quality > best_quality:
best, best_quality = code, quality
return best
def translate_tree(node, code, key=None):
"""Return a copy of ``node`` with translatable leaves swapped for ``code``.
Args:
node: the resolved configuration tree, or any subtree of it.
code: target language code.
key: the mapping key ``node`` was reached through.
"""
if isinstance(node, dict):
return {name: translate_tree(value, code, name) for name, value in node.items()}
if isinstance(node, list):
return [translate_tree(item, code, key) for item in node]
if isinstance(node, str) and key in TRANSLATABLE_KEYS:
return catalog(code).get(node, node)
return node
def ui_strings(code):
"""Return the interface strings for ``code``, keyed by their English source."""
entries = catalog(code)
return {source: entries.get(source, source) for source in UI_STRINGS}

View File

@@ -4,3 +4,10 @@ FLASK_ENV=production
# Makefile (build/run-dev/run-prod) and docker-compose.yml so every # Makefile (build/run-dev/run-prod) and docker-compose.yml so every
# build path produces and consumes the same tag. # build path produces and consumes the same tag.
IMAGE_NAME=portfolio IMAGE_NAME=portfolio
# Comma-separated public hostnames. Leave empty and the app reflects whatever
# Host header arrives into its canonical, hreflang and redirect URLs.
TRUSTED_HOSTS=
# LibreTranslate instance used by `make i18n` to fill app/i18n/content/.
# Only needed when generating translations, never at runtime.
LIBRETRANSLATE_URL=http://localhost:5002
LIBRETRANSLATE_API_KEY=

View File

@@ -0,0 +1,195 @@
import json
import os
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
app_module = None
flask_app = None
i18n = None
_origin = None
_workdir = None
def setUpModule():
"""Import the app with a disposable configuration as the working directory.
``app.app`` reads ``config.yaml`` relative to the process working directory
at import time, so the chdir has to happen before the import.
"""
global app_module, flask_app, i18n, _origin, _workdir
_origin = os.getcwd()
_workdir = tempfile.mkdtemp(prefix="portfolio-routes-")
shutil.copy(
REPO_ROOT / "app" / "config.sample.yaml", Path(_workdir) / "config.yaml"
)
os.chdir(_workdir)
from app import app as imported_module
from app.utils import i18n as imported_i18n
app_module = imported_module
flask_app = imported_module.app
i18n = imported_i18n
flask_app.config["NASA_API_KEY"] = None
def tearDownModule():
os.chdir(_origin)
shutil.rmtree(_workdir, ignore_errors=True)
class AppRouteMixin:
"""Shared setup. A mixin rather than a TestCase subclass, so that every
test class below still names ``unittest.TestCase`` as a direct base — the
lint guardrail in tests/lint/ does not resolve inherited aliases."""
def setUp(self):
self.client = flask_app.test_client()
self.addCleanup(i18n.clear_catalogs)
self.addCleanup(flask_app.config["TRANSLATED_CONFIG"].clear)
class TestRouting(AppRouteMixin, unittest.TestCase):
def test_unrelated_single_segment_paths_are_not_redirected(self):
for path in ("/robots.txt", "/favicon.ico", "/sitemap.xml"):
with self.subTest(path=path):
self.assertEqual(self.client.get(path).status_code, 404)
def test_language_path_redirects_to_the_canonical_trailing_slash(self):
response = self.client.get("/de")
self.assertEqual(response.status_code, 308)
self.assertTrue(response.headers["Location"].endswith("/de/"))
def test_supported_language_renders(self):
response = self.client.get("/de/")
self.assertEqual(response.status_code, 200)
self.assertIn('<html lang="de"', response.get_data(as_text=True))
def test_unsupported_language_is_not_found(self):
self.assertEqual(self.client.get("/xx/").status_code, 404)
class TestNegotiation(AppRouteMixin, unittest.TestCase):
def test_regional_tag_beats_a_lower_ranked_exact_match(self):
response = self.client.get("/", headers={"Accept-Language": "de-DE,en;q=0.8"})
self.assertIn('<html lang="de"', response.get_data(as_text=True))
def test_negotiated_route_declares_that_it_varies(self):
response = self.client.get("/", headers={"Accept-Language": "de-DE"})
self.assertEqual(response.headers.get("Vary"), "Accept-Language")
class TestEscaping(AppRouteMixin, unittest.TestCase):
def test_catalog_content_is_html_escaped(self):
i18n._catalogs["de"] = {"Imprint": "<script>alert(1)</script>"}
body = self.client.get("/de/").get_data(as_text=True)
self.assertNotIn("<script>alert(1)</script>", body)
self.assertIn("&lt;script&gt;alert(1)", body)
class TestExternalUrls(AppRouteMixin, unittest.TestCase):
def test_forwarded_scheme_reaches_the_canonical_and_alternates(self):
body = self.client.get(
"/en/",
headers={"Host": "portfolio.example.org", "X-Forwarded-Proto": "https"},
).get_data(as_text=True)
self.assertIn(
'<link rel="canonical" href="https://portfolio.example.org/en/">', body
)
self.assertIn('hreflang="ja" href="https://portfolio.example.org/ja/"', body)
self.assertNotIn("http://portfolio.example.org", body)
def test_trusted_hosts_are_parsed_from_a_comma_separated_list(self):
self.assertEqual(
app_module.trusted_hosts("a.test, b.test ,"), ["a.test", "b.test"]
)
def test_an_unset_trusted_hosts_value_disables_the_check(self):
self.assertIsNone(app_module.trusted_hosts(""))
self.assertIsNone(app_module.trusted_hosts(" , "))
def test_a_forged_host_is_rejected_once_trusted_hosts_are_named(self):
self.addCleanup(flask_app.config.__setitem__, "TRUSTED_HOSTS", None)
flask_app.config["TRUSTED_HOSTS"] = ["portfolio.example.org"]
forged = self.client.get("/de", headers={"Host": "evil.test"})
honest = self.client.get("/de/", headers={"Host": "portfolio.example.org"})
self.assertEqual(forged.status_code, 400)
self.assertEqual(honest.status_code, 200)
def test_forwarded_host_is_not_trusted(self):
response = self.client.get(
"/de",
headers={"Host": "portfolio.example.org", "X-Forwarded-Host": "evil.test"},
)
self.assertNotIn("evil.test", response.headers["Location"])
body = self.client.get(
"/en/",
headers={"Host": "portfolio.example.org", "X-Forwarded-Host": "evil.test"},
).get_data(as_text=True)
self.assertNotIn("evil.test", body)
class TestConfigurationReload(AppRouteMixin, unittest.TestCase):
def test_reloading_the_configuration_drops_the_catalog_memo(self):
i18n.catalog("de")
self.assertIn("de", i18n._catalogs)
app_module.load_config(flask_app)
self.assertEqual(i18n._catalogs, {})
def test_reloading_the_configuration_drops_the_translation_memo(self):
self.client.get("/de/")
self.assertIn("de", flask_app.config["TRANSLATED_CONFIG"])
app_module.load_config(flask_app)
self.assertEqual(flask_app.config["TRANSLATED_CONFIG"], {})
class TestTrustedHostsWiring(unittest.TestCase):
def test_the_environment_reaches_the_flask_configuration(self):
script = (
"import json, sys;"
f"sys.path.insert(0, {str(REPO_ROOT)!r});"
"from app.app import app;"
"print('TRUSTED=' + json.dumps(app.config['TRUSTED_HOSTS']))"
)
result = subprocess.run(
[sys.executable, "-c", script],
cwd=_workdir,
env={**os.environ, "TRUSTED_HOSTS": "a.test, b.test"},
capture_output=True,
text=True,
check=True,
)
reported = [
line for line in result.stdout.splitlines() if line.startswith("TRUSTED=")
]
self.assertEqual(
json.loads(reported[-1][len("TRUSTED=") :]), ["a.test", "b.test"]
)
if __name__ == "__main__":
unittest.main()

View File

@@ -1,8 +1,17 @@
import re
import tomllib import tomllib
import unittest import unittest
from pathlib import Path from pathlib import Path
def distributions(requirements):
"""Return the distribution names of PEP 508 requirement strings."""
return {
re.split(r"[<>=!~\[; ]", requirement, maxsplit=1)[0]
for requirement in requirements
}
class TestPythonPackaging(unittest.TestCase): class TestPythonPackaging(unittest.TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self.repo_root = Path(__file__).resolve().parents[2] self.repo_root = Path(__file__).resolve().parents[2]
@@ -18,11 +27,20 @@ class TestPythonPackaging(unittest.TestCase):
self.assertEqual(build_system["build-backend"], "setuptools.build_meta") self.assertEqual(build_system["build-backend"], "setuptools.build_meta")
self.assertIn("setuptools>=69", build_system["requires"]) self.assertIn("setuptools>=69", build_system["requires"])
self.assertGreaterEqual( self.assertGreaterEqual(
set(project["dependencies"]), distributions(project["dependencies"]),
{"flask", "pyyaml", "requests"}, {"flask", "pyyaml", "requests"},
) )
self.assertEqual(project["requires-python"], ">=3.12") self.assertEqual(project["requires-python"], ">=3.12")
def test_flask_is_pinned_to_a_version_that_honours_trusted_hosts(self):
requirement = next(
item
for item in self.pyproject["project"]["dependencies"]
if item.startswith("flask")
)
self.assertIn(">=3.1", requirement)
def test_pyproject_defines_dev_dependencies_and_package_contents(self): def test_pyproject_defines_dev_dependencies_and_package_contents(self):
project = self.pyproject["project"] project = self.pyproject["project"]
setuptools_config = self.pyproject["tool"]["setuptools"] setuptools_config = self.pyproject["tool"]["setuptools"]
@@ -30,8 +48,8 @@ class TestPythonPackaging(unittest.TestCase):
package_data = setuptools_config["package-data"]["app"] package_data = setuptools_config["package-data"]["app"]
self.assertGreaterEqual( self.assertGreaterEqual(
set(project["optional-dependencies"]["dev"]), distributions(project["optional-dependencies"]["dev"]),
{"bandit", "pip-audit", "ruff"}, {"bandit", "pip-audit", "ruff", "yamllint"},
) )
self.assertEqual(setuptools_config["py-modules"], ["main"]) self.assertEqual(setuptools_config["py-modules"], ["main"])
self.assertEqual(package_find["include"], ["app", "app.*"]) self.assertEqual(package_find["include"], ["app", "app.*"])

View File

@@ -0,0 +1,78 @@
"""Guards for configuration that no other test observes.
Each assertion here stands for a defect that was found by deleting the line it
checks: the deletion is invisible to every suite, and its effect only shows up
in production or in a fresh checkout.
"""
import re
import tomllib
import unittest
from pathlib import Path
import yaml
REPO_ROOT = Path(__file__).resolve().parents[2]
class TestYamllintConfiguration(unittest.TestCase):
def setUp(self):
self.config = yaml.safe_load(
(REPO_ROOT / ".yamllint").read_text(encoding="utf-8")
)
def test_duplicate_keys_are_an_error(self):
self.assertEqual(self.config["rules"]["key-duplicates"], "enable")
def test_the_directories_that_collect_foreign_yaml_are_ignored(self):
ignored = self.config["ignore"].split()
self.assertGreaterEqual(
set(ignored),
{".git/", ".venv/", "node_modules/", "app/node_modules/"},
)
class TestRunTargets(unittest.TestCase):
def setUp(self):
self.makefile = (REPO_ROOT / "Makefile").read_text(encoding="utf-8")
self.recipes = {
name: body
for name, body in re.findall(
r"^(run-dev|run-prod):[^\n]*\n((?:\t[^\n]*\n)+)",
self.makefile,
re.MULTILINE,
)
}
def test_both_run_targets_exist(self):
self.assertEqual(set(self.recipes), {"run-dev", "run-prod"})
def test_the_container_is_told_which_hosts_are_trusted(self):
for name, body in self.recipes.items():
with self.subTest(target=name):
self.assertIn("TRUSTED_HOSTS", body)
def test_the_container_is_told_which_port_to_bind(self):
for name, body in self.recipes.items():
with self.subTest(target=name):
self.assertIn('-e PORT="$$PORT"', body)
def test_the_whole_env_file_is_not_handed_to_the_web_container(self):
for name, body in self.recipes.items():
with self.subTest(target=name):
self.assertNotIn("--env-file", body)
class TestPackagedCatalogs(unittest.TestCase):
def test_the_interface_catalogs_are_declared_as_package_data(self):
with (REPO_ROOT / "pyproject.toml").open("rb") as handle:
pyproject = tomllib.load(handle)
package_data = pyproject["tool"]["setuptools"]["package-data"]["app"]
self.assertIn("i18n/ui/*.yaml", package_data)
if __name__ == "__main__":
unittest.main()

212
tests/unit/test_i18n.py Normal file
View File

@@ -0,0 +1,212 @@
import re
import shutil
import tempfile
import unittest
from pathlib import Path
import yaml
from app.utils import i18n
class TestNegotiate(unittest.TestCase):
def test_regional_tag_beats_a_lower_ranked_exact_match(self):
self.assertEqual(i18n.negotiate([("de-DE", 1.0), ("en", 0.8)]), "de")
def test_underscore_separated_tag_is_accepted(self):
self.assertEqual(i18n.negotiate([("pt_BR", 1.0)]), "pt")
def test_highest_quality_supported_tag_wins(self):
self.assertEqual(
i18n.negotiate([("xx", 1.0), ("fr", 0.9), ("es", 0.5)]),
"fr",
)
def test_unsupported_tags_fall_back_to_the_default(self):
self.assertEqual(i18n.negotiate([("xx", 1.0)]), "en")
self.assertEqual(i18n.negotiate([], default="de"), "de")
def test_a_refused_language_is_not_selected(self):
self.assertEqual(i18n.negotiate([("de", 0.0)]), "en")
def test_the_first_of_two_equal_tags_wins(self):
self.assertEqual(i18n.negotiate([("fr", 0.9), ("es", 0.9)]), "fr")
class TestDirection(unittest.TestCase):
def test_every_right_to_left_language_is_marked(self):
for code in ("ar", "fa", "he", "ur"):
with self.subTest(code=code):
self.assertEqual(i18n.direction(code), "rtl")
def test_other_languages_are_left_to_right(self):
for code in set(i18n.LANGUAGES) - {"ar", "fa", "he", "ur"}:
with self.subTest(code=code):
self.assertEqual(i18n.direction(code), "ltr")
class TestTranslateTree(unittest.TestCase):
def setUp(self):
self.addCleanup(i18n._catalogs.clear)
i18n._catalogs["xx"] = {"A card": "Eine Karte", "Pictures": "Bilder"}
def test_only_translatable_keys_are_replaced(self):
tree = {
"cards": [
{
"title": "Pictures",
"text": "A card",
"url": "A card",
"icon": {"class": "Pictures"},
}
]
}
translated = i18n.translate_tree(tree, "xx")
card = translated["cards"][0]
self.assertEqual(card["title"], "Bilder")
self.assertEqual(card["text"], "Eine Karte")
self.assertEqual(card["url"], "A card")
self.assertEqual(card["icon"]["class"], "Pictures")
def test_unknown_strings_keep_their_source_value(self):
translated = i18n.translate_tree({"description": "Untranslated"}, "xx")
self.assertEqual(translated["description"], "Untranslated")
def test_source_tree_is_left_untouched(self):
tree = {"name": "Pictures"}
i18n.translate_tree(tree, "xx")
self.assertEqual(tree["name"], "Pictures")
def test_non_string_leaves_survive(self):
tree = {"name": 1, "text": None, "info": True}
self.assertEqual(i18n.translate_tree(tree, "xx"), tree)
class TestReadCatalog(unittest.TestCase):
def setUp(self):
self.directory = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.directory, True)
self.path = self.directory / "de.yaml"
def _read(self, text):
self.path.write_text(text, encoding="utf-8")
with self.assertLogs(level="WARNING"):
return i18n.read_catalog(self.path)
def test_absent_file_is_an_empty_catalog(self):
self.assertEqual(i18n.read_catalog(self.directory / "missing.yaml"), {})
def test_unparsable_yaml_is_an_empty_catalog(self):
self.assertEqual(self._read("Close: [unclosed"), {})
def test_tab_indentation_is_an_empty_catalog(self):
self.assertEqual(self._read("a:\n\tb: 1"), {})
def test_non_mapping_document_is_an_empty_catalog(self):
self.assertEqual(self._read("- a\n- b"), {})
def test_a_non_utf8_catalog_is_an_empty_catalog(self):
self.path.write_bytes(b"\xffClose: Schliessen\n")
with self.assertLogs(level="WARNING"):
self.assertEqual(i18n.read_catalog(self.path), {})
def test_a_directory_at_the_catalog_path_is_an_empty_catalog(self):
(self.directory / "sub.yaml").mkdir()
with self.assertLogs(level="WARNING"):
self.assertEqual(i18n.read_catalog(self.directory / "sub.yaml"), {})
def test_non_string_entries_are_dropped(self):
self.path.write_text(
"Close: 42\nOpen:\nCopy: yes\nImprint: Impressum\n", encoding="utf-8"
)
self.assertEqual(i18n.read_catalog(self.path), {"Imprint": "Impressum"})
class TestCatalogMerge(unittest.TestCase):
def setUp(self):
directory = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, directory, True)
(directory / "ui").mkdir()
(directory / "content").mkdir()
self.addCleanup(setattr, i18n, "UI_DIR", i18n.UI_DIR)
self.addCleanup(setattr, i18n, "CONTENT_DIR", i18n.CONTENT_DIR)
self.addCleanup(i18n.clear_catalogs)
i18n.UI_DIR = directory / "ui"
i18n.CONTENT_DIR = directory / "content"
i18n.clear_catalogs()
def test_the_content_catalog_overrides_the_shipped_interface_string(self):
(i18n.UI_DIR / "de.yaml").write_text(
"Close: Schliessen\nImprint: Impressum\n", encoding="utf-8"
)
(i18n.CONTENT_DIR / "de.yaml").write_text("Close: Zumachen\n", encoding="utf-8")
self.assertEqual(
i18n.catalog("de"), {"Close": "Zumachen", "Imprint": "Impressum"}
)
class TestShippedCatalogs(unittest.TestCase):
def test_thirty_languages_are_offered(self):
self.assertEqual(len(i18n.LANGUAGES), 30)
self.assertIn(i18n.SOURCE_LANGUAGE, i18n.LANGUAGES)
def test_every_code_is_usable_in_the_route_converter(self):
offenders = [
code for code in i18n.LANGUAGES if not re.fullmatch(r"[a-z]+", code)
]
self.assertFalse(offenders, f"Not usable in the route converter: {offenders}")
def test_every_language_but_the_source_ships_a_ui_catalog(self):
missing = [
code
for code in i18n.LANGUAGES
if code != i18n.SOURCE_LANGUAGE
and not (i18n.UI_DIR / f"{code}.yaml").is_file()
]
self.assertFalse(missing, f"No UI catalogue for: {missing}")
def test_ui_catalogs_cover_exactly_the_interface_strings(self):
expected = set(i18n.UI_STRINGS)
mismatched = {}
for code in i18n.LANGUAGES:
if code == i18n.SOURCE_LANGUAGE:
continue
path = i18n.UI_DIR / f"{code}.yaml"
entries = yaml.safe_load(path.read_text(encoding="utf-8"))
if set(entries) != expected:
mismatched[code] = {
"missing": sorted(expected - set(entries)),
"unexpected": sorted(set(entries) - expected),
}
self.assertFalse(mismatched, f"UI catalogues out of sync: {mismatched}")
def test_ui_strings_of_the_source_language_are_the_source(self):
self.assertEqual(
i18n.ui_strings("en"),
{source: source for source in i18n.UI_STRINGS},
)
def test_ui_strings_are_translated_for_a_shipped_language(self):
strings = i18n.ui_strings("de")
self.assertEqual(strings["Close"], "Schließen")
self.assertEqual(strings["Imprint"], "Impressum")
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,280 @@
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import yaml
from app.utils import i18n
from utils import i18n_sync
class TestCollectSources(unittest.TestCase):
def test_prose_keys_are_collected_from_nested_structures(self):
config = {
"cards": [
{"title": "Agile Coach", "text": "I lead transformations."},
{"text": "Another card."},
],
"navigation": {
"header": {
"children": [
{"name": "Apps", "description": "Application menu"},
]
}
},
"platform": {"titel": "Someone", "subtitel": "A tagline"},
}
self.assertEqual(
i18n_sync.collect_sources(config),
{
"I lead transformations.",
"Another card.",
"Application menu",
"A tagline",
},
)
def test_label_and_structural_keys_are_left_alone(self):
config = {
"name": "Mastodon",
"title": "Cybermaster",
"url": "https://example.test",
"link_text": "www.example.test",
}
self.assertEqual(i18n_sync.collect_sources(config), set())
def test_blank_values_are_ignored(self):
self.assertEqual(i18n_sync.collect_sources({"text": " "}), set())
class TestTranslate(unittest.TestCase):
def _session(self, ok, payload=None, status=200, text=""):
response = Mock(ok=ok, status_code=status, text=text)
response.json.return_value = payload or {}
return Mock(post=Mock(return_value=response))
def test_successful_response_returns_the_translation(self):
session = self._session(True, {"translatedText": "Hallo"})
result = i18n_sync.translate(session, "http://lt", "", "Hello", "de")
self.assertEqual(result, "Hallo")
def test_api_key_is_only_sent_when_configured(self):
session = self._session(True, {"translatedText": "Hallo"})
i18n_sync.translate(session, "http://lt", "secret", "Hello", "de")
self.assertEqual(session.post.call_args.kwargs["data"]["api_key"], "secret")
def test_no_api_key_is_sent_when_none_is_configured(self):
session = self._session(True, {"translatedText": "Hallo"})
i18n_sync.translate(session, "http://lt", "", "Hello", "de")
self.assertNotIn("api_key", session.post.call_args.kwargs["data"])
def test_failed_response_returns_none(self):
session = self._session(False, status=403, text="denied")
self.assertIsNone(i18n_sync.translate(session, "http://lt", "", "Hi", "de"))
def test_a_non_json_success_response_returns_none(self):
response = Mock(ok=True)
response.json.side_effect = ValueError("no json")
session = Mock(post=Mock(return_value=response))
self.assertIsNone(i18n_sync.translate(session, "http://lt", "", "Hi", "de"))
def test_a_transport_failure_returns_none(self):
session = Mock(
post=Mock(side_effect=i18n_sync.requests.ConnectionError("reset"))
)
self.assertIsNone(i18n_sync.translate(session, "http://lt", "", "Hi", "de"))
def test_a_non_string_translation_is_refused(self):
session = self._session(True, {"translatedText": ["Hallo", "Welt"]})
self.assertIsNone(i18n_sync.translate(session, "http://lt", "", "Hi", "de"))
class TestSupportedLanguages(unittest.TestCase):
def _session(self, payload=None, side_effect=None):
response = Mock(raise_for_status=Mock())
if side_effect is not None:
response.json.side_effect = side_effect
else:
response.json.return_value = payload
return Mock(get=Mock(return_value=response))
def test_the_offered_codes_are_returned(self):
session = self._session([{"code": "de"}, {"code": "fr"}])
self.assertEqual(
i18n_sync.supported_languages("http://lt", session), {"de", "fr"}
)
def test_an_unreachable_instance_is_reported(self):
session = Mock(get=Mock(side_effect=i18n_sync.requests.ConnectTimeout("slow")))
with self.assertRaises(i18n_sync.BackendError):
i18n_sync.supported_languages("http://lt", session)
def test_a_non_json_listing_is_reported(self):
session = self._session(side_effect=ValueError("no json"))
with self.assertRaises(i18n_sync.BackendError):
i18n_sync.supported_languages("http://lt", session)
def test_json_of_the_wrong_shape_is_reported(self):
for payload in ({"error": "Slow down"}, ["de", "fr"], None, [{"name": "de"}]):
with self.subTest(payload=payload):
with self.assertRaises(i18n_sync.BackendError):
i18n_sync.supported_languages("http://lt", self._session(payload))
class TestLoadExisting(unittest.TestCase):
def setUp(self):
self.directory = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.directory, True)
self.path = self.directory / "de.yaml"
def test_a_missing_file_is_an_empty_catalog(self):
self.assertEqual(i18n_sync.load_existing(self.path), {})
def test_a_readable_catalog_is_returned(self):
self.path.write_text("Hi: Hallo\n", encoding="utf-8")
self.assertEqual(i18n_sync.load_existing(self.path), {"Hi": "Hallo"})
def test_an_unparsable_catalog_is_refused(self):
self.path.write_text('Hi: "unterminated\n', encoding="utf-8")
self.assertIsNone(i18n_sync.load_existing(self.path))
def test_a_non_mapping_catalog_is_refused(self):
self.path.write_text("- a\n- b\n", encoding="utf-8")
self.assertIsNone(i18n_sync.load_existing(self.path))
def test_a_non_utf8_catalog_is_refused(self):
self.path.write_bytes(b"\xffHi: Hallo\n")
self.assertIsNone(i18n_sync.load_existing(self.path))
def test_a_directory_at_the_catalog_path_is_refused(self):
(self.directory / "sub.yaml").mkdir()
self.assertIsNone(i18n_sync.load_existing(self.directory / "sub.yaml"))
def test_an_empty_or_comment_only_catalog_is_an_empty_catalog(self):
for text in ("", "\n\n", "# only a comment\n"):
with self.subTest(text=text):
self.path.write_text(text, encoding="utf-8")
self.assertEqual(i18n_sync.load_existing(self.path), {})
class TestWriteCatalog(unittest.TestCase):
def setUp(self):
self.directory = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.directory, True)
self.path = self.directory / "de.yaml"
def test_the_catalog_is_written_and_no_scratch_file_is_left(self):
i18n_sync.write_catalog(self.path, {"Hi": "Hallo"})
self.assertEqual(
yaml.safe_load(self.path.read_text(encoding="utf-8")), {"Hi": "Hallo"}
)
self.assertEqual([p.name for p in self.directory.iterdir()], ["de.yaml"])
def test_a_write_that_dies_halfway_leaves_the_previous_catalog_intact(self):
self.path.write_text("Hi: HAND-EDITED\n", encoding="utf-8")
complete = Path.write_text
def dies_halfway(target, data, *args, **kwargs):
complete(target, data[: len(data) // 2], *args, **kwargs)
raise OSError("no space left on device")
with patch.object(Path, "write_text", dies_halfway):
with self.assertRaises(OSError):
i18n_sync.write_catalog(self.path, {"Hi": "machine", "Zebra": "Zebra"})
self.assertEqual(self.path.read_text(encoding="utf-8"), "Hi: HAND-EDITED\n")
class TestSync(unittest.TestCase):
def setUp(self):
self.directory = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.directory, True)
self.addCleanup(setattr, i18n, "CONTENT_DIR", i18n.CONTENT_DIR)
i18n.CONTENT_DIR = self.directory
self.path = self.directory / "de.yaml"
def _run(self, sources, offered=("de",), translated="UEBERSETZT"):
listing = Mock(raise_for_status=Mock())
listing.json.return_value = [{"code": code} for code in offered]
answer = Mock(ok=True)
answer.json.return_value = {"translatedText": translated}
session = Mock(get=Mock(return_value=listing), post=Mock(return_value=answer))
with patch.object(i18n_sync.requests, "Session", return_value=session):
i18n_sync.sync("http://lt", "", set(sources), ["de"])
return session
def test_missing_entries_are_written(self):
self._run(["Hello"])
self.assertEqual(
yaml.safe_load(self.path.read_text(encoding="utf-8")),
{"Hello": "UEBERSETZT"},
)
def test_existing_entries_are_never_overwritten(self):
self.path.write_text("Hello: HAND-EDITED\n", encoding="utf-8")
session = self._run(["Hello", "World"])
catalog = yaml.safe_load(self.path.read_text(encoding="utf-8"))
self.assertEqual(catalog["Hello"], "HAND-EDITED")
self.assertEqual(session.post.call_count, 1)
def test_an_unparsable_catalog_is_left_alone(self):
broken = 'Hello: "unterminated\n'
self.path.write_text(broken, encoding="utf-8")
session = self._run(["Hello"])
self.assertEqual(self.path.read_text(encoding="utf-8"), broken)
session.post.assert_not_called()
def test_a_language_the_instance_does_not_offer_is_skipped(self):
session = self._run(["Hello"], offered=("fr",))
self.assertFalse(self.path.exists())
session.post.assert_not_called()
def test_a_run_that_translated_nothing_leaves_the_file_untouched(self):
original = "# Reviewed by a native speaker, keep the order.\nZebra: Zebra\n"
self.path.write_text(original, encoding="utf-8")
self._run(["Hello"], translated=None)
self.assertEqual(self.path.read_text(encoding="utf-8"), original)
def test_the_content_directory_is_created(self):
nested = self.directory / "content"
i18n.CONTENT_DIR = nested
self._run(["Hello"])
self.assertTrue((nested / "de.yaml").is_file())
if __name__ == "__main__":
unittest.main()

View File

@@ -2,7 +2,7 @@ import unittest
from html.parser import HTMLParser from html.parser import HTMLParser
from pathlib import Path from pathlib import Path
from jinja2 import Environment, FileSystemLoader, select_autoescape from jinja2 import Environment, FileSystemLoader
class AnchorCollector(HTMLParser): class AnchorCollector(HTMLParser):
@@ -20,10 +20,12 @@ class TestNavigationTemplate(unittest.TestCase):
template_dir = Path(__file__).resolve().parents[2] / "app" / "templates" template_dir = Path(__file__).resolve().parents[2] / "app" / "templates"
environment = Environment( environment = Environment(
loader=FileSystemLoader(template_dir), loader=FileSystemLoader(template_dir),
autoescape=select_autoescape(), autoescape=True,
) )
environment.globals["url_for"] = lambda _endpoint, filename: ( environment.globals["url_for"] = lambda endpoint, **kwargs: (
f"/static/{filename}" f"/static/{kwargs['filename']}"
if endpoint == "static"
else f"/{kwargs['lang']}/"
) )
environment.globals["asset_src"] = lambda asset: ( environment.globals["asset_src"] = lambda asset: (
(asset or {}).get("external_url") (asset or {}).get("external_url")
@@ -36,6 +38,9 @@ class TestNavigationTemplate(unittest.TestCase):
rendered = environment.get_template("moduls/navigation.html.j2").render( rendered = environment.get_template("moduls/navigation.html.j2").render(
menu_type="header", menu_type="header",
lang="en",
languages={"en": "English", "de": "Deutsch"},
t=lambda source: source,
platform={ platform={
"titel": "Portfolio", "titel": "Portfolio",
"logo": {"cache": "logo.png"}, "logo": {"cache": "logo.png"},
@@ -70,8 +75,15 @@ class TestNavigationTemplate(unittest.TestCase):
and "dropdown-toggle" in anchor.get("class", "") and "dropdown-toggle" in anchor.get("class", "")
] ]
self.assertEqual(len(dropdown_toggles), 1) self.assertEqual(len(dropdown_toggles), 2)
self.assertEqual(dropdown_toggles[0].get("data-bs-toggle"), "dropdown") for toggle in dropdown_toggles:
self.assertEqual(toggle.get("data-bs-toggle"), "dropdown")
language_links = [
anchor for anchor in parser.anchors if anchor.get("hreflang") == "de"
]
self.assertEqual(len(language_links), 1)
self.assertEqual(language_links[0]["href"], "/de/")
if __name__ == "__main__": if __name__ == "__main__":

225
utils/i18n_sync.py Normal file
View File

@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""Fill missing content translations from a LibreTranslate instance.
Reads the prose strings out of the live ``app/config.yaml`` and writes one
catalogue per language to ``app/i18n/content/``. Entries that already exist are
never overwritten, so a hand-corrected translation survives every later run.
"""
import argparse
import os
import sys
from pathlib import Path
import requests
import yaml
REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(REPO_ROOT))
from app.utils import i18n # noqa: E402
DEFAULT_CONFIG_PATH = REPO_ROOT / "app" / "config.yaml"
REQUEST_TIMEOUT = 30
def collect_sources(node, key=None, found=None):
"""Return every prose string in ``node`` that a backend may translate.
Args:
node: the raw configuration tree, or any subtree of it.
key: the mapping key ``node`` was reached through.
found: accumulator, for recursion.
"""
found = set() if found is None else found
if isinstance(node, dict):
for name, value in node.items():
collect_sources(value, name, found)
elif isinstance(node, list):
for item in node:
collect_sources(item, key, found)
elif isinstance(node, str) and key in i18n.AUTOFILL_KEYS and node.strip():
found.add(node)
return found
class BackendError(Exception):
"""The translation backend answered in a way the run cannot continue from."""
def supported_languages(url, session):
"""Return the language codes the LibreTranslate instance at ``url`` offers."""
try:
response = session.get(f"{url}/languages", timeout=REQUEST_TIMEOUT)
response.raise_for_status()
offered = response.json()
except requests.RequestException as error:
raise BackendError(f"{url} is not reachable: {error}")
except ValueError:
raise BackendError(f"{url}/languages did not answer JSON")
if not isinstance(offered, list):
raise BackendError(
f"{url}/languages answered {type(offered).__name__}, not a list"
)
codes = {
entry["code"]
for entry in offered
if isinstance(entry, dict) and isinstance(entry.get("code"), str)
}
if not codes:
raise BackendError(f"{url}/languages listed no usable language codes")
return codes
def load_existing(path):
"""Return the catalogue at ``path``, or None when it must not be rewritten.
Distinct from ``i18n.read_catalog``, which degrades an unreadable catalogue
to English at render time. Here the same file means the hand-written entries
are unknown, and writing would replace them with a fresh machine pass.
"""
if not path.exists():
return {}
try:
loaded = yaml.safe_load(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, yaml.YAMLError) as error:
print(f" ! {path.name}: {type(error).__name__}, refusing to overwrite it")
return None
if loaded is None:
return {}
if not isinstance(loaded, dict):
print(f" ! {path.name}: not a mapping, refusing to overwrite it")
return None
return loaded
def write_catalog(path, catalog):
"""Replace ``path`` with ``catalog`` in one step.
Writing in place would leave a half-written catalogue behind if the process
is interrupted or the disk fills, and the truncated remainder can still be
valid YAML — the next run would then machine-fill the destroyed entries.
"""
temporary = path.with_name(f"{path.name}.tmp")
temporary.write_text(
yaml.safe_dump(catalog, allow_unicode=True, sort_keys=True, width=1000),
encoding="utf-8",
)
os.replace(temporary, path)
def translate(session, url, api_key, text, target):
"""Translate one string into ``target``, or return None on failure."""
payload = {
"q": text,
"source": i18n.SOURCE_LANGUAGE,
"target": target,
"format": "text",
}
if api_key:
payload["api_key"] = api_key
try:
response = session.post(
f"{url}/translate", data=payload, timeout=REQUEST_TIMEOUT
)
except requests.RequestException as error:
print(f" ! {target}: {type(error).__name__}, {error}")
return None
if not response.ok:
print(f" ! {target}: {response.status_code} {response.text[:120]}")
return None
try:
translated = response.json().get("translatedText")
except ValueError:
print(f" ! {target}: answered 200 but not JSON")
return None
if not isinstance(translated, str):
print(f" ! {target}: translatedText was {type(translated).__name__}")
return None
return translated
def sync(url, api_key, sources, targets):
"""Fill and write the content catalogue of every language in ``targets``."""
i18n.CONTENT_DIR.mkdir(parents=True, exist_ok=True)
session = requests.Session()
available = supported_languages(url, session)
for target in targets:
if target not in available:
print(f"- {target}: not offered by {url}, skipped")
continue
path = i18n.CONTENT_DIR / f"{target}.yaml"
catalog = load_existing(path)
if catalog is None:
continue
missing = sorted(source for source in sources if source not in catalog)
if not missing:
print(f"- {target}: complete")
continue
print(f"- {target}: translating {len(missing)} string(s)")
added = 0
for source in missing:
translated = translate(session, url, api_key, source, target)
if translated:
catalog[source] = translated
added += 1
if not added:
print(f" ! {target}: nothing translated, leaving the file untouched")
continue
try:
write_catalog(path, catalog)
except OSError as error:
print(f" ! {target}: could not write {path.name}: {error}")
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--url",
required=True,
help="Base URL of a LibreTranslate instance, e.g. http://localhost:5002",
)
parser.add_argument(
"--api-key",
default="",
help="API key, if the instance requires one.",
)
parser.add_argument(
"--config",
type=Path,
default=DEFAULT_CONFIG_PATH,
help=f"Configuration to read the prose from (default: {DEFAULT_CONFIG_PATH}).",
)
parser.add_argument(
"--languages",
nargs="+",
default=[code for code in i18n.LANGUAGES if code != i18n.SOURCE_LANGUAGE],
help="Language codes to fill (default: every shipped language).",
)
args = parser.parse_args(argv)
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))
sources = collect_sources(config)
print(f"{len(sources)} translatable string(s) in {args.config}")
try:
sync(args.url.rstrip("/"), args.api_key, sources, args.languages)
except BackendError as error:
print(f"ERROR: {error}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())