fix(i18n): refuse a translation another entry already holds

What is wrong with the answers the collapse check reports is not their
shape. A label is caught by its length, but `sq` shares a translation with
four sources at 0.84 of their length, `ga` at 0.73 and `he` at 0.62: full
renderings, spent twice. No length rule separates those from a language
that is simply terse, because the fault is that another entry already says
it, which a predicate reading one entry cannot see.

The writer now carries the translations a catalog already spends on a
source long enough to deserve its own, and turns down a repeat. It lives
where the whole catalog is in hand; `apply` alone sees one chunk. Both
arguments default to None, so a caller without the catalog behaves as
before.

The floor is now inclusive, matching the check it mirrors. `az` collapsed
on sources of exactly forty characters, which the check counts and the
first version of this guard skipped.

`criteria` is threaded through so the run names what it refused. A
duplicate is turned down here rather than in the client, and the summary
reported the discard with no reason for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed 2026-09-29 08:47:16 +02:00
1 parent 9711a4cf89
commit 33880078e9
3 files changed
+122 -5

No files matched your search

+53 -1
View File
@@ -20,7 +20,14 @@ from utils.i18n.catalog import (
)
from utils.i18n.libretranslate import LibreTranslate
from utils.i18n.placeholders import Rejected
from utils.i18n.translate import URL_SUPPRESSION, apply, damaged, pending, retry
from utils.i18n.translate import (
URL_SUPPRESSION,
apply,
damaged,
pending,
retry,
taken_translations,
)
TOKEN = re.compile(r'<x id="\d+"></x>')
@@ -318,5 +325,50 @@ class TestDamagedMarkup(unittest.TestCase):
self.assertEqual(damaged(catalog), [])
class TestDuplicateTranslation(unittest.TestCase):
LONG_A = "Central identity provider issuing tokens for every service"
LONG_B = "Issue and project tracking platform for software teams today"
SHORT_A = "Save the file"
SHORT_B = "Open the file"
def _messages(self, *sources: str) -> list:
catalog = Catalog(locale="de")
for source in sources:
catalog.add(source, "")
return [message for message in catalog if message.id]
def test_a_second_long_source_may_not_reuse_a_translation(self) -> None:
messages = self._messages(self.LONG_A, self.LONG_B)
discarded = apply(
messages,
["Beschreibung eines Dienstes", "Beschreibung eines Dienstes"],
set(),
)
self.assertEqual(discarded, 1)
self.assertEqual(str(messages[1].string), "")
def test_short_sources_may_share_one(self) -> None:
messages = self._messages(self.SHORT_A, self.SHORT_B)
discarded = apply(messages, ["Speichern", "Speichern"], set())
self.assertEqual(discarded, 0)
self.assertEqual(str(messages[1].string), "Speichern")
def test_without_the_set_the_check_is_skipped(self) -> None:
messages = self._messages(self.LONG_A, self.LONG_B)
self.assertEqual(apply(messages, ["Gleich", "Gleich"], None), 0)
def test_what_the_catalog_already_spends_is_seeded(self) -> None:
catalog = Catalog(locale="de")
catalog.add(self.LONG_A, "Schon vergeben")
catalog.add(self.SHORT_A, "Kurz")
self.assertEqual(taken_translations(catalog), {"Schon vergeben"})
if __name__ == "__main__":
unittest.main()