Commit Graph
8 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 bac30b76fa fix(ci): clear the findings of the first quality-high pass
Four independent findings, each at its root.

sys-svc-container: tasks/00_core/gpu.yml included 'nvidia.yml' by bare name,
but the role is entered through tasks_from: 00_core.yml, so a relative include
resolves against tasks/ instead of next to the including file. Every sibling
include in 00_core.yml already carries the 00_core/ prefix.

web-app-seaweedfs: c61b29bd16 made services.seaweedfs.enabled and .shared
literal and stated that the variant pin follows, but never touched
meta/variants.yml. Both variants still repeated that literal, which
variants-static-override counts as four offenders. They set exactly what the
base declares, so they go. The base keeps the literal and regains the
dynamic-flag marker the console split dropped at e8e89d293d: the flag's only
job is joining the engine's shared_net, and its false branch yields a console
container that cannot start, which the lint names as a legitimate exception.

docs: the i8-tuner skill landed without its cheatsheet row, which the skills
coverage guard reads as a missing skill.

The remaining three files are that pass's own formatter output: quote
normalisation in nvidia.yml and line wrapping in two test files.

Not covered here: the two hallucinated URLs in locale/ko/LC_MESSAGES/docs.po,
because a translation run is writing the catalogs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 18:51:03 +02:00
kevinveenbirkenbachandClaude Opus 5 09e9f5d41d feat(svc-ai-s1): record how long each option took, not only which won
The verdict table remembered the winner of every comparison and nothing about speed, so the decider could tell which engine translates well and never which one answers quickly. For 183 catalogs the second question matters as much as the first.

verdict gains a nullable took_ms, and record() writes one duration per offered option. An option that did not answer stores NULL rather than zero, which would read as the fastest of them all, and speed() averages over the timed rows alone for exactly that reason. The widening ALTER exists because CREATE TABLE IF NOT EXISTS leaves an existing table untouched: a deployment that recorded wins before the column would otherwise fail every insert from here on.

The router asks for the average under the same signature the history is keyed by, so a duration belongs to a request class rather than to everything at once, and _criteria prefers it over the in-process last-request figure, which a restart forgets. Each wrapper swallows a history error and returns an empty mapping, as _wins and _streaks already do: a broken log costs the decider a fact, never a translation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 13:17:26 +02:00
kevinveenbirkenbachandClaude Opus 5 2d886ad19b feat(svc-ai-nllb-engine): add a translation model that covers 136 languages
The gateway had two engine kinds and neither reaches the catalogue.
LibreTranslate covers roughly 48 languages. The chat engine is backed
by the single entry in AI_LOCAL_MODELS, llama3.2:1b, which officially
speaks eight, while web-svc-translate/vars/main.yml declares every
language in meta/languages.yml as a chat target. The gateway therefore
asserted 183-language capability for a model that invents the rest:
Belarusian came back as Polish in 86 of 87 new entries, Abkhaz had no
Cyrillic in any of 83, Amharic 10 of 236 in Ethiopic, and Avaric,
Assamese and Bashkir returned the source verbatim.

NLLB-200 is built for translation rather than generation. A third
engine kind serves it beside the two libretranslate instances, since
it is seq2seq and does not belong as an ollama alias.

The honest ceiling is 136 of 184, not 184. flores.py maps 136 ISO
codes and names the 48 the model has no code for, and the test asserts
mapped plus unserved equals exactly the keys of meta/languages.yml so
neither side can drift. Belarusian, Amharic, Assamese and Bashkir are
covered; Abkhaz and Avaric are not, so two of the worst catalogues
still have no local engine.

The script half of each mapping is derived rather than typed: the test
re-derives every endonym's script from unicodedata and asserts it
matches the suffix of the FLORES code, ja and zh excepted because Han
cannot distinguish Jpan from Hans from Hant. The subtag half cannot be
derived, because where ISO 639-1 names a macrolanguage NLLB ships one
variety (ar->arb_Arab, fa->pes_Arab, ms->zsm_Latn and ten more), so it
is stated once in one table.

Counting languages surfaced a real bug: meta/languages.yml has 184
entries, not 183. 'no' is quoted because YAML reads bare no as false.

Inert until deployed. The engine entry is multiplied by zero while
TRANSLATE_NLLB_ENABLED is false, so TRANSLATE_ENGINES is unchanged;
the two new vars are evaluated unconditionally and every key they read
exists in the new role. No variant round was added, so CI does not
exercise this engine yet, and a first deploy needs an inventory
regeneration for the new service key, volume and subnet.

Assumed facebook/nllb-200-distilled-600M, about 3 GB resident, which
fits the envelope svc-ai-s1 already occupies. 1.3B buys roughly
+2 chrF++ concentrated in low-resource languages for twice the memory;
the README tables the sizes so that choice is visible.

Owed before this is offered beyond private use: the weights are
CC BY-NC 4.0, and the role does not yet say so or credit the NLLB
team. Tracked with the per-role licence index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-20 19:52:21 +02:00
kevinveenbirkenbachandClaude Opus 5 6dd4d8c0ed style: take the line breaks autoformat wrote
make quality-high reformatted three expressions this branch added today: the chat-target chain in the translate role's vars, an assertEqual in the gateway engine tests, and the wanted comprehension in the lane probe. No logic and no rendered value changes; the YAML break folds to a space inside a double-quoted scalar.

Kept separate from the changes that caused it, because a reflow mixed into a behaviour commit hides the behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 12:52:42 +02:00
kevinveenbirkenbachandClaude Opus 5 12ed8a763c fix(i18n): translate through the gateway and let a backend declare what it serves
make i18n-translate addressed the engine directly, so a run could never reach more than the 47 languages Argos ships models for, while meta/languages.yml names 183. service_url now resolves web-svc-translate, which routes a pair the engine does not serve to a chat model, and the deploy brings both roles up.

The gateway advertised only what its backends declared, and the chat backend declared nothing, so /languages reported 51 targets of 183 and the readiness wait spun to its hour-long timeout without translating a line. The chat backend now declares every language the table names.

Routing offered libretranslate for every pair, because an undeclared catalogue read as any pair, so a language it holds no model for was answered with HTTP 400 instead of falling through to the next backend. serves() now reads the backend's own /languages, which catalogue() already exposed for the advertisement. Declaring the pairs in the role was the first attempt and carried a second copy that was already stale: the engine reports zh-Hans where the role's list says zh.

A small model copies the prompt markers back, echoes the source line and labels its answer with the target language. All three reached the catalog, and an entry carrying its own source scores below the oracle's middle level. clean() strips them, and only from an answer that carried scaffolding, because on its own a short word and a colon is a translation, not a label.

server() had lost its redeploy parameter while cli/build/i18n kept passing it, so every run raised TypeError before translating a line and the only symptom was an empty log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 09:47:53 +02:00
kevinveenbirkenbachandClaude Opus 5 c33d025bab fix(web-svc-translate): judge an engine on the markup it returns, not only its terms
The gateway masks protected spans as `<x id="N"></x>` before it asks an
engine, so an engine that drops or invents that markup returned an answer the
catalog client later rejected as `structure`. `_mangled` only checked that the
protected terms survived, which let a mangled skeleton count as a win and be
recorded as one. It now compares the markup characters as a multiset between
source and answer: reflowing is allowed, losing or inventing a bracket is not.

Measured on the `ko` catalog, 2854 entries, as the three parts landed: 1
translated entry with 2704 `structure` rejections on the direct engine path, 17
with 2649 once the gateway checked the skeleton, 331 with 796 once a rejected
entry could be re-asked against another engine. `candidates` therefore takes
`exclude`, so a retry reaches a different backend instead of the same one.

`_judged` is split out of `_choose` for a second reason. A decider that timed
out escaped as a 500 and threw away replies already in hand; it now degrades
to None, the fallback winner still answers, and no history row is written.
Charging the fallback engine a win the decider never awarded would let
`saturated` freeze a signature on the configured order and stop the learning
the log exists for. Every reply is still cached; only the row is conditional.

`format` becomes a parameter of `Engine.translate` instead of the hardcoded
"text" that would have destroyed every masked entry, `catalogue()` reads the
pairs a backend reports itself, and `fmt` enters the cache key so a text and an
html answer for one string cannot alias.

The canonical root answers a JSON body naming the service instead of
redirecting to /docs, and accepts HEAD. 90c448c747 introduced that redirect;
meta/domains.yml gives this role the canonical domain api.translate, and
sys-ctl-hlth-csp runs csp-checker against every deployed domain, so the
redirect sent the checker into the Swagger UI whose inline scripts violate CSP.
A HEAD against the root answered 405 and a GET 307, neither of which is a
usable probe result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-05 17:06:55 +02:00
kevinveenbirkenbachandClaude Opus 5 4e5b3ce26f fix(web-svc-translate): drop a search field Weblate cannot parse
WeblateMemory.reviewed appended `AND source_language:{source}` whenever the caller declared a source language. Weblate has no such unit search field and answers `Could not parse query string: Unknown search field: source_language` with 400, so the memory could never return a hit. Nothing surfaced it: Gateway._reviewed swallows every exception by design, so each request fell through to the engines as if the memory simply held nothing.

The clause is removed rather than corrected, because Weblate exposes no unit-level equivalent; the exact source match plus the target language and the approved state already pin the unit. The source language stays on Gateway.translate, which the router needs.

Two unit tests asserted the broken clause and passed, because the FakeClient records the query without ever putting it to Weblate. One is now a guard that fails on any field outside the set Weblate parses. Verified against the running instance: the old query answers 400 and the new one returns the seeded approved unit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-05 01:06:49 +02:00
kevinveenbirkenbachandClaude Opus 5 ffd82294fc feat(translate): serve translation through one gateway with memory and a glossary
Requirement 040. web-svc-translate fronts the translation engines so a consumer talks to one endpoint instead of binding an engine. Its router picks per request and drops an engine after three consecutive failures until it recovers, and the strategy follows the deployment rather than a constant: with the S1 decider present it ranks by system-one judgement, otherwise by fixed preference.

Protected glossary terms travel into the engine instead of being checked afterwards, and an answer that mangles one is refused before it reaches the cache. Weblate supplies the translation memory and the glossary, and reads its admin API token out of the running instance into the token store rather than minting a second one.

svc-ai-ltengine builds LTEngine from source as a local engine. Discourse and WordPress consume the gateway as addons under the unified contract, each deriving its enabled flag from the service flag and shipping its own Playwright spec.

The decider moves out of svc-ai-litellm into roles/svc-ai-s1/files/python/infinito_decider so the gateway and the model router share one audited copy; litellm stages and mounts it with the same play that renders the hook importing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-03 20:45:39 +02:00