kevinveenbirkenbachandClaude Opus 5 b005151a4f test: split the semantic suite by what decides its verdict ...

`tests/semantic/` mixed three kinds of check under one name. They are now
placed by what returns the verdict:

- `tests/oracle/` holds only the checks a model decides, and `make
  test-semantic` becomes `make test-oracle`
- `tests/external/i18n/` holds the back-translation check, which needs a
  served engine
- `tests/integration/i18n/` holds the catalog scan, which reads the `.po`
  corpus and needs nothing served

The shared reachability helpers move to `tests/utils/services.py`.

Two checks were reading `auto_comments` while `utils/i18n/translate.py`
writes the marker to `user_comments`. Over the `core` corpus that is 0
matches against 19,225, so both selected an empty set and passed having
judged nothing. The back-translation check also addressed the engine by
its pre-rename service key.

The model-pin check treated a 200 that carries no file digests as proof
the pinned file was withdrawn. An intermediary that answers with the file
list but drops the `blobs` expansion produces exactly that, so a good pin
went red; the comparison has no input in that case and the pin is now
reported unverified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-28 23:51:20 +02:00

..

ci

feat(ci): spread the deploy matrix over amd64 and arm64

2026-09-19 07:02:44 +02:00

__init__.py

…

bare_bootstrap_probe.py

fix(tests): purge preloaded modules before blocking imports

2026-08-10 00:40:21 +02:00

minitest_junit.rb

feat(test): report the javascript, php and ruby suites in the CI summary

2026-08-22 22:27:12 +02:00

services.py

test: split the semantic suite by what decides its verdict

2026-09-28 23:51:20 +02:00