The gateway routes every request on its own, so a catalog holds the work of several engines, while the mark apply() wrote was the constant translated-by: libretranslate and attributed all of it to one backend. The catalogs on disk show the drift: 1021537 entries carry that constant and 13 carry ollama, written once the gateway began answering from more than one place.
The constant becomes machine_translation(engine), which names what the gateway reported and writes unnamed where it reported nothing, plus is_machine_translation(), which recognises any of them by prefix so every mark already on disk keeps matching. Outcome carries an engines tuple aligned with its values, merge pads it, and _reask returns the name alongside the text: a successful re-ask records the engine that answered, not the one whose rejection it replaced, because the catalog does not carry that rejected text.
apply() now strips an earlier mark before stamping the current one. clear() removes only refusal prefixes, so an entry re-translated after i18n-prune would otherwise accumulate one mark per engine and the record would be ambiguous again, which is the defect this change exists to remove.
This is the half that makes the attribution true. Recording latency in the decider history is the other half; together they let the router weigh quality against speed per engine, which a catalog claiming one engine everywhere cannot support.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A comment is a nocheck directive or a parameter doc, and an Exception: label does not make narration into one of them. Four comments go entirely: the seven-line block on why Avestan is absent, whose reason stands in the body of 5faa4d1aec; the anchor-directory note in the package-frontend CA script, which the two paths directly below it already state; and the Sphinx budget note in the docs Playwright env, where a # line is rendered into the .env rather than staying source.
Seven Exception: paragraphs inside docstrings keep their prose and lose only the label. A docstring is a parameter doc and may carry the text; the defect was a prefix claiming a sanction the policy does not grant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy job name repeats the category path on every row of the matrix. One Egyptian hieroglyph costs four UTF-8 bytes where the two Chinese characters it replaces cost six. Unicode names each sign by its Gardiner code, so the opaque value stays identifiable without a comment per entry.
A label has to stay one codepoint: a sign written with a variation selector costs seven bytes and is worse than what it replaced. tests/integration/meta/categories/test_glyph.py enforces that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
reason() calls itself the single source of truth for both ends of the
pipeline, and the write path honoured that: client.py:228 passes the
directory code into unmask(). Only damaged() read catalog.locale, so
for 50 catalogs the two ends judged against different languages.
44 catalogs carry no locale header, 8 of them non-Latin and therefore
silently past the script check. 6 more name a code their directory
disagrees with, so tl was judged as fil, bh as bho, tw as ak.
The language becomes a required parameter; both production callers
already held the code. Measured over every catalog, header language
against path language, one moves: core/av from 0 to 29. Those 29 hold
pseudo-Norwegian and a degeneration loop in a Cyrillic catalog, and
make i18n-prune has since cleared them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4915a401d4 made a failed swarm row replay its mesh state, so the
priority line this test rebuilds says &direct. The test asks for the
exact failed selection, and the pin is what makes it exact.
It went unnoticed because only the unit suites were run for that change
and this one lives under tests/integration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scripts/meta/resolve/diff/affected_roles.sh emits __ALL__ as soon as
one changed path outside roles/<role>/ is not a .md or .rst file. A
role fix that also adds its unit test under tests/unit/python/roles/
therefore deployed the whole matrix: PR #982 changed
roles/web-app-zammad/ plus
tests/unit/python/roles/web-app-zammad/files/test_search_allocation_precheck.py,
and its "Detect affected roles" job logged "Resolver output: __ALL__".
Paths under tests/ now join the .md/.rst skip-list. No role deploy
reads the top-level tests/ tree, and those suites run in their own
jobs whatever the whitelist says. A diff of only skipped paths keeps
falling back to __ALL__, as before. The same resolver feeds
effective_whitelist.sh, so the narrowing applies to every diff-driven
run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
meta/languages.yml carried `libretranslate: true|false` per entry, which
answered two different questions with one field: which languages the
repository translates, and which engine renders them. The first is the
table's business; the second is not. The gateway offers every backend
that serves a pair, falls through when one refuses or mangles the
answer, drops an engine whose failures pile up, and settles a request
class on the engine that wins its comparisons - and `signature()` is
"{source}:{target}:{length bucket}:{markup|plain}", so that learning is
already per target language. A static column beside it is a second
opinion that goes stale the moment the router learns something.
So the field is gone, 184 lines, and the table now holds name, endonym
and direction. Every language it names carries a catalog in every
domain.
What one engine can do is declared with that engine instead:
services.libretranslate-engine.serves, 47 codes. Not a detour - both
readers run before any engine answers. The lane prefetches one model per
code, and the readiness probe is itself the thing that would have asked.
`translatable()` split along that seam. Catalog-scoped callers take
`domain_languages()`, engine-scoped ones take the new
`served_languages()`: the CI matrix, extraction and translation the
first, the model prefetch and the tuning sweep the second. Collapsing
the two is what this change is guarding against - `baked_languages()`
renders identically before and after, verified by diffing both outputs.
`translate()` also stops dying on a language whose catalog has not been
extracted yet. It reads a missing catalog as nothing pending rather than
raising, because 136 docs catalogs do not exist until the next
extraction and refusing those languages would quietly shrink the set the
repository promises to translate.
The oracle suite follows: tests/oracle/test_translation_score.py asks
utils/i18n/judge.py about every language instead of routing a laya
checkpoint per language. It cannot run yet - the tools lane carries no
LiteLLM - and its skip message says exactly that.
test_catalogs_complete_on_gpu now measures against the table rather than
one engine's coverage. It is live on a GPU host and will report the 136
missing catalogs as the gap they are, which is the point: narrowing it
to the engine would hide exactly the languages worth watching.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ImageRef.name drops the registry host so a role's image can be compared
across sources, and both integration sweeps handed that bare name to
their probe. A probe reaches docker.io with it, where an image published
only on ghcr.io or quay.io does not exist, and read the 404 as "cannot
tell" rather than as a wrong question. ImageRef.addressable puts the
registry back in front for the two callers that address a registry.
It also repairs the mirror read path rather than only silencing 404s.
GHCRProvider.image_base always published under {registry}/{name} while
the reader asked under docker.io/... for every foreign image, so the
mirror hit rate rises and the sweep's own mirror-coverage assertion gets
easier, not harder.
What the corrected probe exposed is the rest of this commit: five
services whose image ships no HEALTHCHECK and whose role declared none,
so nothing could tell whether the container serves.
svc-ai-litellm, web-app-homeassistant and web-app-mini-qr get a real
probe, each in the flavor its own image can run. litellm uses
http_status on health/liveliness: /health and /health/readiness both
require auth and readiness pings the database on every beat, which would
couple the gateway's health to a postgres blip. The wget flavor is
deliberately not used there although the image ships wget, because
--spider sends HEAD and FastAPI sets APIRoute.methods from the declared
methods without adding HEAD, so a HEAD to a GET-only route answers 405
forever. mini-qr is the mirror image of that argument: busybox only, no
bash and no curl, and its busybox wget offers -Y on/off rather than
--proxy=off, which is what the http flavor exists for.
sys-ctl-hlth-csp and test-e2e-playwright are marked instead. Both run as
`container run --rm` and exit with their work, and neither role renders
a compose service that could carry a probe. A probe there would be worse
than useless: sys-ctl-hlth-container greps `container ps --filter
health=unhealthy` with no compose-project filter, so a never-passing
healthcheck on a minutes-long ad-hoc container raises a host-wide alarm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unmounted VOLUME does not fail: docker creates an anonymous volume for
it, the container works, and the gap shows up later as a volume nobody can
attribute. sys-ctl-hlth-volumes fails the deploy over exactly that, and its
whitelist keys on volume ids, which are new on every deploy, so the gap
cannot be suppressed there either. e0b973f9bb fixed one such case in
web-svc-xmpp by hand; this is the check that would have found it.
Attribution is the whole difficulty. meta/services.yml declares an image
under a key, and the compose file often runs it under another name:
web-app-jitsi declares the image under 'jitsi' and renders the service as
'web', web-app-erpnext runs one image as seven services, and the two RDBMS
consumer keys name no service of their own because the container comes from
sys-svc-rdbms. The sweep therefore unions the entry key with its
proxy.service_key and delegates the RDBMS keys to their provider role. A
first version keyed on the entry alone and reported six roles that mount
every path correctly, web-svc-xmpp among them.
Reading a config blob section is now its own module beside the registry
client, because the two readers were the same walk twice and registry.py
sat three lines under the file-size cap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ansible defines it on every play, like playbook_dir already on the list. The
wireguard CLI test reads it to run the mesh CLI with the controller's
interpreter.
_mask_stdin_subblocks states why it blanks a stdin sub-block: Ansible passes
stdin to the spawned process through a pipe, not through the shell, so the
injection this rule guards against cannot happen there. Its regex only matched
the block forms, stdin: | and stdin: >, so the inline stdin: "{{ … }}" stayed
visible and the rule demanded | quote on it.
That demand is wrong twice over. The value is no shell argument, and quoting it
would travel verbatim down the pipe into the target program.
The suite is red at HEAD for this reason: eight findings, six of them nothing
but a file name carrying the token, as in lookup('file',
'php/realign_admin_password.php'), and two inline stdin values. Masking the
inline form the same way the block form is already masked takes all eight,
without a nocheck on any of them.
The mask stays narrow, checked against a command line that follows an inline
stdin on the next line: that command is still scanned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The design queue skipped a hand-kept UI_LESS_ROLES list in
cli/meta/roles/design. The decision now lives in each role: a web-app- or
web-svc- role without a UI to design sets design.enabled: false in its
meta/services.yml, and design_disabled() reads that flag. web-svc-design,
the provider, stays out by its entity name.
A new lint, test_design_off_reason.py, requires a design entry in every
web-app- and web-svc- role and, where the entry is switched off, a
"# nocheck: design-off Reason: <text>" on the design: line or the comment
lines directly above it. The rule is in the suppression catalog, and
requirement 037 decision 13 describes it.
Twelve roles switch design off with a reason checked against their code:
asset and simpleicons were already off; file, onlyoffice, bridgy-fed and
mig were on and no longer receive the shared CSS and design script; cdn,
collabora, coturn, legal, mirror and xmpp had no entry. The four that were
on drop their bond and their design variant overrides, which the
disabled-bond and variants-static-override lints reject on a literal false.
The queue lists 77 candidates; HEAD's list also counted bridgy-fed, mig,
collabora and onlyoffice. Cosmos diagrams and the integration matrix are
regenerated.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Application roles used the host administrator account.
user-administrator/tasks/00_core.yml:17 hashes its password into the OS
user, files/administrator grants that name ALL=(ALL) ALL without
NOPASSWD, and rotate.py lists it in ROTATION_EXEMPT, so the value is
password-authenticated sudo that cannot be rotated to recover. Every role
that typed it into a web form was typing the host sudo credential, and the
staged Playwright env carrying it is written mode 0644.
roles/user-administrator/meta/users.yml now declares platform_administrator
beside administrator: no host account, no rotation exemption, mailbox and
identity so it stays SSO-capable and can receive the mail applications send
to their admin. Every use moves to it, all fields and not only the
password: leaving .email behind built accounts from both identities at
once, with the host operator's address under the platform account's login
name, and an IdP matching on e-mail resolves such an account back to the
host operator.
The account is reachable by two spellings, and both are migrated:
lookup('users', 'administrator') in its two- and three-argument forms, and
the dotted lookup('config', application_id, 'users.administrator.email')
that five roles used. administrator is now referenced nowhere outside
roles/user-administrator.
Four lints hold the boundary:
- no role outside user-administrator may use the host administrator, in
either spelling or as a *user_key literal
- a role that looks a user up must declare it in its own meta/users.yml,
because required_user_policies only pins passwords for users declared
by a role resolved into the inventory, and those are the invokable
roles; a user declared only by a non-invokable role keeps the
{{ 42 | strong_password }} fallback, which re-renders per evaluation
- a role that iterates a user or passes it as a *user_key must declare it
- the biber and mapache test personas may be read only from the
Playwright and CLI test surfaces
Two credentials that were derived by hashing the admin password get their
own declaration, because deriving them tied an unrelated rotation to a
password change: web-app-fider jwt_secret, which signs session and e-mail
sign-in tokens, and web-app-nextcloud integration_suitecrm_client_secret,
whose two ends must be re-registered in lockstep.
web-app-keycloak separates the two admins it has: a new meta/users.yml
declares keycloak_bootstrap for the temporary account the container creates
on first start, while the permanent account is platform_administrator. Both
resolve in vars/main.yml so the lookup runs once per role.
web-app-semaphore's break-glass admin no longer shares ansible_become_password
and no longer forwards its mail to the host administrator; tasks/02_breakglass.yml
realigns the password on every deploy, so it converges on its own.
The six roles whose meta/secrets.yml declared administrator_password as its
only credential lose the file.
tests/integration/ansible/lookups/test_declared_users_resolve.py runs the
lookup through the real plugin loader and fails on a declared user that
cannot be resolved or whose username, e-mail or password renders empty. The
two declarations reading ansible_become_password resolve to an empty string
when that variable is absent, and nothing raises on it.
Existing deployments keep the admin account their application database
already holds, under the old name. Roles that re-apply the credential every
run converge; 30 roles feed it from config with no provisioning task and do
not, so those accounts need a reset. web-app-friendica additionally guards
on "'administrator' not in stdout", which the substring platform-administrator
satisfies, and web-app-n8n initialises an owner that already exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merging main dropped web-app-nextcloud's host-derived php sizing, so the
render test died on the key it still asserted:
KeyError: 'memory_limit'
tests/integration/ansible/lookups/test_resource_in_applications_render.py:79
The test is kept rather than deleted. What it guards is independent of which
consumer it reads: that the lookup resolves inside get_merged_applications
without pulling the payload back through the templar, and that an
unmeasurable value aborts instead of rendering an empty string where a wrong
memory limit does the most damage. Neither is observable against a hand-built
scope, which is why this test drives the real loader.
svc-opt-swapfile still sizes from host_mem_mb and still should: a swapfile
belongs to the host, not to a container, so the move to container limits does
not reach it. It carries the assertion alone now; the nextcloud reads, the
constant naming it and the "both consumers" in the test name go with them.
The second test, which proves an unmeasurable host aborts the render, is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Of the 76 open code scanning alerts, 39 point at code that can change.
svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
Percent-encoding keeps a separator inside its segment, but "." and ".."
need none, so an upstream that resolves dot segments could answer from a
path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
a long run of digits from every start position.
Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
uses select_autoescape now, as six test modules and
utils/templating/jinja.py already did. It escapes templates named
*.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
_SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
lookup test imports its module once.
Closes#807
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts, and no file touched by both sides. Incoming is a run of
Docker image version bumps plus one CI change, which moves the diff
baseline from a force-pushed mirror of main to a fetch in each consumer --
entry-push-latest.yml and the resolve scripts, none of which this branch
touches.
Gates on the merged tree: unit 6199, lint 525, integration 330, external
19, ansible-lint clean, markdownlint clean, ruff clean.
The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.
Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.
baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.
The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.
Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.
test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.
Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.
Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.
baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.
The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.
Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.
test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.
Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the generated role table of README.md. Both sides changed
the same rows: upstream swapped WordPress's Mailu dependency for Stalwart,
while this branch replaced the three svc-net-wireguard-{core,plain,
firewalled} rows with the single svc-net-wireguard it deploys.
Resolved by regenerating rather than by hand, since make readme-index owns
that table and gates it with --check. The regenerated table carries both
sides and the check reports it up to date.
README.md was the only file both sides touched, and nothing in the tree
still names the three replaced roles.
Fourteen real conflicts plus the 66 generated READMEs.
utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.
plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.
roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.
group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.
docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.
inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.
roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.
The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deprecated roles are no longer tested, but Stalwart variant 1 still switched its mailu service key on so CI co-deployed Mailu for the migration e2e. The override and the static mailu key, whose only purpose was that pull, are gone, and the generated cosmos drops the mailu node.
The migration gate keeps only the check that services.stalwart.migration.import_mailu stays declared and off by default. The operator-side import (12_import_mailu.yml, migrate_from_mailu.py) still runs when web-app-mailu is in the inventory; the e2e harness under files/test/ and templates/test.env.j2 now skips itself and is left for removal.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
entry-push-latest.yml lets a push to main cancel the running pipeline only when its head commit starts with the release subject pkgmgr writes. The new integration test reads that prefix from the workflow and the tag schema from scripts/github/resolve/release/tag.sh, then requires every release tag from v13.0.0 on to sit on "Release version X.Y.Z". Six tags between v10.1.0 and v11.6.0 carry "Release X.Y.Z", so the check starts at the first tag of the uniform run.
call-test.yml now checks out with full history and tags, the pair detect-release already relies on; a depth-1 checkout carries no tags and would skip the test. Forks and tagless clones skip it on purpose.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A push to main whose head commit starts with "Release version " (the subject pkgmgr writes for every release) now cancels the main run in progress, so the release starts at once instead of queueing behind it. Every other push to main still leaves the running pipeline alone.
When the pushed main commit carries a vX.Y.Z tag, the orchestrator runs with chunk_gate false: every chunk deploys and reports, and a failed chunk still turns the run red, so release-guard and release-version stay skipped.
entry-cancel-superseded.yml mirrors the new rule behind its push guard, parenthesised so && does not bind the release clause; the mirror test composes the expected rule the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entity name keys a role's instance directory, its compose project and its container name, and was derived only by stripping the longest matching category prefix. Two roles under different categories whose names share a suffix therefore collapse onto one entity and overwrite each other's rendered stack, which no provides: can repair.
A literal entity_name in vars/main.yml now wins over the derivation. web-app-seaweedfs-console takes its shorter name back and declares the entity seaweedfs-console, which frees the seaweedfs key in its own services map for the engine consumer and removes the provides:, covers: and alias entries that stood in for it. A new lint pairs the declaration with the primary key it has to match, because a drift between them would drop the role from the provider set in silence.
Compared across all 288 roles, exactly one entity changes, and it restores what main already keeps on disk, so no instance directory moves.
The function and the filter lose their get_ prefix. Two lints read that prefix as the discriminator between the variable and the filter that fills it; both discriminate on the pipe now. Ansible's templar resolves a filter and a same-named variable independently, verified against ansible-core 2.20.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two concerns, staged together on request.
Variant pins
------------
A matrix round hands every role the round index and clamps to variant 0 when
the role has fewer entries, so a cross-role pairing holds only while two
independently maintained variant lists agree on an index, and it breaks
silently once the partner runs out of variants.
'# variant-pin: <role>#<index|reciprocal>' above a service entry states the
dependency instead of inheriting it. It is a comment, so it never reaches the
merged services map or host_vars the way a YAML key under the service would.
'#reciprocal' resolves to the partner's variant that switches this role back
on, so neither side tracks the other's index. Two roles pinning one partner to
different variants raises VariantPinConflictError before an inventory is
written.
web-app-openproject is the first user: a fourth variant carries the nextcloud
round, pinned reciprocally. The fourth round exists because folding it into
variant 2 next to gitlab was measured at 35.58 GB against the 32 GB
reservation cap, and it has to be the last round because a role with fewer
variants falls back to variant 0 in a later one.
Measured: the pin changes no role's index in any of the 14 rounds today, so
the mechanism sits in the deploy path without moving a single inventory.
OpenProject addons
------------------
c83c00da7d gave the role meta/addons/ and three task hooks, but the role never
included tasks/addons/ at all, so all three hooks were dead files. 05_addons.yml
now resolves the web host and container id and runs the hook of every addon
whose gate resolves true.
gitlab_integration is provisioned: a bot account, its API token and the plugin
setting carrying gitlab_user_id and webhook_secret. The GitLab-side webhook is
not created here and the hook prints the URL to register it against.
storages keeps an empty hook deliberately. web-app-nextcloud owns both halves
of that integration and runs eleven positions later in the deploy order, so at
this hook's dispatch point the storage it would touch does not exist yet.
openid_connect stays off: OIDC is Enterprise-gated at this version.
Probing the pinned openproject/openproject:17 image corrected two defects
before they could ship: the API token was minted without the name the script
fetched but never used, and the Playwright webhook probe omitted the
X-Gitlab-Event header that HookHandler dereferences before any auth check, so
the smoke request answered 500 where the spec demanded less than 500.
The addon specs now assert the integration rather than the route: storages
requires a registered storage addressing the partner Nextcloud, and
gitlab_integration requires the bot account the webhook authenticates as.
Deploying this needs a regenerated inventory: gitlab-web-app-openproject is a
new user key and users_generator pins it at inventory build time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.
Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
Moving the mem_limit exemption out of a Python dict and into the variants
themselves left two lint violations behind in the test that reads them.
The meta path was spelled out twice, which test_file_spot rejects: a rename of
the role meta layout would turn both into silent misses. Both now resolve
ROLE_FILE_META_VARIANTS from utils.roles.mapping.
Both reads went straight to disk, which test_no_uncached_file_reads rejects.
utils.cache.files.read_text shares one decoded read per file across the pytest
session instead of re-reading every variants file.
No behaviour change: the same spans are computed from the same markers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard already checked that every <NAME>_SERVICE_ENABLED flag a role
declares is gated by a service-gating helper in its spec. It now also checks
the reverse: every service a role's own specs gate through one of the STRICT
helpers must have its flag declared in that role's env template. Those helpers
raise on a service the registry does not know, so an undeclared gate fails the
scenario at runtime, and in a swarm round that costs a two-hour job to learn a
typo. The tolerant safeSkipUnlessEnabled / safeIsEnabled pair stays exempt,
since reading an unknown service as disabled is exactly what it is for.
Both directions enumerated a role's specs with spec_path.parent.glob('*.js'),
and glob does not recurse, so the whole files/playwright/addons/ tree was
invisible - the directory the addon work lives in. They use rglob now, fed
through iter_project_files from utils.cache.files, because a raw project-tree
walk in test code is refused by its own lint; that keeps it at one filesystem
walk per session instead of one per role. skipUnlessServiceDisabled is added to
both helper patterns: it is used live in web-app-openproject's gitlab_integration
spec and was named in neither.
Three '# nocheck: playwright-service-gate' markers in web-app-matrix's env
template justified themselves with 'which this check does not scan'. That is no
longer true, and measuring rather than assuming shows they are not merely
mis-worded but obsolete: with the markers removed the test is green, because the
gates inside the addon specs are now credited directly.
Residuals: the scan reaches 60 addon specs it had never seen, so green means
'all of them declare their flags' only as of this run. And addon-gating.js
returns false for an absent env key where service-gating.js throws, so a
mistyped addon id still skips silently and no lint covers that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-discourse's variant 1 turns bigbluebutton, jitsi, mastodon, matrix and
mattermost on, and that round reaches web-app-matrix, whose own variant 1 pins
gitlab true. The footprint therefore transitively bills gitlab's webservice,
migrations and gitaly - about 10 GB of mem_limit discourse never asks for - and
test_no_variant_exceeds_the_memory_budget reports 65.36 GB against a 64 GB cap.
54 placements were measured; none keeps every variant under it, and the best
alternative only moves the overage to web-app-nextcloud v12 at 65.76 GB.
The exemption first lived in a MEM_LIMIT_EXEMPTIONS dict keyed by
(role, variant index) inside the test. That was a second suppression mechanism
beside the repository's own, and it sat where nobody editing the pins would
look. It is now a '# nocheck: mem-limit-budget' marker inside the variant's own
entry in meta/variants.yml, read through suppressed_line_numbers from
utils.annotations.suppress like every other suppression, and registered as a
rule in suppression.md. A new _variant_spans helper maps each top-level entry
to its line range so the marker scopes to one variant rather than the whole
role.
Nothing is given up by the move: the marker still needs text after it or it
grants nothing, mem_reservation stays enforced as a standalone condition at
30.28 of 32 GB, and test_every_mem_limit_exemption_is_still_needed now collects
the marked variants from the annotations instead of the dict, so it still fails
once a marked variant fits again.
Residual: the marker records no figure, so the overage behind it can grow
without either test noticing. The cap is switched off for that one variant
rather than raised to a measured ceiling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-seaweedfs becomes two roles. web-svc-seaweedfs keeps the storage
engine: the container, the seaweedfs_data volume, the per-consumer S3
identities and the single canonical domain api. web-app-seaweedfs-console
takes the filer and master web UIs into its own compose stack behind the
admin-only oauth2 proxy, reaching the engine over the shared object-store
network. A node can now run the store without the consoles.
The console needs a distinct role name because get_entity_name strips the
category prefix: both halves would have resolved to the entity seaweedfs and
collided on the compose project, the instance directory and the container
name.
The filer and master ports move into the engine's services.seaweedfs.ports
.internal as their single owner; 01_init_consumer.yml consumes the master
port and would otherwise have resolved it only by role-var leakage from the
console, breaking bucket provisioning on exactly the console-less node the
split makes possible.
The engine's api vhost gains the prometheus locations. It never had them: the
override satisfied the cross-cutting-include lint only because the strings sat
in the SSO branch that the api domain never took.
Consumers, the objstore lookup and the catalogs follow the role id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.
So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.
`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.
Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.
`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.
Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.
The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 36651124458 jobs 109692286727 and 109692287139 both report
web-app-discourse: browser observed no successful (2xx/3xx) response from
[matomo...] for resourceType(s) [script] ... Observed loads: []
once over Tor and once over clearnet, so the transport is not the cause.
discourse declared no order against matomo and loaded first, with the tracker
script not yet serving when its pages rendered. `run_after: [web-app-matomo]`
sits between `proxy:` and `lifecycle:`, the shape and position the other 39
matomo consumers use.
The guard next to `test_dashboard_is_loaded_after_matomo` matters because the
edge is otherwise protected only by a swarm row: compose rows are blind to the
tracker contract, so a later reshuffle would regress silently on every one of
them. Without the edge it reports `54 not less than 50`, which is the defect
itself: discourse ahead of the tracker.
b0c9391ffd once removed a `run_after` from this role because it produced the
opposite order and crashed the deploy with "No such container: discourse",
wordpress needing the container during its own play. That hazard does not
return: with this edge the order is matomo 53, discourse 55, wordpress 87.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apply() already refuses a translation another entry holds, on the same floor the release gate reads, so within one process the second sharer is discarded. Four entries of docs/vi nevertheless shared one msgstr: one_catalog reads the catalog once, keeps what it has spent in memory and writes the whole file back after every chunk, so two concurrent runs each hold a private view and overwrite each other.
exclusive() takes an flock on build/i18n/<domain>-<code>.lock around the whole read-modify-write. It tries the lock non-blocking first so a second run can say it is waiting instead of looking hung. The lock files sit under build/, which .gitignore already excludes.
The damage could not be repaired either: prune ran damaged(), which reads one entry at a time, and a collapse is only visible across entries - each one is individually sound. over_shared() groups the machine translations by what they say and returns every sharer beyond MAX_SHARING_SOURCES. prune clears those first, so damaged() no longer sees them and the printed counts cannot overlap. Measured on the corpus: 10 in docs/vi, 132 overall.
MAX_SHARING_SOURCES lands in damage.py next to COLLAPSE_FLOOR and the gate imports both from there. fe80a23725 split a criterion across the two ends once already and the run never converged: every pass wrote what the gate condemned and prune emptied it again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`tests/semantic/` mixed three kinds of check under one name. They are now
placed by what returns the verdict:
- `tests/oracle/` holds only the checks a model decides, and `make
test-semantic` becomes `make test-oracle`
- `tests/external/i18n/` holds the back-translation check, which needs a
served engine
- `tests/integration/i18n/` holds the catalog scan, which reads the `.po`
corpus and needs nothing served
The shared reachability helpers move to `tests/utils/services.py`.
Two checks were reading `auto_comments` while `utils/i18n/translate.py`
writes the marker to `user_comments`. Over the `core` corpus that is 0
matches against 19,225, so both selected an empty set and passed having
judged nothing. The back-translation check also addressed the engine by
its pre-rename service key.
The model-pin check treated a 200 that carries no file digests as proof
the pinned file was withdrawn. An intermediary that answers with the file
list but drops the `blobs` expansion produces exactly that, so a good pin
went red; the comparison has no input in that case and the pin is now
reported unverified.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fixture holds one value per dispatch input, distinct from its default, so
an input the retrigger drops shows up as a missing key. Adding the input
without adding it here made that check fail on its own absence.
variants-services-match offered its exemption per line, so a service that is
a resolver-only hook had to repeat the same marker in every variant that
pinned it. The hook is a property of the role, not of one round, and the
repetition states one fact several times. The marker now exempts that key
throughout the file.
The exemption went through a line-number scan that recorded nothing for some
variant shapes: dsk-gnome yielded an empty map, so the rule reported a key
there that no marker could have exempted. Reading the marker off the text
needs no structural knowledge and drops the scanner with the defect.
Checked by writing an undeclared key into a real variants.yml: bare it fails,
marked it passes, and marked once while used twice it passes, which is the
loosening this commit makes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The catalog checks read the same English source out of about fifty language
catalogs and scanned it again every time. Two mechanisms follow from that.
mask() and matches() are pure functions of a string, so their result is
memoised. mask returns a frozen dataclass and can be handed out as is;
matches returns a list a caller may sort or trim, so the cache sits on
_claimed() and matches copies out of it. protected_spans stays uncached
because a Counter is mutable and sharing one would hand every caller the
same object. This also shortens make i18n-extract and i18n-translate, which
mask each source once per target language.
test_i18n_catalogs ran six methods over every catalog in turn, and one of
them read the docs catalogs off disk a second time although setUpClass had
already parsed them. A worker now reads its catalog once, answers all six
questions and returns capped finding lists. The expected core message set
travels through a pool initializer rather than being pickled per catalog,
and the docs shape comparison travels as a sha256, since the test only ever
reported which languages drifted.
test_catalogs_complete_on_gpu got the same treatment and returns two counts
per catalog instead of the catalog.
Measured: the lint from 216.76s to 9.53s, the GPU check from 284.46s to
17.26s with byte-identical findings, which is what says the rewrite changed
no verdict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The README generator read its author URL out of the ## Credits sections it
writes itself. .mailmap now folds every address a person commits under into
their @infinito.nexus one, whose local part names their social profile, and
that profile becomes the single point of truth for an author.
utils/meta/identity.py parses .mailmap rather than shelling out to
git check-mailmap, which refuses the worktree the test container mounts.
Cross-checked against git over all 14 identities in the history: no
disagreement.
A new env handler carries the committing identity into the container, which
mounts the checkout but not the contributor's git configuration. Without it
both new tests would skip exactly where they are meant to fire:
* tests/integration/meta/test_commit_identity.py fails when the configured
address does not fold, and prints the .mailmap line to add.
* tests/external/repository/test_author_profile.py probes that profile
live, behind a canary so an unreachable instance is not reported as a
missing profile.
Both look at the current address only. Probing every declared author would be
red for people who have not created a profile yet, and a suite that is red for
someone else's omission stops being read.
Marko Pjevac is declared as a role author but has no address here, so his
Credits line stays unlinked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`roles/` set as inline code reads as a literal path, yet the directory
holds an index page the build resolves. Writing it as a link costs nothing
and gives the reader the page; leaving it as code hides it, which is how
this branch degraded six working references in one pass.
The lint strips markdown links from a line before scanning it, or it would
report the code span inside a finished link, and it skips fenced blocks. It
reads the index filenames from the same place the link lint does.
Two false-positive classes came out of the first run and shaped the rule. A
span of nothing but dots and slashes is a path word, not a path: '/' names a
filesystem in a table of mount points and '../' names the traversal an
argument must not perform. An absolute span is skipped because this
repository writes Keycloak group paths that way, and /roles/web-app-bluesky
is a group, not a folder.
35 spans across 12 files become links, and the documentation build resolves
every one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
