The project licence governs this repository's code and says nothing about the application a role installs, which is where a restriction on commercial use usually sits: a deployment cleared to use Infinito.Nexus commercially is still barred from NLLB-200's CC BY-NC weights.
Each invokable role names its licence in meta/<role>/license.yml with the url it was read from. The rights of a named licence live once in meta/licenses.yml, fifteen fields following the taxonomy choosealicense publishes. AND folds to the strictest combination, OR keeps the most favourable whole licence, and the seven contexts are derived from the rights rather than declared, so a heading cannot contradict the matrix under it.
The generated page renders one matrix per context, a lint refuses a role that declares nothing, and a Playwright spec asserts the page against the deployed docs. 21 of 22 deprecated SPDX ids were resolved against the upstream text; EPSON's remains unread because its AUR page blocks automated access.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instruction replay provisions a fresh inventory inside a throwaway
target, so its compose pull fetched every application image from the
upstream registry while every other CI deploy pulls through the GHCR
mirror. On the moodle row of run 39082924588, 512 s of the 1646 s
ansible run went into that pull.
mirror.sh rewrites the extracted block and not the README template the
site publishes: it inserts the mirrors.yml resolver ahead of the
provision call and passes --mirror. A reader keeps the upstream
references, because a reader has no GHCR mirror to pull from, and the
replay exists to prove the instructions a reader copies. Outside CI the
two GitHub variables are empty and the block stays byte-identical.
The prefix is read from default.env in the checkout, the way machine.sh
already reads INFINITO_PARENT_IMAGE, so no Ansible template has to look
up an INFINITO_ key that belongs in an inventory.
A block that carries no provision call fails the harness instead of
quietly falling back to the upstream registry, and test_mirror.py
renders both published shapes to keep the two sed patterns matched to
what the template emits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard patched the silence budget down to 0.3 s, but the watchdog starts its clock at Popen while python -c still has to boot before it can print, so under the parallel load of a full gate run the startup alone exhausted the window and the process was killed before its first line. The same file passed in four seconds on its own. The budget is now 1.0 s against a process that prints every 0.2 s for three seconds, which keeps all three conditions true at once: the total run outlives the budget so the old total-runtime timer still kills it, the gap between lines stays under it, and so does interpreter startup. Verified against the old shape in isolation: it dies with signal 9 after five of fifteen lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The watchdog armed a single timer at process start, so it killed the German build with SIGKILL while that build was still emitting progress, and Library.failed() then answered 500 for the rest of the run. The container was not out of memory: OOMKilled is false, the restart count is zero and neither the kernel ring buffer nor the journal records a kill, while the sphinx worker BrokenPipeError that follows is the signature of killing the parent alone. The watchdog now stamps the time of every line read and kills only after a full silence window, which the existing test name already described. A regression guard runs a process that keeps printing past the deadline and requires its last line in the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
identity.py built the services.yml path by hand, which test_file_spot.py names: utils/roles/mapping.py:175 holds ROLE_FILE_META_SERVICES, and bed4b3c84f already uses it in this role. The module meant to make services.yml the single source had duplicated a path of its own.
test_mirrored_unit_tests.py also required the matching test. It proves three things against a synthetic tree rather than the real values, so it cannot pass tautologically: the configured values arrive under the environment names conf.py reads, an undefined group variable is refused instead of rendering empty, and the repository's own configuration resolves with no {{ left in any value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
services.docs.sphinx is the single place the project, copyright and author are configured. conf.py takes them from the environment, which env.j2 fills inside the container, but two callers run outside it: the host side of make i18n-extract and the unit tests. Both had a copy of the values, which is the duplication this removes.
infinito_docs/identity.py renders services.docs.sphinx with the group variables it references, through the same Jinja environment utils/templating/jinja.py:62 uses: StrictUndefined, so an undefined variable raises instead of rendering empty, and select_autoescape for html and xml, which leaves a from_string template unescaped and satisfies ruff S701.
utils/i18n/extract.py loads that module from the tooling path it already computes for the subprocess PYTHONPATH, without touching sys.path. The two tests import it through the _TOOLING entry they already insert. conf.py is unchanged and still requires the environment: reading the configuration there would turn it back into a default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The third post-merge quality-high pass wrapped the two context managers of 00393bb1b1 into the parenthesised form. Both revisions parse to an identical AST.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
conf.py reads DOCS_SPHINX_PROJECT, _COPYRIGHT and _AUTHOR with os.environ[...] since 9dc7f0cbc2, because a missing value must stop the build rather than render an empty project name into every title and the footer. The test loads the module at import time with only sys.argv patched, so it raised KeyError.
Supply the three through _REQUIRED_ENV in the same patch block that loads the module, and again in the reload test, whose own patch.dict opens after the import patch has closed. The test now states the requirement instead of a default hiding it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
After a version build the builder queued every translated language, not the
configured ones. The list now reaches the library: server.py reads
DOCS_I18N_PRELOAD_LANGUAGES, Library freezes it into a set that drops the empty
string ''.split(',') yields when nothing is configured, and builder.py queues a
language only when it is in that set. Every other translation is built when a
visitor opens it.
The warm-up task accepted 202, which means queued, so the deploy walked on
while the sites were still building. A second task now waits for 200. Both read
DOCS_I18N_PRELOAD_PATHS, hoisted out of the loop it used to be written inline
in, because two tasks need the same language-code-to-URL-path mapping.
DOCS_BUILD_PARALLEL divides by overhead + 2 * process_mb instead of
overhead + process_mb: with a translation building beside the version build a
parallel slot holds two process-sized allocations. The new divisor is strictly
larger, so the value can only fall, never rise; with the configured 3072 and
2048 MB it differs from the old one only between 10 and 16 GB of service
memory, where it goes from 2 to 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve jobs of run 37865281419 failed on the docs language spec:
tests/test-languages.js:25:7 the deployed working tree is published in de
with a language switcher (54.6m)
Error: Expected the de site of the deployed working tree to be built
The spec polls /deployed/de/ until the suite budget runs out, so a build that
hangs is reported as a spec timeout with nothing to read. None of the five
subprocess calls the build makes carried a timeout, so any of them could hold
the queue open forever:
library.py _git() rev-parse, tag --list and remote update --prune
library.py fetch() git clone --mirror
i18n.py each generator of generate_commands
i18n.py sphinx -b gettext
builder.py _run() the sphinx build of one version
The two git paths reach the network, which is where an unbounded call hangs
without failing.
`_run` streams its child's output into the build state, so it cannot take
`communicate(timeout=)` without losing the progress that `_append_log` and
`_save_state` record, and Popen has no timeout of its own. A threading.Timer
kills the child instead. The kill lands in the existing returncode branch, so
the build is reported through `_failed`, which already writes the log tail into
the state file.
RUN_TIMEOUT_SECONDS is set to catch a hang, not to beat the spec's clock: no
measurement of a successful de build exists, so a tighter value risks turning a
slow build into a failing one. With one, the value belongs below the poll
window the spec scales to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The prepared tree is reused through hardlinks, so its files share inodes and a build writing a source file in place would rewrite the cache for every later reuse. _seal drops their write bits to make that a loud failure, and the test proved it by expecting a PermissionError on a write.
pytest runs as root in the test container, and root writes through missing write bits, so the test failed there while passing on a host. It now asserts st_mode & 0o222 == 0, which is what _seal actually does and holds for any user, and keeps the write attempt as an enforcement probe that is skipped under root rather than reported as a defect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scripts/lint/playwright.sh derives a stub value per env key by name suffix, and only *_JSON, *SLUGS and *OVERRIDE get []. Everything else gets the literal string stub, which decodeDotenvJsonList rejects at module scope, so the spec contributes no tests and playwright test --list reports none. The exported key becomes DOCS_I18N_PRELOAD_LANGUAGES_JSON on both sides; the Ansible variable keeps its own name, because only the wire name is what the stub matcher reads.
The prepared-tree fixture is walked with rglob, which the raw-walk guard flags so repo scans share one cached walk. A tempdir is the opt-out that rule names, and its marker has to sit on the call's own line, so the call moves onto one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A translated site repeated the whole checkout and generation of its version, and the builder ran one marker at a time. The prepared tree is now cached per (ref, tooling digest), hardlinked into each build's scratch and sealed read-only, and a bounded thread pool claims the markers a free slot can build now - a language only once its version is current, so a stale one is no longer claimed and bounced in a spin.
services.docs.build.jobs and services.docs.build.parallel replace build_jobs and default to half the host's CPUs, parallel capped at 2 so the two knobs cannot multiply past mem_limit. DOCS_I18N_PRELOAD_LANGUAGES replaces DOCS_I18N_LANGUAGE, defaults to the host language and warms [en, de] in both matrix variants, where docs_sites is backed up under swarm so the DR drill restores the built sites instead of rebuilding them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cli/build/docs/integration_matrix was 1189 lines a human ran through
'make integration-matrix', and its output sat frozen as a 125-line grid in
requirement 027. Its ENTITIES and EDGES lists were hand-maintained, so the
grid could only drift: it covered 84 entities while the tree holds 98 roles,
named two roles that no longer exist, and omitted sixteen current ones -
api, checkmk, docs, hermes, homeassistant, jellyfin, meta, mirror, openbao,
openclaw, penpot, pihole, s1, seaweedfs, seaweedfs-console and semaphore.
infinito_docs/generators/integrations.py replaces it and joins the nine
sibling generators in that package, so the overview, services and addons
pages are written into generated/integrations/ on every docs build rather
than when someone remembers the target. index.rst takes the overview and the
new docs/architecture/integrations.rst references the other two.
Requirement 027 becomes prose. The make target and its i8inma alias are
removed as a pair, which keeps the alias lints satisfied.
Coverage is not strictly larger. The removed EDGES list carried a curated
upstream project URL per integration, 96 of them, and the new edges have no
such field; only 48 of 159 addon declarations hold update.upstream_id or
config.repository, so about a third is re-derivable and the rest existed only
in that list. Its fourth field is no real loss - 57 entries read 'check', so
it was a near-constant rather than an axis. In exchange the old module had
zero tests, not one file under tests/ referenced it, and the generator ships
310 lines including generate().
Residuals: the HTML build runs without -W, but i18n.py:57 runs the gettext
builder with -W and check=True, and that build has never seen
integrations.rst - no catalog mentions generated/integrations. And
docs/requirements/036:72 still carries a checked criterion asserting a table
in 027 that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On a cold container the docs Playwright suite triggers four serial Sphinx builds and waits for the last one. test-languages.js polls /deployed/de/ for 200 and got 202 for 55.6 minutes in job 110003713825 of run 36745666606, after which the suite budget expired and five further specs never ran. The suite has never passed on a first attempt; it used to go green only on the harness retry, which 7127c6b96b removed for budget timeouts, so a structural overrun became a red job.
Measured cost per build, from run 36558500327 where the whole queue drained on a kept stack: container up 12:15:52, the spec green at 13:21:41 answering on its first poll, next container start only at 13:30:45. Four builds in 65 minutes 49 seconds, so at most 16.5 minutes each - not the sixty to ninety minutes 40688e4ded estimated.
Three of the four builds were none of the test's business. fetch() queued latest in the foreground although nothing waits on it at startup, and _refresh_stale_sites did the same for every servable version; both now use the background lane, which is what its own docstring describes. test-front-page.js reached GET /latest/, and reaching an unbuilt version queues its build - it now asserts the redirect from the Location header without following it, and checks the same page shape against /deployed/, which is built anyway. The release tag that test-on-demand-build.js clicks stays, and now runs behind the awaited work instead of in front of it.
The translation could only be queued after its base finished, because the 202 branch returns before _translation is reachable, so it always sorted behind whatever the sibling specs had queued meanwhile. The 202 branch now carries the language segment along, admitted by translatable(), which answers from the snapshot catalog on disk before any build; build_language still re-checks the written index.
Moving a version into the background lane exposed that status() read only the foreground one, so a waiting version would have been reported as missing rather than queued to the versions overview and to /api/versions. is_queued() now covers both lanes, mirroring _dequeue, which always removed from both. Two of the five unit failures this change first produced disappeared on that fix alone, which is what shows they pinned real behaviour; the three that pinned the lane itself were rewritten to assert the behaviour instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`/snapshot` is a tar of `DOCS_SNAPSHOT_PATHS` and the list carried 32 entries,
neither of them `Makefile` or `aliases`. Both are tracked at the repository
root and `commands.py:80-90` hands them to two generators,
`make_targets --makefile {src}/Makefile` and `aliases --aliases-file
{src}/aliases`. `generators/make_targets.py:45` reads with no existence guard,
so the build dies with FileNotFoundError before sphinx is reached.
It dies without saying so anywhere. `_failed` wrote the exception into
`states/deployed.json` and printed nothing, and `library.status` iterates
`[LATEST, *tags]`, so `deployed` never appears in `/api/versions`. No log, no
API and no test carried the failure. `/deployed/de/` therefore answered 202 for
as long as the container lived and the Playwright suite polled a dead build for
its entire budget - eight jobs of run 36651124458, three hours each.
The failing test's trace settles which build it was: the 202 body carries
`<title>Building deployed</title>`, not `deployed/de`. The English `deployed`
version is the one that never lands, so the German build is never even queued.
That also gives the discriminator: `latest` and the release tags build from a
`git archive` of the mirror, where both files are present, and they reach
`ready` in the same run.
Three parts. The list gains the two entries, in the file's ASCII order - that
is the fix. `_failed` also prints to stderr, because without it the next such
gap is just as invisible; the moodle job's docs container log was 484 access
log lines and nothing else. And a test derives the requirement instead of
restating it: it calls `generate_commands()` and requires every first path
segment under `src` to be in the snapshot, `generated` excepted, since that one
the build creates. Adding two entries by hand would have broken the same way on
the next generator.
Main is unaffected: the role does not exist there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`request()` ended in `touch(exist_ok=True)`, and on an existing file that
refreshes the mtime. `next_queued()` sorts by mtime and serves the oldest
marker, so re-requesting a build that is already queued moved it behind
everything else. The server calls `request()` on every visit, and a visitor
waiting for a page polls every thirty seconds, so waiting for a build was
what prevented it:
queued de, then a tag : ['deployed:de', 'v14.2.0'] -> builds deployed:de
after one poll on de : ['v14.2.0', 'deployed:de'] -> builds v14.2.0
One poll is enough. In the docs suite the spec before it deliberately clicks
an unbuilt release tag, so another hour-long build is always sitting there to
be overtaken by.
Observed in run 36596159856: `/deployed/de/` answered 202 and never 200 for
two hours and four minutes in one deployment and two hours fifty-seven in
another, from those jobs' own rescue artifacts, while the suite budget is
fifty-nine minutes on clearnet and eighty-nine on tor.
This removes the starvation, not the arithmetic: a full Sphinx build costs
sixty to ninety minutes here and the suite needs two of them in sequence,
English then German. Whether one round now fits is for a run to say.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every commit here went in with `--no-verify`, so the pre-commit `make test`
never ran and neither the formatter nor the linter saw the change sets. The
quality gate collects that debt now.
`test_aliases.py` built a lookup table by handing a generator to `dict()`,
which ruff C402 rejects and offers no autofix for; a dict comprehension does
the same work. `selections.py` and `test_translate.py` each wrapped a call
across three lines that fits on one, which is the formatter's own rewrite
from the run that then aborted on the C402 above.
All three edits preserve behaviour: the comprehension is equivalent to the
generator it replaces, and the rest is whitespace.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make.md names 'make help' as the authoritative target list and the site never carried one, so a reader had to run the repository to see what it offers. The same held for the i8 aliases and for the infinito command tree.
Three generators join the ones the build already runs for the YAML index and the role glossary. make_targets reads the comment block above each target, the same block make help prints, and turns 'Param x:' into a field. aliases reads the trailing comment, falling back to the target an alias wraps, because a plain make wrapper carries no comment by repository rule; an unparseable line raises instead of vanishing from the page. cli_commands asks cli.core.discovery, the rule the dispatcher itself applies, so the page cannot promise a command the CLI rejects.
cli_commands parses each __main__.py with ast rather than importing it: rendering a page would otherwise run 65 modules' import side effects, and a broken one degrades to 'Undocumented.' instead of losing the page.
literal() escapes the inline markup starters in prose taken from those files. 3b81b4de2e fixed this class once already, where a role description reading (*.parent) opened an emphasis that never closed; a Makefile comment naming two globs does the same. Measured: the unescaped form reports three docutils warnings, the escaped form none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The site answers 202 while it builds a version, and 200 once a build
exists. Whether that build matches what the version currently names was
never asked at the serving gate: servable() only checks that an index.html
is on disk. So the first request after a new snapshot got the previous
build with 200, and the rebuild it triggered finished into a directory
nobody was waiting for.
For a release tag that cannot happen, and for latest it is a reasonable
trade, since a visitor would rather read the previous page than a spinner.
For deployed it is wrong by definition: the name means the working tree
this run staged, so an earlier snapshot is not a stale answer but a
different question's answer. The post-deploy test that reads the published
page therefore validated the round before it, and reported a role broken
whose page had already been fixed.
The comparison already existed. _current() weighs built_ref against
_wanted_ref, which resolves deployed to the snapshot digest, and the
builder consults it to decide whether to rebuild. Only the server did not.
outdated() exposes it for deployed alone and the serving branch now treats
a mismatch as not yet servable, so the poll waits for the build it asked
for.
Measured on the stack while the previous round was live: the served page
was written 16:44:41 from the older snapshot, its own source carried the
newer content, and the correct build landed at 17:10:56, twenty-six
minutes after the test had already read the stale one and failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docs round ended the same way twice: seven specs passed, four never ran,
and the suite was killed at its 89-minute cap after an hour and a half. Three
separate causes, each of which hid the next.
A waiter could outlive the suite that contained it. The specs budget a poll
against the role's run timeout, but Playwright is handed that timeout minus a
60-second buffer, so the global timeout fires first. When it does, Playwright
counts the in-flight test, and every test behind it, as "did not run", which
names nothing and reads like a suite that was merely slow. resolveTimeout now
caps what it returns at the budget the suite actually has, so an exhausted
waiter reports which predicate it was waiting on.
What it was waiting on could never arrive. 59e7b0a2a7 retargeted the yaml page
from /latest to /deployed, but /deployed is built from DOCS_SNAPSHOT_PATHS and
that list does not carry compose.yml, while /latest is built from a full git
archive and does. A servable version whose file does not resolve answers 302,
not 404 and not 202, and the poll rejects redirects, so it waited 88 minutes
for a page the version could not produce.
With the suite finishing, the CLI test's own failure became reachable: it reads
the published instructions from /latest, and /latest is stale. It still serves
web-app-sphinx and desk-zoom, names this repository renamed away, so ten roles
have no page there at all and web-app-docs is one of them. It now reads the
version the run just deployed, which is what a post-deploy test should replay.
Measured on the stack: /deployed/roles/web-app-docs/README.html answers 200
with the production block, /latest answers 302, and the published index lists
275 of 282 roles. The round now passes nine specs in 17.5 minutes.
Only five of the 1512 resolveTimeout call sites across 87 roles are large
enough for the cap to reach: three here and two in web-app-openwebui, whose
suite has not been re-run.
ruff reflows the guide_role signature and its two call sites, the same for
three test modules, and the YAML formatter prepends the document marker to
the machine's compose file. These files were created one commit too late to
be caught by the previous formatting pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff format reflows long calls and normalises quotes, isort reorders the
imports of the documentation lint package, SIM117 folds nested `with`
statements into one parenthesised block, TC003 moves typing-only imports
under `if TYPE_CHECKING`, the JavaScript linter replaces a `var`, and the
YAML formatter rewraps five role files. None of it changes behaviour.
One entry does more than that: the formatter had reflowed two calls in
test_builder.py so their `# nocheck: cache-read` markers landed on the
closing parenthesis, where they suppress nothing and the cache-read lint
would have reported two real violations. Both calls are hoisted into a
variable so the marker sits on the reading line again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The replay of a role's "### Production" block moves out of the CI guide
scripts and into the documentation role itself. The block is read off the
rendered page over HTTP, so what runs is what a reader copies, and the role
that publishes the page is also the role that proves it.
scripts/github/guide/{one,compose_deploy,host_deploy}.sh and
cli/build/docs/readme/production.py are gone with the workflow step that
called them. The instructions marker now only adds web-app-docs to the row's
apps; test-e2e-cli discovers the role through its new templates/test.env.j2
and runs files/test/test.sh.
The block runs in a throwaway machine, never on the stack host. compose.yml
starts one service built from files/test/Dockerfile, whose base is
INFINITO_PARENT_IMAGE out of the mounted checkout's .env, so the machine is
the stack host's own distribution and the replay exercises the installation
instead of a prebuilt image. files/test/init.sh adds an init when the base
carries none and keeps the full -Syu that ef0f03ea98 established for arch.
scripts/tests/workspace/base/01_install.sh then prepares the machine before
the block runs.
Nothing is skipped any more. The previous host-install branch exited 0 and
passed silently over 65 of the 178 invokable roles; a block without a
"docker run" is now replayed against systemd in that same machine, with the
git clone dropped in favour of the mounted tree under test.
The target role comes from the new guide_role lookup, which reads
list_invokable_app_ids() rather than matching role-name prefixes, and falls
back to the asking role when the round deploys nothing else. deployment.py
grows running_apps() so both lookups share one definition of what the round
deploys. required_by_coverage.role_is_invokable drops its own walk over
categories.yml and delegates to the same SPOT; measured equal across all 280
roles.
Carried in the same index, because the refactor pass ran over this change
set: library.py splits into sites.py and build_queue.py mixins, builder.py
hands translated_languages to catalogs.py and folds three repeated log-tail
and state writes, and placeholders.py becomes a facade over spans.py,
repairs.py and damage.py. No test changed for those splits, and the two
ref-resolution dicts builder.py carried are replaced by _wanted_ref, which
also stops hashing the deployed tree on every tag build.
Verified: 7132 unit and lint tests pass over the whole tree, the guide_role
lookup renders through a real Templar, the base image resolves from .env,
and compose config validates the service. The replay itself has never been
executed; that needs a deploy of web-app-docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cosmos, Quick Setup and Credits follow from meta/services.yml, categories.yml
and galaxy_info.author, yet 279 role READMEs carried a copy that went stale
whenever the metadata moved and nobody re-ran make docs. The documentation
build now runs the generators itself, against its own scratch checkout, and
the committed READMEs keep only the prose a person writes.
utils.cache.PROJECT_ROOT derives from the module's location, so putting the
checkout first on PYTHONPATH makes the generator operate on the scratch tree.
i18n.py and builder.py add it; the one remaining coupling, a relative_to over
PROJECT_ROOT in a log line, is gone.
CI extracted the "### Production" block from the committed README with awk.
Stripping the section made guide_deployable() return "" for every role, which
selects no deploy row and turns nothing red: a fail-open regression. It now
asks _is_role_invokable, which matches the old condition exactly (178 roles
either way, both difference sets empty), and cli/build/docs/readme/production.py
renders the block for the two guide scripts. All 178 renders are byte-identical
to what HEAD served, and both scripts' extraction pipelines produce identical
output for a web app, a service and a desktop role.
test_readme.py inverts: the three sections are now a hard failure in a
committed README. readme_md.md documents what each is derived from.
Removed: the docs, cosmos, readme-generate, readme-index and readme-check
targets, the five aliases pointing at them, and docs from make quality.
Verified by running generate_commands() in series against a copy of the
checkout: every command exits 0, no README is left without Cosmos or Credits,
and exactly the 178 invokable roles get a Quick Setup. The ansible_roles step
needs pandoc, which only the docs image carries, so that one link of the chain
is unexercised here; it is unchanged by this commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every role README carries a ```mermaid flowchart and the documentation
showed it as source. A post-transform now replaces those literal blocks with
<pre class="mermaid"> in html builds, which the runtime picks up; every other
builder keeps the fence as it was.
sphinxcontrib-mermaid injects an inline init script, which this deployment's
CSP refuses, so the runtime and its initialisation are two linked files
instead: mermaid-init.js calls mermaid.initialize with securityLevel strict
and follows the reader's colour scheme. The diagram source is html-escaped
before it is embedded, or an & or < in a node label would break the page.
The runtime is fetched at image build time through ADD --checksum, so a
changed upstream file fails the build rather than shipping. Version and
digest are not ARG defaults, which test_arg_without_default forbids; they
travel meta/services.yml -> vars/main.yml -> compose.yml.j2 the way
DOCS_IMAGE and DOCS_VERSION already do, and an update block registers the
pin with the npm version watcher instead of exempting it.
Verified by building a page with sphinx: the diagram arrives as
<pre class="mermaid"> with its source escaped, a bash fence stays a
highlighted block, and both scripts are linked. A missing mermaid.min.js does
not fail that build, so a local run without the image stays green.
scripts/lint/mermaid.sh keeps validating diagram syntax through mmdc, which
npm installs unpinned. It therefore checks against a newer mermaid than the
11.4.1 readers get, and a diagram using syntax from in between would pass the
lint and render blank.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
untranslated_markup.py and untranslated_yaml.py shared a name prefix in one
directory, which test_prefix_grouped_files reads as a folder in disguise. They
are now extensions/untranslated/markup.py and yaml.py, beside the __init__.py
that extensions/local/ already carries for the same reason.
A module named yaml.py inside a package does not shadow PyYAML, because
imports are absolute: all nine extensions conf.py registers still import, and
the docs-role unit suite stays green.
The unit tests move with them, keeping the parents[9] root derivation their
sibling package uses at the same depth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_brands, test_titles and test_yaml_source spelled out meta/main.yml,
meta/services.yml and meta/info.yml where utils.roles.mapping already names
them. test_file_spot reports exactly that, and it did: the three files were
its only findings in a full make test.
A hard-coded path keeps pointing at nothing once a metadata file is renamed,
and a fixture that writes to the wrong place fails somewhere else than where
the mistake is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One predicate now answers what no translator may be handed, and both ends of
the pipeline read it: the extraction withholds the message from the catalog,
the client refuses to send it. What the catalogs keep back is exactly what the
client would have declined.
Two shapes qualify. A message with no word is markup. A message that is only a
name is a label: a brand, this software, a credited person, or a bare URL.
Inside a sentence the name stays protected by the span machinery and the
sentence still needs translating.
Three details each come from a finding. The comparison runs against the prose,
because **Mastodon** is not the string Mastodon and prose() already strips the
emphasis. Case is ignored, because a page writes the name the way its sentence
needs it and **mailu** is still Mailu. The address is matched first, on the raw
text, because masking protects the address of mailto:a@b.c but leaves the
scheme behind and its prose reads as a word.
Measured over the docs domain: 22745 messages fall to 22402 with zero
warnings, and 145 of the 377 removed entries carried a corrupted brand that
was being shipped - Keycloak read Schluesselmantel, OpenProject read offenes
Projekt, micro read Mikrowelle.
SOFTWARE_NAME is read from the group_vars SPOT beside SOFTWARE_REPOSITORY. It
lived there already and no python had ever read it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A table cell holding only a link has nothing to translate, yet it became a
msgid like any sentence: 2999 of the docs domain's 25742 messages, 11.7
percent. They can never be filled, so they count as pending for good and
skew every completeness figure.
The messages come from sphinx's own extract_messages, which is what the
gettext builder collects. That is not cosmetic: sphinx takes a paragraph's
rawsource, so judging astext() would read 'e2e/' out of [`e2e/`](e2e/),
find a word in it and let the markup through. A test now pins that
difference, because the first version of the test could not see it.
The predicate is has_words, the one the translation client already applies
before it sends a batch, so extraction and translation share one definition
rather than two that drift.
Measured: 25742 messages fall to 22745, no surviving msgid loses its
translation, and 2924 of the 2948 removed entries carried their own source
as their translation.
The import is guarded because the site builds arbitrary refs and one older
than utils/i18n/ cannot supply the predicate; the transform then does
nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sphinx prefers nothing: dir/index and dir/README are two independent
documents. The preference lives wherever something maps a folder onto a
page, and this repository had two such places disagreeing.
local.file_headings has always hidden README.md from a directory carrying
an index.rst, while yesterday's resolver saw README only, so a link led to
the page the sidebar hides.
Both now try index.rst, index.md, README.md, README.rst in that order. The
extension is renamed accordingly, and the tuple lives once in the lint
package rather than a third time.
That also repairs two links this branch degraded: roles/ holds an index.rst
and no README, so a README-only rule reported every reference to it as
broken and they were turned into code spans.
Three links stay code spans on purpose. Their text names something the
target is not - a suite, a file - so resolving them would restore a
misleading reference rather than a working one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apidoc now excludes roles/ and library/ the way it already excluded tests/.
A role name carries hyphens and is therefore not an importable module path,
so automodule reported 44 invalid signatures and rendered nothing; library/
holds ansible modules whose ansible.module_utils imports exist only inside a
play. Measured: 168 generated pages drop to 151, and the 17 that go consist
of a heading and automodule directives on unresolvable paths.
The role page generator fits every section underline to its title. pandoc
counts one character fewer than docutils where a heading ends in a stray
variation selector left behind by a removed emoji. It also escapes inline
markup in role metadata, because one description reads (*.parent) and opened
an emphasis that never closed.
The index generator underlined a 21-character caption with 19 characters.
conf.py adds the tel scheme, without which the phone number in SECURITY.md
is an unresolvable reference, and the extract copies the image assets into
its conf directory the way the site builder does, without which every run
reports a missing html_static_path and favicon.
Four tests pinned values this changes. The fifth, the extension count in
test_conf, asserted nothing the loop below it does not already prove and
broke on every legitimate addition; it now asserts the list is non-empty, so
that loop cannot run over nothing.
With this, make i18n-extract domain=docs finishes with zero warnings under
the -W gate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Almost every role declares galaxy_info.company as a block scalar. The
generator rendered each entry as '- **key**: value', so the second line
landed in column 0 and ended the bullet list it sat in. 280 of the build's
remaining 354 docutils messages came from generated/roles/*.rst, which is
this generator's own output; folding every value onto one line clears 277
of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both extensions landed in bd15eae9fa without the mirrored unit test the
repository lint requires, which left that lint red.
The resolver's cases include the mistake its first version made: it
filtered on refdomain == 'doc', while myst builds a document reference
only when the target is_file(), so a directory arrives with refdomain
None and the transform did nothing at all. Reintroducing that filter
fails three of the six cases.
The parser's cases pin what the build relies on: one yaml-highlighted
literal block, no block quote, a title from the filename, and the orphan
docinfo as the first child, which is the only place sphinx reads metadata
from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fetch() requested latest and run_builder() requested deployed once at startup, so a release tag was built on demand and then never revisited. A tag carrying a languages.json from before the translated-site split stays servable, nothing ever requests it, translates() stays false for every code and /v1.0.0/de/ answers a hard 404 forever.
_refresh_stale_sites() asks for every version that is already servable; request() checks _current() itself, so only a genuinely stale one is queued and an unbuilt tag stays on demand instead of triggering a mass rebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
request(..., background=True) skipped a language as soon as translation_servable() found an index.html, which is an existence check with no ref in it. A language was therefore built exactly once: every later commit advanced the English site while every translated site stayed frozen on the commit it was first built from. The eager loop this replaced rebuilt them all, so the freeze arrived with f5eff500c1.
_publish_site already stamps the ref beside each translated site and nothing ever read it. translation_ref() reads it and _translation_current() compares it against _wanted_ref(), which also replaces the eager {LATEST: head, DEPLOYED: snapshot_ref()} dict so an unreadable snapshot no longer kills the builder while building an unrelated version.
Two more from the same review: build_language wrote into the version's state file, so a failing language reported the whole version as failed and leaked its sphinx log into /api/versions; it now uses its own version:code key. And the language index was written after the site was published, so a failure in between left a current site with a stale index, which _current() rejects forever and fetch() would rebuild on every tick; the index is written first.
library.py had grown past the 500-line cap, so the build machinery moves into builder.py as a Builder mixin. An AST diff proves every function, method and constant is byte-identical apart from _write_json -> write_json, and the six build-focused test cases move to test_builder.py with the count unchanged at 16.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
build() ran sphinx once per translated language before returning, so a deploy blocked for roughly 30 minutes per language. With 29 translated catalogs German sat eighth and its site appeared after about three and a half hours, which is why the language spec polled /deployed/de/ into a 404 for its whole run. The server already answered an unbuilt version with 202 and a build request; an unbuilt language got a bare 404.
Give languages the same treatment. request() takes a code and writes a version:code marker, build_language() builds exactly one from its own checkout via the extracted _checkout(), and the server answers a translated but unbuilt language with 202 while an untranslated one keeps its 404. No language is dropped: build() now queues every translated one into queue/background/, which next_queued() drains only after the lane a visitor is waiting on, so the requested language overtakes the backlog.
languages.json grows a translated list beside known, and _language_index() reads a pre-split file as known-only so an existing volume keeps naming its languages instead of rendering an empty switcher.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every version whose tree holds a locale/<code>/LC_MESSAGES/docs.po with at
least one translation now gets one site per such language next to the
English one, built with `-D language=<code> -D html_copy_source=0` after
the English site is published. Sphinx reads one docs.po per language
(locale_dirs, gettext_compact = "docs"); YAML pages stay untranslated
through an extension that runs before the Locale transform.
The deployed working tree is built as the unlisted version `deployed` from a
snapshot baked into the image, because the git mirror only holds what is
pushed to main. The server answers /<version>/<code>/ and
/api/languages/<version>, a language without a site is a 404 rather than the
English page, and a switcher on every page lists the built languages and
keeps the page path. min_storage grows to 13GB for the translated sites.
The 47 docs.po catalogs hold the 24869 documentation messages; German is
partially machine-translated so far. The raw-docker and primary-domain lints
skip .po files, which carry the Markdown's own commands and hosts as
translation sources.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The prefix-grouped-files lint rejects two siblings sharing the local_
prefix. They now live in extensions/local/ and import nav_utils
absolutely, which TID252 requires for a parent-package import.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Sphinx configuration, extensions and generators lived in the separate
infinito-sphinx repository, whose image baked a snapshot of core at build time
from a base image name core no longer publishes. Only core consumed it, and its
generators read core's role layout, so the two repositories drifted without any
CI covering the pair.
The tooling now ships in roles/web-app-docs/files/, and web-app-sphinx is
renamed to web-app-docs. The container is a small service instead of a static
image:
- it keeps a git mirror of the repository in the docs_sites volume;
- /latest/ is the last commit of the default branch, rebuilt whenever a fetch
finds a new one while the previous build stays online;
- every release tag is selectable in the sidebar switcher and on /versions/,
and is built the first time someone opens it, behind a progress bar;
- replicas share the volume: the one holding builder.lock fetches and builds,
the queue and the build states are files beside the sites, so another
replica resumes the queue when the builder dies.
The theme rewrites every page serially after the build, which took about 75
minutes for the 4544 pages; that pass now runs on the -j workers with the
theme's own function. sphinx-apidoc skips tests/, because
tests/utils/bare_bootstrap_probe.py installs an import blocker when autodoc
imports it and killed the Sphinx workers. The roles overview is generated by a
separate process, so autodoc resolves utils from the version being built, not
from main. The role pages now show galaxy_info.description instead of the
top-level key that never existed.
The Sphinx packages join the dev extra, because the mirrored unit tests of the
tooling import them. min_storage now budgets the built image, the latest site
and a running build's scratch instead of the old image's size. The Playwright
suite is split into front page, versions and on-demand build files; the last
one selects an unbuilt tag in the overview and asserts the build gets queued.
Operators must rename the web-app-sphinx inventory group to web-app-docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
