Files
computer-playbook/roles/web-app-docs/files/python/infinito_docs/i18n.py
T
kevinveenbirkenbachandClaude Opus 5 a289dac85b fix(web-app-docs): give every build subprocess a deadline
Twelve jobs of run 37865281419 failed on the docs language spec:

    tests/test-languages.js:25:7 the deployed working tree is published in de
    with a language switcher (54.6m)
    Error: Expected the de site of the deployed working tree to be built

The spec polls /deployed/de/ until the suite budget runs out, so a build that
hangs is reported as a spec timeout with nothing to read. None of the five
subprocess calls the build makes carried a timeout, so any of them could hold
the queue open forever:

  library.py  _git()     rev-parse, tag --list and remote update --prune
  library.py  fetch()    git clone --mirror
  i18n.py                each generator of generate_commands
  i18n.py                sphinx -b gettext
  builder.py  _run()     the sphinx build of one version

The two git paths reach the network, which is where an unbounded call hangs
without failing.

`_run` streams its child's output into the build state, so it cannot take
`communicate(timeout=)` without losing the progress that `_append_log` and
`_save_state` record, and Popen has no timeout of its own. A threading.Timer
kills the child instead. The kill lands in the existing returncode branch, so
the build is reported through `_failed`, which already writes the log tail into
the state file.

RUN_TIMEOUT_SECONDS is set to catch a hang, not to beat the spec's clock: no
measurement of a successful de build exists, so a tighter value risks turning a
slow build into a failing one. With one, the value belongs below the poll
window the spec scales to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-09 22:50:14 +02:00

92 lines
2.6 KiB
Python

"""Extract the messages of the documentation into the ``docs`` gettext template.
Usage:
python -m infinito_docs.i18n --src <checkout> --output <docs.pot> --jobs <n>
"""
from __future__ import annotations
import argparse
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from infinito_docs.commands import generate_commands
PACKAGE_DIR = Path(__file__).resolve().parent
DOMAIN = "docs"
GENERATE_TIMEOUT_SECONDS = 1800
EXTRACT_TIMEOUT_SECONDS = 3600
def extract(src: Path, output: Path, jobs: int) -> None:
"""Generate the sources of ``src`` and write their messages to ``output``.
Args:
src: scratch checkout; the generators write into it.
output: path of the resulting ``.pot`` file.
jobs: parallel Sphinx processes.
"""
tooling = str(PACKAGE_DIR.parent)
inherited = [
path for path in os.environ.get("PYTHONPATH", "").split(os.pathsep) if path
]
with tempfile.TemporaryDirectory(prefix="infinito-docs-i18n-") as scratch:
conf = Path(scratch) / "conf"
out = Path(scratch) / "out"
shutil.copytree(PACKAGE_DIR, conf)
if (src / "assets" / "img").is_dir():
shutil.copytree(
src / "assets" / "img", conf / "assets" / "img", dirs_exist_ok=True
)
env = {
**os.environ,
"PYTHONPATH": os.pathsep.join([str(src), tooling, *inherited]),
}
for command in generate_commands(src):
subprocess.run(
command,
check=True,
env=env,
cwd=scratch,
timeout=GENERATE_TIMEOUT_SECONDS,
)
subprocess.run(
[
sys.executable,
"-m",
"sphinx",
"-b",
"gettext",
"-q",
"-W",
str(src),
str(out),
"-c",
str(conf),
"-j",
str(jobs),
],
check=True,
env=env,
cwd=scratch,
timeout=EXTRACT_TIMEOUT_SECONDS,
)
shutil.copyfile(out / f"{DOMAIN}.pot", output)
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--src", required=True, type=Path)
parser.add_argument("--output", required=True, type=Path)
parser.add_argument("--jobs", required=True, type=int)
args = parser.parse_args()
extract(args.src.resolve(), args.output.resolve(), args.jobs)
if __name__ == "__main__":
main()