4aba897ca2 gave the pull and build subprocess a 3600s ceiling so a hang
could not stall a deploy. A pull of a large image set legitimately outlives
any fixed ceiling while it is still making progress, and the ceiling then
kills a healthy run.
run_streamed measures the gap between two output lines instead. Two reader
threads keep stdout and stderr separated the way run_cmd does, because
compose writes its warnings to stderr and merging them poisons every parser
downstream. Only the two calls that legitimately run long use it; the short
calls whose output is parsed keep run_cmd.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
121 lines of restated code, step narration, section banners and rationale
prose, in workflow YAML, compose overrides, group_vars, lookup and action
plugins, role tasks, shell scripts and tests.
Three lines go the other way: where a reason has to survive it is now a
registered nocheck directive carrying it, not free prose. Two of them point at
an invariant further down the same file, the third records why an end-of-life
object store keeps no Playwright surface.
Scope was decided per line rather than per file. Classifying by "no additions"
would have swept roles/web-app-erpnext/tasks along with it, where the removed
lines are delegate_to, environment and cmd — a behaviour change that has no
business in a comment commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The package cache was configured in six places at once: the nginx upstream
map, the consumer compose override, pip.conf, .npmrc, the apt sources and the
twenty ensure_proxy_repo calls that bootstrap Nexus each spelled the same
hostnames, repository names and suites by hand. A new upstream meant editing
five files and hoping none was missed, and the docs told contributors to do
exactly that.
A role now declares what it needs under cache: in its meta/networks.yml, and
everything else is derived from it. svc-cache-package holds the general
upstreams; web-svc-libretranslate holds its own. All nine derived files are
generated and gitignored, so a second spelling of a hostname cannot exist.
utils.cache.hosts reads the declarations without PyYAML, because the .env
generator reaches them before any dependency is installed; it raises rather
than truncate when a comment sits at column 0 inside the block, which would
otherwise drop every declaration below it in silence. utils.cache.records
owns the Nexus record format and the Ubuntu mirror resolution, so the dev
stack and the role provision from one rendering.
A build may only resolve a cached host it can verify: the plain-HTTP hosts
are hijacked for every image, and the TLS hosts only for an image whose
Dockerfile installs the frontend CA. Hijacking all of them would fail every
pip and npm build with CERTIFICATE_VERIFY_FAILED.
svc-cache-package deploys the cache and its TLS frontend by calling the same
shell scripts the dev stack calls, rather than reimplementing them as tasks.
The role is invokable through the new svc-cache category and inert until a
host carries it.
Verified: the twenty Nexus payloads, the fifteen nginx server blocks and all
seven package-manager configs are byte-identical to the hand-written files
they replace, and the new category adds one role to the deploy order while
preserving the relative order of the other 180.
The role has not been deployed yet.
sys-svc-compose already emits a transient build.extra_hosts override for every
service that has a build, driven by INFINITO_CACHE_PACKAGE_FRONTEND_IP and
gated on the frontend CA being present. Its host list held the three apt hosts
and not argos-net.com, so 3af579449b rebuilt the same thing inside the
libretranslate role.
test_no_ansible_env_lookups found it: a role may not pull a default.env key
with lookup('env', ...), because default.env is the dev and CI contract rather
than something deployment-agnostic Ansible should reach into. Adding the host
to the shared list removes the rule violation and the duplicate in one move,
and with it LIBRETRANSLATE_CACHE_FRONTEND_IP, LIBRETRANSLATE_BUILD_HOSTS and
the container_extra_hosts call.
The URL lint probes every URL the repository mentions and a bare GET on
argos-net.com answers 404, measured, while the model paths Nexus prefixes
underneath /v1/ answer 206. The base URL is what Nexus needs, so the line is
marked the way the deb.debian.org redirector two lines above it already is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pull.py ran docker compose build and pull through subprocess.run without a timeout. The ansible task keyword eventually kills the task, but the python in between kept waiting, and the handler name says "compose pull" while pull.py also builds, so the phase the deploy sits in is not what the log claims. A first libretranslate build downloading every Argos model outlived the task timeout, and each retry restarted the RUN layer from zero.
run_cmd now passes timeout=RUN_TIMEOUT_SECONDS and turns TimeoutExpired into rc 124 plus whatever the command had already written, which every caller already treats as a failure. The matrix compose path carried no task timeout at all: its compose up has retries: 12, so a hang would never have triggered a single one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A version bump that only changes a build argument never reached a running
container. In compose mode a changed compose.yml only notified compose-up,
and `compose up` reused the existing image under the unchanged tag. In swarm
mode registry_sync skipped every push whose tag already existed in the
registry, so a rebuilt image never reached the nodes.
Compose mode now reads the rendered compose file back and records the
docker_build_digest of its build sections (context, dockerfile, args) in
.build-inputs; only a change of that digest notifies compose-build, so a
port or env change still costs no build. registry_sync records the local
image ID it pushed for every locally built image in .pushed-images.json and
skips a push only when the tag is in the registry and the ID is unchanged.
External images keep the plain registry skip.
Comparing against the registry manifest instead would depend on the image
store: with the containerd store or BuildKit attestations the local ID is an
index digest, not the config digest the manifest carries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5