Computing the distro list no longer justifies a runner of its own: it is
two steps, and the job that consumes them already checks the repository
out. The workflow now occupies one runner instead of seven.
The step also went silent for minutes at a time, printing five pids and
then nothing until every build had finished. Each build's output is piped
through a [distro][HH:MM:SS] stamp as it arrives, so the log shows what is
happening while it happens; tee still files the per-distro logs for the
artifact, and the replay block at the end is gone as duplication.
A pipeline hands $! the pid of tee rather than of the build, which would
have swallowed every failure, so each build runs in a subshell under
pipefail and wait reads that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix spent five runners on work that is almost entirely network
wait: the collection install measured 0% CPU for its whole duration, and
the parent-image pull, the package managers and the push are no
different. One runner carries all five concurrently and frees the other
four.
Losing the matrix loses fail-fast: false with it, so push_all.sh awaits
every build and reports each one by name before exiting non-zero. A
broken distro no longer hides the other four, which is what made today's
triage find three causes instead of one.
Starts are spread by fibonacci seconds because each build opens with a
parent-image pull of several hundred MB. Per-distro logs are grouped in
the job output and uploaded as an artifact, and the pressure report now
carries free -h: memory is the one resource this consolidation contends
for, and an OOM kill would otherwise read as a build error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ignore-error traded a loud fault for a silent one. The cache writes to
the same GHCR repository as the image and under the same packages:write
token, so every shared failure mode -- permissions, outage, rate limit --
already fails --push. What remains is buildkit's own cache manifest type,
which a registry can reject while accepting an ordinary image push.
That case is exactly the one worth hearing about. Swallowed, it leaves CI
green while the cache never fills and every build silently pays the full
855-899s again; raised, the first run answers it once and for good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
COPY . preceded every expensive RUN, so each commit rebuilt the whole
chain and ansible-galaxy ran on every build of every distro. The
collection install now sits in its own layer fed only by default.env,
scripts/install, the dev-python shell scripts, requirements/ and the
three files the version check imports. Everything that needs the full
tree, the project's own distro package included, stays behind COPY ..
Measured on a role-only change: the collection layer stays CACHED and
the build drops to 3m10s against a CI baseline of 855-899s. An
unchanged rebuild is 15 cached layers in 1.5s.
The gha cache imported and exported nothing in a plain run step, so the
export moves to the registry the job already authenticates against.
ignore-error keeps a cache write failure from taking the build with it,
which it otherwise does: proven against an unreachable registry, where
the build fails hard without it and exits 0 with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Output of 'make autoformat': ruff collapses expressions that were wrapped across lines without need, shfmt and the yaml formatters drop surplus blank lines. No semantic change - the whole diff is 10 insertions against 39 deletions, and every hunk joins lines that already said the same thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Transient Docker Hub timeouts resolving docker/dockerfile:1 aborted two deploy jobs at '#2 resolve image config'. Pre-pull the frontend from the Dockerfile's syntax line with 3 bounded retries; fail soft so the build still resolves live if Docker Hub stays down. Real build errors remain unretried.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
INFINITO_PYTHON_INSTALL_SCRIPT and INFINITO_DOCKER_CLI_INSTALL_SCRIPT join the default.env SPOT: the Dockerfile consumes them as required args (:? in the RUN as last defense), all three image builders (compose.yml, build.sh, push.sh) pass them hard-required, the shell consumers read the SPOT via default.env grep and the Makefile via BASH_ENV. INFINITO_SWARM_NFS_EXPORT_BASE is no longer a static default: a new env handler derives it from the deploy SPOT group_vars/all/15_storage.yml, 00_topology greps that file (and the STATE_SUBDIR constant) directly, and the swarm test extras stop restating the group_vars value. Stale hadolint directives are restored and recognised; leftover narration comments in the touched files are dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/image/build.sh drives 'make build' and the test-environment
matrix's dev-image build, but missed the new INFINITO_VENV_DIR build
arg. Dockerfile's ARG resolved to empty, ENV set INFINITO_VENV_DIR=""
in the image, and entry.sh's assert tripped on first --compile.
Adds the assert and the matching --build-arg, parallel to the
INFINITO_SRC_DIR plumbing already in this file and to the equivalent
plumbing in scripts/image/push.sh.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds INFINITO_VENV_DIR=/opt/venvs/infinito to default.env (registered
in passthrough's STATIC_KEYS so make dotenv propagates it to .env) and
threads it through the build chain: compose.yml forwards it as a build
arg; scripts/image/push.sh asserts and --build-args it; the Dockerfile
declares the matching ARG/ENV and derives PYTHON/PIP/PATH from the
variable. scripts/meta/env/python.sh now prefers the value when set
before its fallback discovery so the SPOT is effective in env
resolution.
Fixes the dev image build regression where 'infinito --version' in
scripts/docker/entry.sh exited 127. Root cause: the Dockerfile RUN
SHELL uses bash -lc, which sources /etc/profile and resets PATH to
the Debian default, dropping ${INFINITO_VENV_DIR}/bin from it. The
fix invokes the script via its absolute path "${INFINITO_VENV_DIR}/bin/infinito"
so the post-install version check no longer depends on a PATH that
the login-shell init has just clobbered.
scripts/lint/wrapper.sh's existing PATH-reset note now references the
variable instead of the literal path. test-install-make.yml and
test-install-pkgmgr.yml read INFINITO_VENV_DIR from default.env up
front, pass it into their pkgmgr containers, and source the venv
activate script via the variable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
default.env decomposed into INFINITO_PARENT_IMAGE_{OWNER,TAG,VARIANT}
atoms. New utils/env/handlers/infinito_parent_image.py owns the URL
composition `ghcr.io/{owner}/pkgmgr-{distro}:{tag}` as the single
definition site; scripts/image/{push,build}.sh consume the handler-
emitted INFINITO_PARENT_IMAGE from .env instead of inlining their own
format string.
Side effect: `compose up --build` now picks the parent image that
matches INFINITO_DISTRO. Previously default.env's hardcoded
pkgmgr-arch:stable won regardless, so debian-named containers were
silently built FROM the arch base. scripts/image/build.sh already
overrode per-distro, so its behavior is unchanged.
Other cleanups landed in the same pass:
- Slim variant removed end-to-end (Dockerfile stage, --variant CLI
flag, slim suffix logic). Zero in-repo callers used it.
- compose.yml build-args INFINITO_PARENT_IMAGE / INFINITO_SRC_DIR now
carry the same `:?Run 'make dotenv'…` guard as the runtime entries,
so a missing .env fails fast with a clear message instead of with
buildkit's `base name should not be blank`.
Note: fork-CI (pull_request_target trusted-helpers path) will stay red
until main's scripts/image/push.sh and scripts/meta/wait/images.sh
adopt the INFINITO_* names too — that is the deliberate cost of
keeping a single SPOT instead of carrying a bridge alias.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Commit 5e0036dfa added a strict-read `${INFINITO_SRC_DIR:?...}` to the
docker buildx build-arg in push.sh. The CI workflow images-build-ci
invokes push.sh directly without sourcing scripts/meta/env/load.sh, so
the variable was never populated and the run aborted on attempt 1/7
(run 26200267778, job 77088881319). Sourcing load.sh inline keeps push.sh
self-sufficient regardless of caller and matches the existing pattern in
the lint/install/test wrappers under scripts/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit a77a57040 dropped the inline default from the Dockerfile's
INFINITO_SRC_DIR ARG so env/default.env stays the only SPOT. The
compose-driven build path already forwards the value via
compose.yml's build.args, but the standalone scripts/image/{build,push}.sh
paths (local `make build`, CI `🏗️ Build (<distro>)`) had not been
updated. Without --build-arg INFINITO_SRC_DIR=..., the ARG resolves
to empty inside the image, so `RUN /bin/bash ${INFINITO_SRC_DIR}/roles/...`
becomes `/bin/bash /roles/...` and exits 127.
Both scripts now assert INFINITO_SRC_DIR (sourced from
scripts/meta/env/load.sh) and pass it through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:
* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
env var → defaulted to infinito.localhost, which never resolved
because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
(env-builder, role default, run_one.yml docker exec, playwright.config.js,
lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
compose forwards it as build-arg, volume mount path, and container env;
every Python CLI + shell helper + CI workflow now reads it strictly
(os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
/opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
container) through compose.yml's environment block.
Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hard-coded self-path references rot the moment the file moves: a header
comment ``# utils/foo/bar.py`` at the top of a module, a docstring
mentioning ``utils/foo/bar.py``, or a docs entry linking
``utils/foo/bar.py`` from inside ``utils/foo/bar.py`` itself all silently
desync from reality after the next ``git mv``.
The new lint walks every project file, computes its POSIX-style relative
path from PROJECT_ROOT, and reports any line whose text contains that
path as a substring. Files at the repo root (no ``/`` in their relative
path) and setuptools-generated ``*.egg-info/`` artefacts are exempt.
Per-line opt-out via ``# nocheck: self-path-reference`` on the offending
line, registered in the suppression catalog.
Existing offenders cleaned up:
- 72 redundant ``# <self-path>`` header lines removed across roles,
cli, plugins, scripts, tests, and utils.
- 4 prometheus templates that told consumers to "update
<self-template-path>" rewritten to drop the redundant self-reference.
- 8 docstring/comment self-references in cli, plugins, tests, utils,
and the dashboard oidc.js.j2 header rewritten to describe the layout
without naming the file's own path.
- 17 legitimate usage examples (shell scripts, test data lists, one
template self-include doc-block) marked with ``nocheck:
self-path-reference``.
- ``container.sh`` heredoc usage strings switched to ``${0}`` so the
marker stays out of the user-visible help output.
- ``healthcheck/http.yml.j2`` doc-block rewritten to prose because
``{# ... #}`` Jinja comments cannot host an inner ``{# nocheck #}``
(the inner ``#}`` closes the outer comment).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- deploy.py: use find_provider_roles() to map service names (e.g. 'matomo')
to provider app IDs (e.g. 'web-app-matomo') before filtering deploy_ids;
forward SERVICES_DISABLED into the container via extra_env
- container.sh: rename INSPECT_CMD to CMD for consistency; fix indentation
- Makefile: expose CMD variable on exec target for non-interactive use
- environment.sh: source scripts/meta/env/all.sh for INVENTORY_DIR; add
inventory verification after SERVICES_DISABLED deploy using make exec
- variables.md: document CMD variable
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add scripts/meta/resolve/image/local.sh as SPOT for the local Docker
image tag (<repo-basename>/<distro>). Wire it into Makefile build
targets (IMAGE_TAG), compose.py (_base_env sets INFINITO_IMAGE), and
compose.yml (no default fallback). Fix build.sh to accept a pre-set
IMAGE_TAG without requiring REPO_PREFIX. Add make build + build-no-cache
to environment.sh test flow with correct section ordering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
