ensure_php_toolchain installed the php and composer binaries and stopped
there. A distro ships the interpreter and its extensions as separate
packages, and ensure_command_present returns early once shutil.which
finds php - which it always does since the image bakes the interpreter -
so the extensions were never provisioned on any path.
composer then refused to resolve the vendor tree with
"phpunit/phpunit requires ext-dom * -> it is missing from your system"
and test-unit-php failed in every workspace job. The loaded ini list in
those logs shows 20-mbstring.ini present and no 20-dom.ini, 20-xml.ini
or 20-xmlwriter.ini, i.e. php-xml missing.
Extensions are now probed against php -m and installed from a per-manager
mapping, then re-probed so a package that does not deliver the extension
raises instead of leaving composer to fail later. The loop sits in
ensure_php_toolchain because both entry points pass through it: the image
build via utils.install.interpreters and the test run via
scripts/install/php.sh, so a prebuilt image is repaired at test time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test-unit-php and test-unit-ruby exited 0 whenever the interpreter or the
vendor tree was missing, so both suites were silently absent from every
gate. They now provision their toolchain through install-php/install-ruby
and fail loudly when that is impossible.
The three ensure_*_present helpers collapse into ensure_command_present
in system_pkg, and the ruby runner rejects an empty suite because
minitest reports success when nothing was required.
test-unit-javascript ran `node --test` without declaring where node
comes from, while its python sibling declares `install`. On a bare
distro image the target died with `exec: node: not found` (127), which
took down the pre-commit hook and all five test-workspace jobs.
The installer stack already knows how to pull a command from the system
package manager; node was simply missing from the table. Wiring the
target to install-node provisions it the same way php, ruby and npm are
provisioned, and fails loudly when the runtime cannot be installed
rather than skipping the suite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four parse checks join the `lint-*` family, each using the language's own
parser rather than a reimplementation: `php -l`, `ruby -c`, `sqlfluff parse`,
and stylelint with an empty rule set (it raises CssSyntaxError whatever the
rules say, so the check stays syntax-only and grows no style opinion). Three
unit runners join `test-unit`: node:test, PHPUnit, minitest -- the first and
third ship with their runtime, so only PHPUnit adds a dependency.
Provisioning is best effort and the lint script decides whether it can run. A
machine without PHP must not fail the gate, but it must not look like success
either: the skip names how many files went unchecked. The first version of
these scripts read `git ls-files`, which inside the compose container returns
nothing (git refuses the bind-mounted repo as "dubious ownership") and reported
"No SQL files found." over 52 files -- green, and a lie. They use `find` now,
as the surrounding lint scripts already did.
sqlfluff needed two corrections before it parsed anything real. `%(name)s` is a
psycopg parameter marker, not SQL, so the placeholder templater is configured
rather than the files exempted; and the dialect follows each role's declared
engine (`meta/services.yml`), which is postgres for six roles and mysql for
svc-db-mariadb and pixelfed.
Recipes stay in scripts, not in the Makefile: the targets delegate, matching
every other target in the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A single anonymous GET for the pinned hadolint binary completed its TLS
handshake and its request, then got a clean FIN with no status line after
3.6s. download_file was a bare urlopen with no retry, and make install-lint
is a prerequisite of autoformat, lint and every lint-* target -- so one shed
socket took down the whole quality gate. The same asset downloaded fine five
times elsewhere in that run, 15s after the failure on another distro.
Retry four times with exponential backoff, catching (OSError, HTTPException).
Two names, not the obvious four: ssl.SSLError is an OSError but NOT a
URLError, so the narrow tuple would have missed it, while OSError subsumes
URLError, ConnectionError, TimeoutError and gaierror in one token.
HTTPException is needed separately because IncompleteRead is not an OSError.
HTTPError is caught FIRST and re-raised immediately for any status outside
{408,429,500,502,503,504}. That ordering is load-bearing -- HTTPError is
itself an OSError, so a broader catch first would swallow it -- and it is
what keeps this robustness rather than masking: a 404 is exactly what a
stale version pin looks like after roles/sys-lint is bumped, and it still
fails on the first attempt with the URL. The final attempt re-raises the
original with its traceback, and write_bytes stays in the else branch so a
full disk is never retried.
apt was the other uncached, unretried input in the same target (its default
Acquire::Retries is 0), so give both invocations -o Acquire::Retries=3,
matching what eight Dockerfiles in this repo already do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).
The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.
test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.
Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.
Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
system_pkg.py's apt-get update/install failed immediately ('Could not get lock /var/lib/dpkg/lock-frontend') when a concurrent deploy held the dpkg lock, so make autoformat/test could not run while a deploy was in progress. Pass -o DPkg::Lock::Timeout=600 to both apt-get invocations so they wait for the lock instead of failing, and update the unit test to assert the lock-wait. Scope is the dev/lint package installer only; the Ansible deploys use ansible.builtin.apt and are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Refactor scripts/install/lint.sh (~870 LOC bash) into a Python package
under utils/install/. scripts/install/lint.sh becomes a 7-line wrapper
that delegates to `python3 -m utils.install.lint`.
Layout:
- utils/install/{primitives,pip,npm,github_release,system_pkg}.py:
channel-agnostic installers
- utils/install/lint/<tool>.py: one ensure() per tool (actionlint,
shfmt, mbake, ruff, ansible_lint, galaxy_importer, markdownlint_cli2,
eslint, shellcheck, ansible_commands, ansible_collections)
- utils/install/lint/__main__.py: CLI + group dispatch + stamp logic
INFINITO_LINT_RUNNER dispatch:
- New env var (host | docker, default docker), tracked in env/static.env
+ passthrough handler + drift test.
- scripts/install/wrapper.sh routes `make install-lint` to host or
container based on the runner.
- scripts/lint/wrapper.sh already dispatched; container branch now uses
bash (not --login) so /etc/profile does not reset PATH, and forwards
BASH_ENV so the .env keys land in the lint script's env.
Stamp logic:
- Per-interpreter stamp at build/install-lint-<sha256(sys.executable)>.stamp
so host venv and container venv track installs independently despite
bind-mounted build/.
- `--force` drops the stamp; group args bypass the stamp.
Tests + docs:
- 43 unit tests under tests/unit/utils/install/{,lint/} cover primitives,
pip/npm/github_release/system_pkg fallback chains, and the main CLI
(stamp logic, group dispatch, error path).
- utils/install/README.md + utils/install/lint/README.md document the
structure and the per-tool ensure() contract.
- docs/contributing/tools/make.md install-lint row updated to mention
the runner dispatch + per-env stamp + install-lint-force.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5