Add jlumbroso/free-disk-space before checkout (swap-storage off since this
workflow has no enlarge_swap.sh replacement) and raise registry-cache to 16g
and package blobstore to 8g to absorb post-prune in-job re-pulls. CentOS runs
were hitting "no space left on device" during layer registration; reclaiming
the host's unused toolchains and default Docker images fixes the root cause.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each role's Playwright spec is staged the same way `test-e2e-playwright` does at deploy time (central config + personas tree + service-gating) and `npx playwright test --list` runs against it. `--list` skips the browser launch and any test body, so this catches refactor regressions like missing helpers, broken exports, or syntax errors in seconds instead of waiting for a full deploy. Stub env values keyed off the variable-name suffix satisfy the few specs that build URLs or call `.replace()` at module-load time.
Parallelism caps at `INFINITO_WORKER_CPU` (same env knob as `scripts/make/parallel.sh`); output is replayed in `SPEC_FILES` order after all workers drain so stdout stays deterministic. `@playwright/test` ships as a devDep via the existing eslint `npm ci`. Workflow wires into `ci-orchestrator.yml` behind the `lint-gate`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`role_summary.py` no longer truncates to a configurable top-N: drops
`DEFAULT_TOP_N`, `TOP_N_ENV_KEY`, and `_read_top_n`; `_format_table`
renders all rows from `parse_role_times`. Renames the Markdown
heading and the corresponding GitHub Actions step in the four
`test-deploy-*` workflows from `Top role runtimes` to
`Role runtimes` for accuracy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tags `Free disk space` / `Enlarge swap space` / `Resolve repository owner` /
`Install jq` / `Deploy app` / `Dump runner state` across the four
`test-deploy-*` flows plus `images-build-ci`, `images-cleanup-ci`, and
`test-dns` — mirroring the existing 📊 prefix on `Top role runtimes`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.
Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.
Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.
Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Enable the ansible.posix.profile_roles callback alongside profile_tasks
so every deploy run emits a per-role runtime block. Each test-deploy
workflow now writes the Ansible log to ANSIBLE_LOG_PATH and runs a new
post-step that parses the block, aggregates duplicate role entries,
sorts by total duration, and writes the top-N (default 20, override via
INFINITO_PROFILE_TOP_N) as a Markdown table to $GITHUB_STEP_SUMMARY so
the table appears on the Actions job overview page. Includes 15 unit
tests covering aggregation, ANSI stripping, task/total filtering,
env-knob handling, and step-summary writes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.
Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
switched from globbing env/*.env to targeting default.env directly;
test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
STATIC_ENV + every docstring/error message
Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three inline `run:` blocks moved into `scripts/github/runner/`:
* `images_build_outputs.sh "$distros" "$image_tag"` — replaces the
matrix/image_tag computation in the "Compute matrix JSON" step; emits
both outputs to $GITHUB_OUTPUT in one call.
* `distros_matrix.sh "$distros"` — the matrix-JSON primitive
(`split(" ") | map(select(length>0))`), reused by test-dns.yml's
matrix step. Removes the jq duplicate that lived in both workflows.
* The two pre/post-build `Disk pressure` steps now invoke the already-
existing `disk_pressure_report.sh PRE-BUILD` / `POST-BUILD` instead of
duplicating the df/docker-system-df/buildx-du block inline.
Behaviour unchanged; the workflow runs the same commands via the SPOT
helpers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:
* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
env var → defaulted to infinito.localhost, which never resolved
because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
(env-builder, role default, run_one.yml docker exec, playwright.config.js,
lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
compose forwards it as build-arg, volume mount path, and container env;
every Python CLI + shell helper + CI workflow now reads it strictly
(os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
/opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
container) through compose.yml's environment block.
Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote the key from GHA-only (gha_passthrough) to always-emitted (passthrough)
so the deploy CLI can read it strictly via os.environ[...]. The previous
INFINITO_NO_BUILD layout left the key unset locally, which crashed
cli/.../up.py and cli/.../compose.py with KeyError after the env-pipeline
refactor that introduced gha_passthrough.
New semantics:
INFINITO_BUILD=1 build the local image (default; local mode)
INFINITO_BUILD=0 skip build, pull image (CI overrides via -e)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
INFINITO_LINT_RUNNER is fixed to host on this workflow, so no docker
interaction happens during the lint — the step was pure log noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tighten the names shown in the GitHub Actions UI summary so the
job rail reads as a function tag rather than a sentence:
test-code-{lint,unit,integration,external}: "🧪 Run X tests" -> "🧪 Test X"
security-codeql: "🔐 Check security" -> "🔐 Security"
images-build-ci: "🏗️ Build image (<distro>)" -> "🏗️ build (<distro>)"
test-install-make: "🛠️ Test make install" -> "🛠️ Makefile"
test-install-pkgmgr:"📦 Test pkgmgr install" -> "📦 PKGMGR"
test-environment: "🧑💻 Test Environment (<img>)" -> "🧑💻 Environment (<img>)"
fork-prereqs-ready: "🚦 Gate fork prerequisites" -> "🚦 Gate Dependencies"
code-quality-gate: (unnamed, fell back to id) -> "🚦 Gate Quality"
Step labels and workflow-level `name:` fields are untouched -- only
the per-job display names change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four test-deploy workflows (server/universal/workstation/local)
each started the full compose stack just so scripts/meta/resolve/
apps.sh could call its meta-CLI inside the container. The stack
spin-up dwarfed the actual discovery query.
Introduce INFINITO_APP_DISCOVERY_RUNNER (host|docker, default
`docker` for local dev) alongside the existing INFINITO_TEST_RUNNER
/ INFINITO_LINT_RUNNER dispatch knobs. In host mode apps.sh runs
the meta-CLI through the venv python directly; docker mode keeps
the existing `docker compose --profile ci exec` path.
Workflows switch to host mode and replace `make up` with
actions/setup-python@v5 + `make install`. make install was already
a prereq of make up, so the install cost is unchanged -- only the
docker pull + container start time disappears.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
build-ci-images previously depended on (fork-prereqs-ready,
test-code-gate). That let an image build start while lint or
security-codeql were still in flight. Reorder so the build gate is
code-quality-gate, which already aggregates lint-gate + test-code-gate
+ security-codeql -- a single barrier instead of two halves.
The reordering also removes the redundant `code-quality-gate` edge
on test-dns: with build-ci-images now downstream of code-quality-gate,
test-dns inherits the dependency transitively through build-ci-images.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/docker/entry.sh's two compile-control reads (INFINITO_COMPILE
and INFINITO_COMPILE_SILENCE) crashed during Dockerfile RUN because
that context has no env-pipeline: env/default.env exists in the repo
but no make/BASH_ENV/load.sh chain materialises it into the running
shell. The bare reads under `set -u` triggered unbound-variable exit
even though the keys were registered in default.env + passthrough.
Move build/silence control from env to script flags so entry.sh is
self-contained at bootstrap time. Each flag now triggers its action
directly inside the parse loop -- no COMPILE/SILENCE state vars:
--compile -> run_local_build (streaming)
--compile-silent -> run_local_build >/dev/null 2>&1
-- -> end flag parsing, rest is the exec target
Drop INFINITO_COMPILE + INFINITO_COMPILE_SILENCE from the whole
registry surface: env/default.env, passthrough.STATIC_KEYS, the
infinito_compile handler (gone entirely), compose.yml's env
injection block, the dotenv-generator baseline test, the
build_env unit-test fixture, and the now-no-op `-e INFINITO_COMPILE=0`
in test-install-{make,pkgmgr} workflows.
Dockerfile RUN switches to `entry.sh --compile -- true`; runtime
ENTRYPOINT receives no flags so the default (no-compile) holds and
exec hands off to systemd via the CMD.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These three workflows install only their own toolchain (ansible-core +
ansible-lint, ESLint via npm, mbake) and then call `make lint-<type>`.
The make target has `install-lint` as a prereq which dispatches on
INFINITO_LINT_RUNNER (default `docker` from env/default.env), tries
`make up`, imports `cli.administration.deploy.development`, and crashes
because ruamel.yaml / the full Python toolchain isn't installed on the
runner.
Set `INFINITO_LINT_RUNNER: host` as job env so install-lint stays on
the host (delegates to `python3 -m utils.install.lint`) and the lint
wrappers exec the per-type scripts directly. lint-ansible passed by
accident last run but had the same bug -- fixed preventively.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four test-code workflows (lint/unit/integration/external) now
run unittest discover directly on the GitHub-hosted runner via
INFINITO_TEST_RUNNER=host, bypassing the docker compose container.
A fresh Python 3.11 from actions/setup-python feeds make install,
which builds the venv on the runner — no `make up`, no docker
image pull, no `INFINITO_IMAGE_TAG` / `NIX_CONFIG` plumbing.
The orchestrator reverses the dependency direction: build-ci-images
now needs test-code-gate so a broken commit never produces images.
Tests start in parallel with fork-prereqs-ready instead of waiting
for the build chain.
Galaxy lint workflow stays dropped alongside the earlier
galaxy-importer removal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the shell helpers under scripts/github/ into five subfolders
that match how the workflow files consume them:
runner/ host-runner hygiene (disk, swap, Docker daemon, diagnostics)
cancel/ run cancellation on PR-close and branch-delete events
resolve/ derivation of workflow inputs and outputs from repo state
update/ scheduled Update: Versions pipeline
release/ release-time gates
Drop the now-redundant subfolder prefix from each filename, e.g.
dump_runner_state.sh becomes dump_state.sh and cancel_pull_request_runs.sh
becomes pull_request_runs.sh. Each subfolder carries a README that
describes its scope per documentation.md rules; the top-level README
points at the subfolders without inventorying files.
Propagate the new paths through ten workflow files, three docs
(workflow.md, pipeline.md, secrets.md), and two tests. Fix a real
runtime bug in enlarge_swap.sh, which still referenced the
pre-rename dump_runner_state.sh path. Extend the inline-block-size
lint hint to describe the new subfolder convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workflow-driven update PRs opened with GITHUB_TOKEN do not start
further workflow runs, so the CI orchestrator never executed against
them. Each PR-opening job in update.yml now mints a short-lived
GitHub App installation token via actions/create-github-app-token@v2
and forwards APP_SLUG to open_update_pr.sh, which derives the
verified-commit identity from the App user.
The job-level permissions block drops to read-only since write access
is provided exclusively by the App token. Behaviour with the secrets
unset is fail-loud: the token step fails, the PR step is skipped,
and no PR with an unintended identity is ever created.
Requires the BOT_APP_ID and BOT_APP_PRIVATE_KEY repository secrets
documented in docs/contributing/tools/github/actions/secrets.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- New `utils/github/playwright_summary.py` parses every
`playwright-junit.xml` under a given artefact directory and prints a
markdown table to stdout with one row per test case: App, Test,
Status (🟢 passed / 🔴 failed / 🔵 skipped), Duration (sub-second
precision), Message. Sub-second formatting keeps useful per-test
signal that integer rounding would erase. Edge cases (missing dir,
no junit files, all-unparseable) emit a friendly placeholder note
instead of crashing. Covered by 26 unit tests in
`tests/unit/utils/github/test_playwright_summary.py`.
- `test-deploy-server.yml` runs the script with `if: always()` after
the deploy step and pipes its output into `$GITHUB_STEP_SUMMARY`, so
every matrix job's run page surfaces a per-app test table without
needing to download artefacts.
- Playwright config adds the `github` reporter so failing tests also
emit inline `::error file=...,line=...::`-annotations directly on the
workflow run page and in PR diffs.
- New lint `tests/lint/filesystem/test_plugin_paths_files.py` enforces
that files under the Ansible plugin source directories declared in
`ansible.cfg` (`action_plugins`, `filter_plugins`, `lookup_plugins`,
`library`, `module_utils`) are Python source files (README.md
exempt). Paths are read live from `ansible.cfg` so a future config
edit automatically extends the lint's coverage. Counterpart to the
existing `test_scripts_folder_files.py` lint that enforces
`scripts/` contains only `.sh`.
Known limitation (pre-existing pipeline behaviour, not introduced by
this change): when a role runs through multiple matrix variants, the
container path `/var/lib/infinito/logs/test-e2e-playwright/<app_id>/`
has no variant component, so each variant overwrites the previous
variant's reports. The summary therefore reflects only the LAST
variant per role. A future change to namespace the reports per
variant (or copy them after each variant pass) will make the table
cover every variant without touching the script or workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops `MAX_LINES` from 12 to 11 in `test_inline_script_block_size.py`,
catching the five blocks that were sitting exactly on the previous
threshold. Pure-bash blocks move to dedicated `.sh` files invoked via
`ansible.builtin.script:` or `run: bash …`; the keycloak heredoc with a
templated JSON payload splits into a `set_fact` plus a folded
`printf | kcadm … -f -` one-liner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Forwards `PLAYWRIGHT_KEEP_ALL` from host env / GitHub Actions repo var
through `TEST_E2E_PLAYWRIGHT_KEEP_ALL` (inventory-overridable) into the
container and flips trace / screenshot / video to `on` for every test
when truthy.
SPOT for the propagation chain lives in
`docs/contributing/actions/testing/playwright.md#artefact-retention-`;
CI Configuration and the agent iteration loops link to it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires JavaScript linting into the project for the first time, scoped to
the Playwright specs and persona helpers under `roles/**/files/**/*.js`.
Setup
-----
- `package.json` + `package-lock.json` pin eslint v9, eslint-plugin-
playwright v2, @eslint/js, globals.
- `eslint.config.js` (flat config) loads `eslint:recommended` and
`playwright/flat/recommended`. Browser globals are injected so
`page.evaluate(() => window.foo)` callbacks lint cleanly. Project-
idiomatic patterns are explicitly turned off with rationale: the
service-gating contract (req 006/019) uses test.skip on disabled
services; persona helpers branch on isVisible/isServiceEnabled before
expect; some specs deliberately use waitForTimeout / networkidle for
OIDC settle. `no-empty` allows the empty-catch idiom; the autofixer
for `prefer-web-first-assertions` and `no-wait-for-navigation` is
unsafe in this codebase (broken diff in dashboard.js), so both are
off pending a hand-migration pass.
- `scripts/lint/javascript.sh` mirrors the markdown.sh shape (skip on
missing, exit 127 with "make install-lint" hint).
- `scripts/lint/autoformat.sh` gets a `run_eslint` worker (treats rc≤1
as OK, like markdownlint).
- `scripts/install/lint.sh` gets `ensure_eslint` (npm-local install in
the repo root, matching the markdownlint pattern), a `javascript`
group, and `install_javascript_tools` is wired into the `all` group.
- `Makefile` exposes `lint-javascript` and folds it into the aggregate
`lint` target.
- `.github/workflows/lint-javascript.yml` follows the lint-makefile.yml
shape (Node 20, npm cache, `npm ci`, `make lint-javascript`); the
ci-orchestrator wires it into the lint pipeline + lint-gate `needs:`.
- `.gitignore` adds `node_modules/`.
Real lint findings fixed in this commit
---------------------------------------
- `roles/web-app-decidim/files/playwright.spec.js` was using
`runBiberFlow` / `runAdminFlow` without ever requiring `./personas`
(would have failed at runtime). Adds the require.
- Same file: two nutzlose `\\'csrf-token\\'` escapes in a double-quoted
query selector — eslint flagged, fixed.
- `roles/web-app-moodle/files/playwright.spec.js`:
`expect(await locator.count()).toBe(0)` →
`await expect(locator).toHaveCount(0)` (Playwright auto-retrying
assertion; safe `prefer-to-have-count` autofix).
Lint baseline: 0 errors, 285 warnings (mostly `no-unused-vars` and
`playwright/expect-expect` — addressable as follow-up cleanup, not
blocking).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tightens "🧹 Lint Galaxy (galaxy-importer)" → "🧹 Lint Galaxy" and
"🧹 Lint Makefile (mbake)" → "🧹 Lint Makefile" so the job titles
are uniform with the other lint-* jobs and don't lock the displayed
name to the underlying tool implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 971fda535: the new "Generate role include files" step
failed with `ModuleNotFoundError: No module named 'utils'` because
cli/build/include/__main__.py imports `utils.cache.yaml` and friends.
Locally scripts/meta/env/python.sh sets `PYTHONPATH:=.` via the
Makefile env layer, but the CI step invokes `bash scripts/setup.sh`
directly and bypasses that layer. Inline `PYTHONPATH=.` on the run
line; verified locally with
`PYTHONPATH=. python3 -c "from utils.cache.yaml import load_yaml"`
and `PYTHONPATH=. python3 -m cli.meta.categories.invokable -s "-"`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes for the lint-ansible CI job (failing run 25604499233):
1. Add `community.docker` to requirements/requirements.galaxy.yml so
`ansible-galaxy collection install -r ...` pulls it in CI. The
collection is referenced in 9+ files (web-app-discourse handlers,
svc-db-mariadb/openldap, sys-svc-compose/container, web-app-keycloak/
decidim) but was missing from the requirements list, so syntax-check
could not resolve `community.docker.docker_container`.
2. Add a "Generate role include files" step to .github/workflows/
lint-ansible.yml. `tasks/groups/*-roles.yml` are gitignored generated
artefacts (per tasks/groups/.gitignore: `*-roles.yml`) referenced
from tasks/stages/*.yml; without them ansible-lint reports a fatal
load-failure on tasks/groups/desk-roles.yml. The new step runs
`PYTHON=python3 bash scripts/setup.sh` (same path the local `make
setup` target invokes); pyyaml added to the pip install line as
that's the only non-stdlib dep the setup script needs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lint-ansible workflow runs `ansible-playbook playbook.yml --syntax-check`
inside scripts/lint/ansible.sh, which needs every collection the playbook
references to resolve module names. Without them CI would fail with
"couldn't resolve module/action ...". Add an explicit
`ansible-galaxy collection install -r requirements/requirements.galaxy.yml`
step right after the pip install of ansible-core/ansible-lint so the
syntax-check can fully load community.general / hetzner.hcloud /
kewlfft.aur on a fresh runner.
lint-galaxy needs no parallel change — galaxy-importer in --legacy-role
mode treats unresolved modules as warnings, not fatal errors, so the
workflow can stay collection-less.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Narrow the test-deploy-{server,universal,workstation} matrix to roles
actually impacted by the branch's diff against origin/main, expanded
transitively over run_after + dependencies + shared services.
Why: full-matrix deploy CI on every PR is wasteful; explicit per-PR
whitelisting is error-prone. The resolver derives the affected set
automatically while remaining fail-safe.
Behaviour (in resolve_effective_whitelist.sh precedence order):
- whitelist input "__ALL__" -> empty whitelist (force full deploy)
- whitelist input non-empty -> verbatim
- whitelist input empty -> diff vs origin/main
- empty diff or any path outside roles/<role>/ -> __ALL__
- all paths under roles/<role>/ -> reverse closure
Fail-safe: any seed that is not modellable in CombinedResolver
(no application_id and not referenced by any role's run_after) and
any resolver runtime error MUST trigger a fall-back to __ALL__.
The closure never silently shrinks the deploy matrix.
test-deploy-local.yml is intentionally exempt (local/manual dispatch
keeps explicit whitelist semantics).
Documented in pipeline.md and workflows.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Narrow the test-deploy-{server,universal,workstation} matrix to roles
actually impacted by the branch's diff against origin/main, expanded
transitively over run_after + dependencies + shared services.
Why: full-matrix deploy CI on every PR is wasteful; explicit per-PR
whitelisting is error-prone. The resolver derives the affected set
automatically while remaining fail-safe.
Behaviour (in resolve_effective_whitelist.sh precedence order):
- whitelist input "__ALL__" -> empty whitelist (force full deploy)
- whitelist input non-empty -> verbatim
- whitelist input empty -> diff vs origin/main
- empty diff or any path outside roles/<role>/ -> __ALL__
- all paths under roles/<role>/ -> reverse closure
Fail-safe: any seed that is not modellable in CombinedResolver
(no application_id and not referenced by any role's run_after) and
any resolver runtime error MUST trigger a fall-back to __ALL__.
The closure never silently shrinks the deploy matrix.
test-deploy-local.yml is intentionally exempt (local/manual dispatch
keeps explicit whitelist semantics).
Documented in pipeline.md and workflows.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add tests/lint/repository/test_inline_script_block_size.py: scans
roles/, tasks/, playbooks/ and .github/ YAML for literal block scalars
(`|` / `|-` / `|+`) following the keys `shell`, `command`, `script`,
`raw` (Ansible) and `run` (GitHub Actions). Counts logical lines —
trailing-`\` continuations collapse to one line, blank lines are
ignored — and fails when a block exceeds 12.
FOLDED scalars (`>` / `>-` / `>+`) are exempt: they are one shell
statement after YAML folding, even if they span many source lines for
readability.
The walk goes through `utils.cache.files.iter_project_files(...)` per
`docs/agents/action/testing.md` so the shared project-tree cache is
reused across the lint suite.
The fail message tells the author exactly how to fix: Ansible tasks
get the built-in `script:` module (collapses the task to one line);
GitHub Actions workflows extract into `scripts/github/<name>.sh` and
call it via `run: bash scripts/github/<name>.sh` (or
`_helpers/scripts/github/<name>.sh` when the workflow checks out the
repo into `_helpers/`).
Extract the pre-existing literal-block offenders so the new lint
ships green:
Ansible (script: module, file under <role>/files/)
web-app-friendica sync_addon.sh
web-app-fider create_admin_user.sh
web-app-fider configure_oidc_provider.sh
web-app-mediawiki install_or_update.sh
web-app-mediawiki composer_install_extension.sh
web-app-xwiki debug_superadmin.sh
GitHub Actions (scripts/github/)
pick_distro.sh
guard_release.sh
resolve_release_inputs.sh
disk_pressure_report.sh
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-up lint fixes after the bluesky bridge SPI deletion +
WebFetch domain auto-additions during the iteration session:
- Run `bash scripts/lint/sort_claude_settings.sh` (via `make exec`
because the host sandbox denies writes to `.claude/settings.json`).
Sorts `permissions.allow` (265 entries) — Claude Code had appended
new WebFetch domain grants to the end as the iteration ran, which
tripped `tests.lint.repository.test_claude_settings_sorted`.
- Comment out `language: java-kotlin` in
`.github/workflows/security-codeql.yml`. The previous commit
removed the only Java source in the repo (the unused
`roles/web-app-bluesky/files/keycloak-bluesky-bridge/` SPI
scaffold), and `tests.integration.workflows.test_security_coverage`
fails when CodeQL has a language enabled with no matching files.
Re-enable once a real Java component lands again.
`make test` now green (1833/1833). External URL test still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the previous OR-based assertion in scripts/tests/environment
(which trivially passed because the Nexus log always grows from
health probes) with a richer set of helpers split across two files
under utils/ for SRP/SPOT.
scripts/tests/environment/utils/cache.sh (new):
* expected_system_proxy_for_runtime maps DEV_RUNTIME_IMAGE to the
Nexus proxy that handles its system package manager (apt-debian,
apt-ubuntu, yum-fedora, yum-rocky; empty for Manjaro/Arch where
pacman is not cached).
* nexus_repo_requests counts only /repository/<name>/... entries in
the Nexus request log, optionally scoped to one proxy. Filters out
health probes, admin REST calls, and Web-UI traffic.
* registry_cache_flows counts every cache-flow status (HIT, MISS,
REVALIDATED, UPDATING, STALE) so cold deploys also count.
* assert_caches_used fails with a per-cache breakdown when any of
three independent conditions is violated: total Nexus repo
traffic = 0, the runtime's expected system proxy has 0 cached
requests, or the deploy itself produced 0 registry-cache flows.
* probe_caches actively fetches a known upstream resource through
each cache (apt-debian/dists/bookworm/Release for the package
cache, /ca.crt for the registry cache).
* probe_did_inner_build drives a throwaway docker build whose
apt-get update runs against deb.debian.org. The build container
is placed on the compose network and pinned via --add-host to the
frontend IP, exactly like the runtime override that
roles/sys-svc-compose/files/compose.py injects for app builds.
After the build the test snapshots
``docker logs infinito-package-cache-frontend`` and asserts the
frontend recorded new requests, proving DiD apt traffic flows
through nginx -> Nexus apt-debian and not directly to upstream.
DOCKER_BUILDKIT=0 because BuildKit refuses --network=<compose-net>.
scripts/tests/environment/utils/common.sh (renamed from lib.sh):
keeps the bootstrap, constants, inventory inspect, and
assert_http_status; cache helpers moved out so the file is
single-concern.
scripts/tests/environment/07_deploy_minimal.sh sources both files;
order is snapshot, deploy, probe_caches, probe_did_inner_build,
snapshot, assert_caches_used. Active probes contribute traffic that
the AND assertion verifies, so the assertion is robust on deploys
that do not naturally trigger inner-build apt traffic.
.github/workflows/test-environment.yml: propagate DEV_RUNTIME_IMAGE
into the inner ``docker run`` so per-distro probe target and proxy
expectation resolve correctly inside the orchestrator.
.gitignore + .dockerignore: also ignore .claude/settings.local.json,
which compose mounts as /dev/null and which the classic docker
builder would otherwise try (and fail) to read into the build
context.
Verified end-to-end with five iterations of
``make act-workflow ACT_WORKFLOW=.github/workflows/test-environment.yml
ACT_JOB=test-environment ACT_MATRIX='dev_runtime_image:debian:bookworm'``.
The final iteration reports +81 apt-debian, +235 registry-cache
flows, +13 frontend log lines from the DiD probe, and "[OK] both
cache stacks observed activity" plus "[OK] DiD inner-build flowed
through cache frontend".
The wrapper convention applies inside Ansible roles. Bootstrap shell
scripts and CI workflows live outside `roles/` and either pre-date the
wrapper's installation or run on hosts that never see it; flagging them
adds noise without protecting the contract. Add a roles/ path-prefix
filter, rename the test method, drop the now-redundant `nocheck:
raw-docker` headers from the 6 GitHub workflows, and rewrite the two
suppression-doc examples on realistic in-scope scenarios (a per-line
marker on an Ansible task, a file-level marker on a role-bundled .sh).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the local `# no-raw-docker: skip` regex in favour of
utils.annotations.suppress.is_suppressed_at / is_suppressed_in_head
under rule key `raw-docker`, gaining per-line opt-outs (`# noqa:
raw-docker` / `# nocheck: raw-docker`) on top of the existing
file-level head suppression. Migrate the 6 GitHub-hosted-runner
workflow markers to the new grammar, lift the head scan to the catalog
default of 30 lines, and add the rule to the suppression catalog with
two examples.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reintroduce the no_raw_docker integration test that was lost in the
flat-to-clustered tests/integration reorg, scoped to .yml/.yaml/.j2/.sh
and powered by utils.cache.files.iter_non_ignored_files plus read_text
so file enumeration respects .gitignore and content reads are
lru-cached. Catch two argv-/inline-form blind spots that the original
single-line regex missed (argv: - docker / shell: "docker exec ...")
and add a `no-raw-docker: skip` header marker for files where the
container wrapper is genuinely unavailable.
Apply the marker to the 6 GitHub-hosted-runner workflows that cannot
reach the wrapper, and rewrite the 7 role-task argv invocations
(BigBlueButton, Discourse, Matomo, Joomla 03/04/06) onto
'container' so the engine stays swappable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
