The repository installs mbake unpinned and shipped no .bake.toml, so the
Makefile's final newline depended on the mbake version:
- mbake 1.4.6 (e.g. the pkgmgr venv that wins on a host PATH) defaults
to ensure_final_newline = false: `mbake format` strips the newline and
`mbake format --check` accepts the stripped file.
- mbake 1.5.0 (what pip installs today) rejects the same file with
"Missing final newline", which is what CI reported on PR #1255.
So a local `make autoformat` could strip the newline, the local
lint-makefile stayed green, and CI failed; 6f653ce633 and #1262 had to
restore the newline by hand.
The tracked .bake.toml sets only ensure_final_newline = true, so every
other formatter rule keeps its default. autoformat.sh and makefile.sh
pass it with --config, so the result no longer depends on the version
or on the directory mbake runs from. Checked with both versions: main's
Makefile passes `--check --config .bake.toml`, and a copy without the
final newline fails it with "Missing final newline".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
make lint runs install-lint once and then dispatches its fifteen targets
in parallel. scripts/lint/mermaid.sh provisioned chrome-headless-shell
inside that parallel phase with `npx puppeteer browsers install
--install-deps`, whose `apt-get satisfy` carries no lock timeout. In a
fresh stack container another target held the apt lock, the shared
libraries stayed missing, the script only warned, and all 289 Markdown
files then failed on libnspr4.so (#795).
utils/install/lint/mermaid_cli.py now provisions the browser right after
mmdc, inside the serial install-lint step, as root with --install-deps
and as a user without it, because puppeteer refuses to install libraries
without root. A failed install deletes the chrome-headless-shell cache
folder and retries once under a 900 second timeout per attempt; a cut-off
download otherwise leaves a folder without its executable, and puppeteer
refuses every later install of that build. The install-lint stamp counts
as fresh only while the browser executable exists, so a recreated
container provisions it again.
mermaid.sh no longer calls apt-get or npx. It renders one minimal diagram
first and, when that fails, exits with mmdc's output instead of reporting
every Markdown file.
Verified: tests/unit/python/scripts/lint/test_mermaid.py stubs mmdc,
npx and apt-get. With a browser that cannot load libnspr4.so the script
exits 1 with that line, before any Markdown file renders, and mmdc is the
only tool it calls. Against main's mermaid.sh the same test fails: it
calls apt-get and npx and reports the files one by one. The installer and
stamp cases pass, shellcheck is clean.
Not verified: a full make lint in a recreated stack container. The race
reproduced in one of two runs in the issue.
feature/store-i18n changed the provisioning in mermaid.sh with 2f5f52bd1a
and dc67bce5f0 (timeout, deleting a cut-off browser). Both behaviours
moved into mermaid_cli.py here, so its merge takes this version of
mermaid.sh.
Closes#795
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The config held literals for values vars/main.yml already owned, and the two
drifted without a sound. actionTimeout carried `|| 30_000` for its whole life
while nothing ever set PLAYWRIGHT_ACTION_TIMEOUT, so every run used the
unscaled 30 s. expect had no entry at all, leaving Playwright's built-in 5 s to
govern every assertion over Tor while its neighbours were scaled by the onion
multiplier. That is what killed the second Nextcloud login attempt: an
expect(usernameField).toBeVisible() that gave up after 5 s inside a 150 s
budget.
vars/main.yml now owns the values in TEST_E2E_PLAYWRIGHT_CONFIG_TIMEOUTS, the
harness renders that map into the staged .env both the deploy and
scripts/tests/e2e/rerun-spec.sh already pass, and the config demands each name
through requiredTimeout() instead of inventing a fallback. The two `-e` flags
that fed the old pair are gone with it.
scripts/lint/playwright.sh stages the same config and runs `playwright test
--list`, which loads it, but built its stub env only from each role's
playwright.env.j2 and no such template carries a timeout. Without this it would
have aborted all 87 specs on a missing variable. It now takes the names from
the same map; the value is a placeholder because --list consumes none of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
87f762bd7 replaced the named helper lists in the deploy task and
rerun-spec.sh with a glob. There was a third stager. lint-playwright
keeps its own staging tree, so it kept its own list, and every MCP spec
failed to parse there: 8 of 87 roles, the same 'Cannot find module
./mcp-endpoint' the CI lint job reports.
All three lists held different sets - the deploy task four helpers,
rerun-spec.sh three, this one four again but not the same four - and none
of them the one the specs actually needed.
f14b1ab3f6 already fixed this drift once, by adding addon-gating here to
match the deploy stager. Adding a name to a list that exists three times
is what made a third occurrence certain; the list is gone now.
The rule learns about the third stager as well, since one that goes back
to naming helpers is exactly what nobody notices.
Four parse checks join the `lint-*` family, each using the language's own
parser rather than a reimplementation: `php -l`, `ruby -c`, `sqlfluff parse`,
and stylelint with an empty rule set (it raises CssSyntaxError whatever the
rules say, so the check stays syntax-only and grows no style opinion). Three
unit runners join `test-unit`: node:test, PHPUnit, minitest -- the first and
third ship with their runtime, so only PHPUnit adds a dependency.
Provisioning is best effort and the lint script decides whether it can run. A
machine without PHP must not fail the gate, but it must not look like success
either: the skip names how many files went unchecked. The first version of
these scripts read `git ls-files`, which inside the compose container returns
nothing (git refuses the bind-mounted repo as "dubious ownership") and reported
"No SQL files found." over 52 files -- green, and a lie. They use `find` now,
as the surrounding lint scripts already did.
sqlfluff needed two corrections before it parsed anything real. `%(name)s` is a
psycopg parameter marker, not SQL, so the placeholder templater is configured
rather than the files exempted; and the dialect follows each role's declared
engine (`meta/services.yml`), which is postgres for six roles and mysql for
svc-db-mariadb and pixelfed.
Recipes stay in scripts, not in the Makefile: the targets delegate, matching
every other target in the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The runner sent both streams to /dev/null and then classified by exit code, so
a failure produced one line -- "FAILED ansible-lint" -- with no code and no
message. A lint finding, a crash and a lock conflict were indistinguishable,
which is exactly the state a quality run left me in while ansible-lint was
clean on the host at the same commit.
Output now goes to a temp file, is discarded on success because ansible-lint is
verbose when it passes, and is written to stderr together with the exit code
when it fails. The accepted codes are untouched, so a passing run behaves
exactly as before. run_ruff already reports this way, which is why the TID252
error it hit was fixable on sight.
This changes no outcome. The failure that motivated it has not recurred, so the
new path is still unexercised against a real fault.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Host and container share this checkout through a bind mount but run as
different uids -- 1000 and 0, measured. Whichever writes .ruff_cache or
.import_linter_cache first owns it, and the other is then locked out:
ruff aborts with "Failed to create temporary file: Permission denied" and
test_import_contracts fails the same way, reporting a broken contract that
is in fact perfectly intact.
Point both at build/<tool>-cache-$(id -u). build/ is already 0777 and
gitignored, and this is the mechanism the lint stamp
(build/install-lint-<hash>.stamp) already uses to keep the two environments
apart. Passing --no-cache would have silenced the symptom at the cost of the
cache on every single run.
The pre-existing .ruff_cache and .import_linter_cache are nobody-owned and
cannot be removed without root; they are simply unused from now on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub shipped the concurrency queue property on 2026-05-07. actionlint 1.7.12 and act v0.2.89 both predate it: actionlint reports an unexpected key, act refuses to parse the whole workflow. Ignore the finding in the lint call, and hand act a copy with those lines stripped so local iteration survives once the workflows use the key.
Refs: rhysd/actionlint#654, nektos/act#6095, nektos/act#6152
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve conflicts, resolved by intent rather than by side.
Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.
web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').
handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.
svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.
Six follow-on failures the merge produced, fixed at the root:
- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
the prefix lint forbids; the pair now lives in utils/github/variant/ as
bundles.py and bundle_limits.py, with the -m entry point, the ruff
per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
three bare timeout literals; both route through gotoOnion and resolveTimeout
now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
distros that ship docker without it, the loop device is already detached when
the probe failed before attaching, and modprobe zfs may fail because the
/dev/zfs check decides.
make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The file list came from `find .`, so every path carried a leading './'.
shellcheck writes those paths verbatim into its `-f diff` output, and
`git apply` rejects a diff whose paths start with './' as "invalid path",
which aborts the whole autoformat run.
The autofix branch was therefore dead from the start and nobody noticed:
it only runs when shellcheck actually has something to correct, and until
now it never did. The first real finding turned a silent no-op into a red
`make quality`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container image ships with /var/lib/apt/lists emptied, so puppeteer's
--install-deps resolved no package name and chrome-headless-shell then died on
a missing libnspr4. All 262 markdown files with a mermaid block failed to
render, and the wrapper discarded the reason into /dev/null.
Refresh the lists when apt-get is present, and surface the provisioning output
instead of swallowing it, so a failure on a distribution puppeteer cannot serve
names its own cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The alias lint requires an i8 shorthand for every make target and the filesystem lint requires the execute bit on every script under scripts/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The separate CI job is gone; hadolint now runs as lint-dockerfile inside the parallel make lint fan-out through the standard wrapper, with the binary installed by the lint toolchain installer like every other tool. The SARIF gate script had no consumer left and is removed. Scope stays the root Dockerfile at the warning threshold, matching the old job.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container shellcheck flags the render_one body as unreachable
because the function is only invoked through xargs + bash -c; extend
the existing SC2329 directive.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mermaid lint rendered one file at a time, booting a fresh
chrome-headless-shell per diagram. Fan the renders out with xargs -P
capped by INFINITO_WORKER_CPU (the shared parallel-make constant) and
print collected failures serially afterwards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).
The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.
test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.
Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.
Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote the Firefox onion fixture (secure-context .onion prefs, SOCKS
remote-DNS) from web-app-dashboard to test-e2e-playwright as the single
source of truth; the dashboard copy now re-exports it. Stage the shared
fixture on all three paths (deploy runner, lint-playwright, rerun-spec).
gotoOnion: onion navigations default to waitUntil=domcontentloaded —
heavy chunked SPAs (Element) serialise 30+ bundle requests over Tor, so
the load event regularly exceeds the navigation cap; callers' explicit
onion-scaled selector waits cover app boot instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The lint stager only copied top-level *.js, so role-local subdirs
(fixtures/, addons/) never reached the staged tree: the dashboard's new
fixtures/onion-test require failed, and addons/*.spec.js were silently
never linted. Mirror the test-e2e-playwright staging (copy subdirs plus
the shared addon-gating.js) so --list covers the full spec tree; addon
specs of 11 roles are now actually linted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A default like ${BACKUP_KEY_PATH:-/tmp/swarm-nfs-backup.key} duplicates
a path contract that belongs in a SPOT (default.env or a role vars
file) and drifts silently. The guard covers every ALL-CAPS variable
whose inline default starts with a slash, allows per-line nocheck
suppression, and its failure message forbids dodging the syntax via
conditional assignments. Legacy offenders carry explicit nocheck
reasons (act tooling runs before default.env exists, POSIX TMPDIR
convention).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run the python suites through pytest-xdist when pytest is importable
(id-level collection parity with unittest discover verified per suite;
serial unittest stays the fallback), with -rP so warnings printed by
passing tests, like the update annotations, still reach the log. run.sh
self-provisions the dev extras through scripts/install/dev-extras.sh,
which stamps inside the venv and skips read-only venvs; make onboard
pre-warms the same path in the running container.
Hoist per-test cache resets and roles scans to setUpClass in the lookup
and resolvable suites (a per-test fingerprint-memo reset closes an
id()-reuse cache-poisoning flake the hoist exposed), inject generation
names into the nfs backup script test instead of sleeping, and narrow
ansible-lint scope to real ansible content. ansible-lint additionally
sits behind a content fingerprint gate that skips the run when nothing
it walks has changed; without a usable git (container bind mounts) the
gate disables itself and the full run remains.
Drop the import-linter layer and ignore for the deleted deps module;
the contract now locks compose -> common -> env. Measured on the host:
unit 101s to 20s, integration 90s to 24s, lint tests 43s to 19s,
repeated ansible-lint 53s to 0.4s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ansible-lint --fix exits 8 (RC.FIXED_VIOLATIONS) when it writes an auto-fix with zero failures remaining -- a normal outcome, same class as 0 (clean) and 2 (violations remain that --fix could not auto-resolve, same as the markdownlint/eslint rc<=1 pattern in this file). Previously only 0/2 were accepted, so a successful auto-fix could FAIL the whole autoformat run.
Also delete narration comments across the file that only restated the adjacent code; keep the two comments that flag a real trip-wire (status_dir crash-avoidance, write_status param doc).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a lint-packages target/workflow that parses the generated packaging metadata with each ecosystem's native parser (dpkg-parsechangelog, rpmspec, bash -n for the PKGBUILD), failing on any parser stderr so warnings cannot slip through. Wired into the make lint fan-out and ci-orchestrator's lint gate; an install group provisions the parsers (best-effort, cross-distro).
Hardens package_mirror so the generated changelogs parse without warnings: every RPM body line gets a '- ' continuation prefix (a column-0 '*' was read as a new %changelog entry -> 'bad date'), and the archived-releases notice is folded into the oldest kept entry instead of trailing after the final entry (free text there made dpkg-parsechangelog warn). Regenerates packaging/debian/changelog and the Fedora spec accordingly.
Adds a test-lint check (test_package_changelog_completeness) asserting both package changelogs list every CHANGELOG.md + docs/changelog release and that their inline window matches CHANGELOG.md, plus unit regressions for the RPM continuation prefix and the Debian footer fold.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds INFINITO_VENV_DIR=/opt/venvs/infinito to default.env (registered
in passthrough's STATIC_KEYS so make dotenv propagates it to .env) and
threads it through the build chain: compose.yml forwards it as a build
arg; scripts/image/push.sh asserts and --build-args it; the Dockerfile
declares the matching ARG/ENV and derives PYTHON/PIP/PATH from the
variable. scripts/meta/env/python.sh now prefers the value when set
before its fallback discovery so the SPOT is effective in env
resolution.
Fixes the dev image build regression where 'infinito --version' in
scripts/docker/entry.sh exited 127. Root cause: the Dockerfile RUN
SHELL uses bash -lc, which sources /etc/profile and resets PATH to
the Debian default, dropping ${INFINITO_VENV_DIR}/bin from it. The
fix invokes the script via its absolute path "${INFINITO_VENV_DIR}/bin/infinito"
so the post-install version check no longer depends on a PATH that
the login-shell init has just clobbered.
scripts/lint/wrapper.sh's existing PATH-reset note now references the
variable instead of the literal path. test-install-make.yml and
test-install-pkgmgr.yml read INFINITO_VENV_DIR from default.env up
front, pass it into their pkgmgr containers, and source the venv
activate script via the variable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`actions/checkout` configures git's http.extraheader to authenticate
as GITHUB_TOKEN, which update.yml scopes to `contents: read`. The
subsequent `git push --force` in open_pr.sh therefore tripped 403
("Permission to infinito-nexus/core.git denied to github-actions[bot]")
even though the token-minting step had already produced an App
installation token with `contents: write` in GH_TOKEN.
Drop checkout's extraheader and rewrite origin's URL to embed
GH_TOKEN as x-access-token before pushing, so the App's permissions
actually reach the wire.
Also pin two actionlint `-ignore` patterns in scripts/lint/action.sh
to silence false positives for the `client-id` input on
actions/create-github-app-token@v3 (the action's action.yml carries
the input and marks `app-id` deprecated; actionlint 1.7.12's bundled
schema lags behind the upstream release).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each role's Playwright spec is staged the same way `test-e2e-playwright` does at deploy time (central config + personas tree + service-gating) and `npx playwright test --list` runs against it. `--list` skips the browser launch and any test body, so this catches refactor regressions like missing helpers, broken exports, or syntax errors in seconds instead of waiting for a full deploy. Stub env values keyed off the variable-name suffix satisfy the few specs that build URLs or call `.replace()` at module-load time.
Parallelism caps at `INFINITO_WORKER_CPU` (same env knob as `scripts/make/parallel.sh`); output is replayed in `SPEC_FILES` order after all workers drain so stdout stays deterministic. `@playwright/test` ships as a devDep via the existing eslint `npm ci`. Workflow wires into `ci-orchestrator.yml` behind the `lint-gate`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.
Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
docker-compose command builder; down.py drops its now-unused
Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
meta/resolve/diff/affected_roles): plain \`docker compose\` instead
of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
reflect the missing --profile pair
Docs:
- docs/contributing/artefact/files/compose.yml.md: the
\"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
\`docker compose ... --profile ci config\` debug snippet drops the
flag; while touching the file the two em-dash clause separators in
the reproduction-profiles shell block are rewritten to colons
(documentation.md em-dash rule), and one list-item in
compose.yml.md is collapsed to one sentence per line for
consistency with its siblings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.
Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
switched from globbing env/*.env to targeting default.env directly;
test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
STATIC_ENV + every docstring/error message
Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename INFINITO_PARALLEL -> INFINITO_WORKER_ENABLED and add two derived
keys so consumers no longer multiply by their own factor: the env
generator resolves the cascade once at `make dotenv` time and bakes
the integer counts into `.env`.
Chain:
- INFINITO_WORKER_ENABLED (operator toggle, default true)
- INFINITO_WORKER_CPU = cpu_count() when ENABLED truthy, else 1
- INFINITO_WORKER_FETCH = CPU * 20 for I/O-bound bulk fetches,
collapsing to 1 when CPU == 1
Handlers:
- utils/env/handlers/infinito_worker.py (the prior single key)
removed
- utils/env/handlers/infinito_worker_cpu.py + infinito_worker_fetch.py
added; registry orders them after `passthrough` so the cascade
reads its dependency from the env-builder state.
Consumers (now 1-liners):
- tests/external/repository/test_urls_reachable.py reads
INFINITO_WORKER_FETCH directly.
- utils/update/base.py::resolve_max_fetch_workers() reads it directly;
utils/update/{docker,repository}.py keep using that helper plus a
ThreadPoolExecutor for the actual bulk fetch (with `dict(pool.map(...))`
satisfying PERF403).
- scripts/lint/autoformat.sh + docs/contributing/tools/make.md
rename INFINITO_PARALLEL -> INFINITO_WORKER_ENABLED.
Tests:
- EXPECTED_BASELINE_KEYS in test_dotenv_generator updated to track the
three new keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
galaxy-importer's legacy-role mode was dropped upstream in 0.5.0
(CLI no longer accepts --legacy-role / --namespace), and the project
does not publish roles to Ansible Galaxy. The check enforces a
publishing schema that never gets exercised, against an abandoned API
that can only be kept alive via pinning galaxy-importer==0.4.39 +
setuptools<81 in an isolated venv. The cost has stopped buying
anything.
Removed:
- scripts/lint/galaxy.sh
- utils/install/lint/galaxy_importer.py
- install_isolated_pip_pkg() in utils/install/pip.py (only galaxy_importer
used it)
- Makefile lint-galaxy target + the entry in the `lint:` aggregate
- pre-commit make-lint-galaxy hook
- The galaxy_importer ensure() call in _install_ansible_tools
- Test assertion against cli.galaxy_importer in test_main.py
- Documentation rows mentioning lint-galaxy / galaxy-importer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refactor scripts/install/lint.sh (~870 LOC bash) into a Python package
under utils/install/. scripts/install/lint.sh becomes a 7-line wrapper
that delegates to `python3 -m utils.install.lint`.
Layout:
- utils/install/{primitives,pip,npm,github_release,system_pkg}.py:
channel-agnostic installers
- utils/install/lint/<tool>.py: one ensure() per tool (actionlint,
shfmt, mbake, ruff, ansible_lint, galaxy_importer, markdownlint_cli2,
eslint, shellcheck, ansible_commands, ansible_collections)
- utils/install/lint/__main__.py: CLI + group dispatch + stamp logic
INFINITO_LINT_RUNNER dispatch:
- New env var (host | docker, default docker), tracked in env/static.env
+ passthrough handler + drift test.
- scripts/install/wrapper.sh routes `make install-lint` to host or
container based on the runner.
- scripts/lint/wrapper.sh already dispatched; container branch now uses
bash (not --login) so /etc/profile does not reset PATH, and forwards
BASH_ENV so the .env keys land in the lint script's env.
Stamp logic:
- Per-interpreter stamp at build/install-lint-<sha256(sys.executable)>.stamp
so host venv and container venv track installs independently despite
bind-mounted build/.
- `--force` drops the stamp; group args bypass the stamp.
Tests + docs:
- 43 unit tests under tests/unit/utils/install/{,lint/} cover primitives,
pip/npm/github_release/system_pkg fallback chains, and the main CLI
(stamp logic, group dispatch, error path).
- utils/install/README.md + utils/install/lint/README.md document the
structure and the per-tool ensure() contract.
- docs/contributing/tools/make.md install-lint row updated to mention
the runner dispatch + per-env stamp + install-lint-force.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`roles/__pycache__/` is created whenever Python imports the `roles`
package (it has an `__init__.py`), then `find` picks it up and
galaxy-importer fails the run with `FAIL __pycache__`. Same root cause
as the test_mapping.py fix in 635a0808d.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires JavaScript linting into the project for the first time, scoped to
the Playwright specs and persona helpers under `roles/**/files/**/*.js`.
Setup
-----
- `package.json` + `package-lock.json` pin eslint v9, eslint-plugin-
playwright v2, @eslint/js, globals.
- `eslint.config.js` (flat config) loads `eslint:recommended` and
`playwright/flat/recommended`. Browser globals are injected so
`page.evaluate(() => window.foo)` callbacks lint cleanly. Project-
idiomatic patterns are explicitly turned off with rationale: the
service-gating contract (req 006/019) uses test.skip on disabled
services; persona helpers branch on isVisible/isServiceEnabled before
expect; some specs deliberately use waitForTimeout / networkidle for
OIDC settle. `no-empty` allows the empty-catch idiom; the autofixer
for `prefer-web-first-assertions` and `no-wait-for-navigation` is
unsafe in this codebase (broken diff in dashboard.js), so both are
off pending a hand-migration pass.
- `scripts/lint/javascript.sh` mirrors the markdown.sh shape (skip on
missing, exit 127 with "make install-lint" hint).
- `scripts/lint/autoformat.sh` gets a `run_eslint` worker (treats rc≤1
as OK, like markdownlint).
- `scripts/install/lint.sh` gets `ensure_eslint` (npm-local install in
the repo root, matching the markdownlint pattern), a `javascript`
group, and `install_javascript_tools` is wired into the `all` group.
- `Makefile` exposes `lint-javascript` and folds it into the aggregate
`lint` target.
- `.github/workflows/lint-javascript.yml` follows the lint-makefile.yml
shape (Node 20, npm cache, `npm ci`, `make lint-javascript`); the
ci-orchestrator wires it into the lint pipeline + lint-gate `needs:`.
- `.gitignore` adds `node_modules/`.
Real lint findings fixed in this commit
---------------------------------------
- `roles/web-app-decidim/files/playwright.spec.js` was using
`runBiberFlow` / `runAdminFlow` without ever requiring `./personas`
(would have failed at runtime). Adds the require.
- Same file: two nutzlose `\\'csrf-token\\'` escapes in a double-quoted
query selector — eslint flagged, fixed.
- `roles/web-app-moodle/files/playwright.spec.js`:
`expect(await locator.count()).toBe(0)` →
`await expect(locator).toHaveCount(0)` (Playwright auto-retrying
assertion; safe `prefer-to-have-count` autofix).
Lint baseline: 0 errors, 285 warnings (mostly `no-unused-vars` and
`playwright/expect-expect` — addressable as follow-up cleanup, not
blocking).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shellcheck flagged three `[[ cond ]] && B || C` chains in
scripts/lint/autoformat.sh (ruff, shellcheck, mbake worker tails) as
SC2015: if `B` exits non-zero, `C` runs even when the test was true.
Rewrites each as `if [[ cond ]]; then B; else C; fi` so the success
and failure branches stay disjoint.
Reason: `make lint-shellcheck` aborted on these three info-level
findings; production-profile lint runs treat them as failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The reference at scripts/lint/makefile.sh:4 pointed at
github.com/mavyhq/bake which 404s; pypi metadata (mbake project) lists
the canonical homepage as github.com/EbodShojaei/bake.
Reason: tests/external/repository/test_urls_reachable.py was failing
with HTTP 404 on the stale URL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* .pre-commit-config.yaml: add 6 scoped lint hooks (lint-action,
lint-ansible, lint-python, lint-shellcheck, lint-markdown, lint-galaxy)
with `files:` patterns so a markdown-only commit no longer triggers
ansible-lint or galaxy-importer. Aggregate make-test, make-test-lint
(md-only branch) and pre-push make-test-external are kept.
* Makefile: drop `lint` from the `make test` dependency chain. Per-file-
type lint stages now live exclusively in pre-commit; running them again
from `make test` would just duplicate work.
* scripts/lint/autoformat.sh: add markdownlint-cli2 --fix and
ansible-lint --fix passes alongside ruff/shfmt/shellcheck.
* roles/svc-db-openldap/meta/main.yml: collapse the wrapped description
into a single line (still under Galaxy's 255-char limit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
