`security-apparmor-teardown` ran `aa-teardown`, which unloads every
profile including dockerd's in-memory `docker-default`. dockerd only loads
that profile when it starts, so every container start afterwards that
requests it fails until dockerd restarts. Since #1067 runs the teardown on
the GitHub runners, BuildKit failed each RUN of the swarm node image three
times (run 39332343490, 39 of 40 swarm jobs):
runc run failed: unable to start container process: error during
container init: unable to apply apparmor profile: apparmor failed to
apply profile: write .../attr/apparmor/exec: no such file or directory
The teardown now removes each loaded profile by name through securityfs
and skips `docker-default` and child profiles, which leave with their
parent. Dev hosts gain the same: no dockerd restart is needed after
`make environment-bootstrap`.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
RFC 6761 reserves .test for testing and .example for documentation, and the
project's own domain was doing both jobs under the documentation name. It is
what default.env hands every local deploy and what the test fixtures assert
against, so it is a testing name.
Rename it wherever it stands for the running machine: the environment
defaults, the development inventory, the workspace and trust scripts, the test
suites, and the four documents that describe real behaviour - the DNS mapping
to loopback, the browser CA trust, WSL2 resolution. Those documents would have
started lying the moment default.env changed.
The sandbox allowlist moves with them, since egress is matched per pattern and
nothing resolves the old name any more, and the sandbox document that mirrors
that list follows it.
Two places keep .example on purpose: the link-style sample in the
documentation guide, which is documentation about documentation, and every
generic fixture name elsewhere in the suite - a.example, foo.example and their
kin. Those are a larger sweep of their own and are not touched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
default.env declares INFINITO_DOMAIN and everything is meant to derive from it.
Nothing checked that, so copies accumulated. A copy is a second source nobody
updates, and it fails in the least useful way: the code keeps running and
quietly talks about the wrong host. integration_mastodon is the worked example
-- it asserted the provisioned partner equalled a clearnet literal, so the spec
passed on a clearnet node and failed on an onion one while the product was
right in both.
tests/lint/repository/spot/test_no_hardcoded_primary_domain.py now fails on the
literal anywhere in tracked source. It reads the value it hunts for FROM
default.env, so the rule follows the SPOT instead of repeating it and would
otherwise be its own first violation.
Scope, and why each exclusion is not a loophole: default.env itself is the
source; .md is documentation; tests/ and scripts/tests/ hold fixtures that must
assert against a named host; inventories/ is where writing the real values is
the entire point; gitignored output carries the value because it correctly
derived it. Everything else is in.
Two real hardcodes are repaired rather than suppressed.
utils/domains/default_primary.py mirrors utils/roles/lifecycle.py -- one
function reading one key through parse_static_env -- and replaces the module
constant in utils/cleanup/nginx_vhosts.py and the argparse default in
cli/meta/domains/__main__.py. Both resolve to the same value as before.
Seven documentation sites become <primary-domain> placeholders instead of
carrying a nocheck: a Python docstring cannot host one without polluting the
docstring, and a placeholder does not rot when the default changes. The single
nocheck is scripts/system/tls/trust/wsl2.sh:61, where the marker is matched
verbatim against what an earlier run wrote into the Windows hosts file --
deriving it would orphan that block instead of replacing it.
Collateral, not intent: touching roles/sys-ctl-hlth-csp/files/script.py brought
it under the comment lint, which found five pre-existing narration comments.
Three deleted, `fn[:-5] # strip ".conf"` replaced by fn.removesuffix(".conf"),
and the root-user comment rewritten in the sanctioned Exception form. Its 33
tests stay green.
tests/lint/repository/test_no_hardcoded_container_name.py (05f425ea5) is the
same family for INFINITO_CONTAINER and could move into spot/ later; this change
leaves it where it is.
make test-lint: 326 passed. The rule found 18 sites before the cleanup and none
after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cli build include` generated one tasks/groups/<group>-roles.yml per invokable category during setup, and the stage plays included those files. They were written by the in-container UID, which is why a dedicated clean-container-owned helper had to delete them from inside the container.
The stage plays now loop over lookup('group_roles', <group>) and include tasks/utils/setup/{group,role}.yml directly. ordered_roles() in utils/roles/order.py holds the run_after topological sort the generator used to perform at build time.
utils/roles/stage.py:_group_names() no longer globs the generated files; it reads get_invokable_paths(), the same categories.yml source those files were derived from.
Measured: handlers still flush after every role (ansible-core 2.20.2, three loop items, three separate handler runs), and resolving all 18 groups costs 0.60s cold against 1.4s for the generator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
git worktree remove deletes the checkout first and the metadata dir
second. Under the agent sandbox, which holds config.worktree and
commondir as read-only bind mounts, it therefore dies with EBUSY after
the checkout is already gone, aborting make worktree-down with rc 255
even though the branch is in fact free. down.sh now falls back to the
unregistration prune.sh already used (drop gitdir + HEAD, best-effort
rm -rf) and reports any dir the sandbox still pins. That routine moves
into lib.sh so both callers cannot drift apart, and gains a third
outcome for the case where gitdir/HEAD themselves survive: only then is
the branch still claimed, and only then must the teardown fail.
The discriminator for the fallback is whether the checkout survived,
not a match on "Device or resource busy" — the identical half-removed
state arises from EACCES, EPERM and EROFS.
A failing git status probe no longer reads as clean: on a broken .git
link it exits 128 with empty output, which used to satisfy the guard,
tear down the compose stack, and only then die at git worktree remove
with an unexplained 128.
Measured against the unfixed scripts with real fixtures: pinned
metadata dir 255 -> 0, broken .git link 128 -> 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
git worktree remove deletes the checkout first and the metadata dir
second. Under the agent sandbox, which holds config.worktree and
commondir as read-only bind mounts, it therefore dies with EBUSY after
the checkout is already gone, aborting make worktree-down with rc 255
even though the branch is in fact free. down.sh now falls back to the
unregistration prune.sh already used (drop gitdir + HEAD, best-effort
rm -rf) and reports any dir the sandbox still pins. That routine moves
into lib.sh so both callers cannot drift apart, and gains a third
outcome for the case where gitdir/HEAD themselves survive: only then is
the branch still claimed, and only then must the teardown fail.
The discriminator for the fallback is whether the checkout survived,
not a match on "Device or resource busy" — the identical half-removed
state arises from EACCES, EPERM and EROFS.
A failing git status probe no longer reads as clean: on a broken .git
link it exits 128 with empty output, which used to satisfy the guard,
tear down the compose stack, and only then die at git worktree remove
with an unexplained 128.
Measured against the unfixed scripts with real fixtures: pinned
metadata dir 255 -> 0, broken .git link 128 -> 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, none resolved by picking a side wholesale:
- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
rc now counts as failure; the previous default(0) let an aborted Playwright
run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
which this branch's lint enforces, and adopted the incoming retry-inside-poll
for the OIDC button plus its networkidle wait.
It shipped with the persistent-base change; the constraint it described - git forbids a branch feature beside feature/x, so the nested directories cannot collide - belongs in that commit's message, where it already is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The default base was /tmp, which is tmpfs on this host: a reboot took a whole worktree and its uncommitted work with it. Derive the base from the origin remote instead, as ~/.local/share/worktrees/<domain>/<account>/<repo>, and let the branch keep its slashes so it nests, which also makes the infinito-wt- prefix redundant. Without an origin remote it now fails loudly rather than inventing a location.
up.sh bumped only INFINITO_SWARM_LAB_SUBNET per slot while the five swarm lab node IPs stayed at default.env's 192.168.244.x, so every worktree from slot 1 on got a lab network in a different /24 than the nodes it places in it, and collided with the primary checkout's lab. Pin all six values from one octet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
841a5e0e1 claimed to keep entries whose checkout cannot be inspected, but the check resolved dirname(dirname(gitdir)) -- the checkout's PARENT, e.g. /tmp -- which always exists. The branch was dead code and every entry fell through to `rm -f gitdir HEAD`, including a live worktree whose metadata git can no longer read: that path drops it from the liveness list and it was then unregistered, which is the reported symptom.
Make the checkout directory itself the criterion, and refuse to conclude anything when its parent is not traversable -- `test -d` succeeds on a directory without +x, so a first attempt using -d alone reproduced the same blind spot. Reproduced in throwaway repos: a live worktree with a 0-byte commondir keeps its registration, a deleted checkout is pruned, and a metadata corpse without gitdir is cleaned.
Remaining limit: `git worktree prune`, which the script runs first, shares the blind spot and wins where it can unlink; this guard therefore only decides in the sandbox-pinned case that produced the bug.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Staleness was decided by reading gitdir inside the metadata dir. Under the agent sandbox that dir is pinned as read-only bind mounts, so a live worktree can read as having no gitdir at all -- prune then removed gitdir and HEAD and left the checkout as 'fatal: not a git repository' with no way back. Decide liveness from git's own worktree list plus each checkout's .git pointer instead, keep entries whose checkout base is not visible rather than guessing, drop the expected EBUSY noise, and report the pinned leftovers once with the command that clears them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
841a5e0e1 claimed to keep entries whose checkout cannot be inspected, but the check resolved dirname(dirname(gitdir)) -- the checkout's PARENT, e.g. /tmp -- which always exists. The branch was dead code and every entry fell through to `rm -f gitdir HEAD`, including a live worktree whose metadata git can no longer read: that path drops it from the liveness list and it was then unregistered, which is the reported symptom.
Make the checkout directory itself the criterion, and refuse to conclude anything when its parent is not traversable -- `test -d` succeeds on a directory without +x, so a first attempt using -d alone reproduced the same blind spot. Reproduced in throwaway repos: a live worktree with a 0-byte commondir keeps its registration, a deleted checkout is pruned, and a metadata corpse without gitdir is cleaned.
Remaining limit: `git worktree prune`, which the script runs first, shares the blind spot and wins where it can unlink; this guard therefore only decides in the sandbox-pinned case that produced the bug.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Staleness was decided by reading gitdir inside the metadata dir. Under the agent sandbox that dir is pinned as read-only bind mounts, so a live worktree can read as having no gitdir at all -- prune then removed gitdir and HEAD and left the checkout as 'fatal: not a git repository' with no way back. Decide liveness from git's own worktree list plus each checkout's .git pointer instead, keep entries whose checkout base is not visible rather than guessing, drop the expected EBUSY noise, and report the pinned leftovers once with the command that clears them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
git worktree prune removes the metadata directory as a whole, but the agent
sandbox pins config.worktree and commondir as read-only bind mounts, so the
unlink fails with EBUSY and the stale entry keeps claiming its branch.
Dropping gitdir and HEAD unregisters it without needing the pinned files gone.
git worktree prune removes the metadata directory as a whole, but the agent
sandbox pins config.worktree and commondir as read-only bind mounts, so the
unlink fails with EBUSY and the stale entry keeps claiming its branch.
Dropping gitdir and HEAD unregisters it without needing the pinned files gone.
The node container has no udev, so LOOP_CTL_GET_FREE hands out a minor whose
/dev node never appears; it also cannot load kernel modules, so those nodes
stay dead unless the host carries the loop driver. Materialize the nodes,
verify one can actually be attached, and expose the host prerequisite as a
bootstrap step instead of failing three hours in with a cryptsetup message.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parallel checkouts collided on the compose subnet, host ports and container
names, so a quality run in a worktree died at compose-up. Slot each checkout
via custom.env and layer two overrides on top: the shared cache network in
place of a second cache stack, and the shared .git directory without which
git aborts inside the container.
Both overlays gate on the resource they need, not on the slot number, so a
stray INFINITO_INSTANCE cannot make compose demand a value nobody set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the constructor's pre_deploy_snapshot.sh loop with the loader
itself: volume/nfs/secrets instant-start their oneshot in the preload
pass under MODE_BACKUP (force_flush_instant + state started), so the
live pre-update state is snapshotted before any app mutation; nfs keeps
run_after svc-storage-nfs-server, volume pulls sys-svc-container for a
live dockerd and pre-creates /opt/compose. remote pulls at end of play
via the terminator (force_flush_final) once every provider snapshot is
done. Dropping suppress_flush arms the result assert: a failed
pre-state backup aborts the deploy. Role READMEs carry the new trigger
schema as mermaid diagrams.
Known residual: on a host whose pull identity is not yet provisioned
the terminator-forced remote pull fails and turns the deploy red; the
swarm lab installs the trust only in the DR drill after round 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace scripts/system/diagnostics/rescue.sh with utils/diagnostics/
container.py: one workflow call on failure captures the current level
(containers with logs+inspect, swarm services, journal, host resources,
in-play role dumps) and recurses itself through every DiD boundary via
self-copy, from the runner down to the innermost runtime (RESCUE_MAX_DEPTH).
compose and swarm now share an identical failure step; the swarm per-node
collector wrapper is gone. All 17 script-only Ansible rescue: blocks are
deleted and their bare block: wrappers inlined (plus a wordpress leftover);
the three justified blocks (postgres pg_hba, xwiki extensions, keycloak
cross-node recovery) keep their evidence tasks and now end in a fail task.
Two lints enforce the state: rescue: is forbidden without a
'# nocheck: rescue-<reason>' first-task marker, and a block without
rescue/always/when or another scoping key must be inlined.
sys-service-terminator hoists become to the include apply. Also move
routine/08_cleanup.sh to utils/clean/teardown.sh and topology/ to
utils/topology/ so routine/ holds only the sequenced flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pre-deploy snapshot task only evaluates changed_when for executed loop
items, where the script module always sets stdout, so the default('') filter
hid nothing. pre_deploy_snapshot.sh is reindented by shfmt with no logic
change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.
The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.
Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container and NFS pre-deploy snapshots ran two near-identical role
scripts and two constructor tasks. Merge them into one shared script under
scripts/system/backup/ whose content check handles a file (databases.csv)
or a directory (export base), and drive both from a single looped
constructor task keyed by role, unit, and source. Each iteration keeps its
own group_names gate, its own timeout, and reports changed only when it did
not skip.
The shared script also reaches the fresh-host skip loudly: systemctl
list-unit-files exits non-zero when the version glob matches nothing, so the
listing now sits in an if instead of a bare command substitution that
aborted the snapshot under set -euo pipefail (the compose test-development
failure). The volume e2e test starts the unit through an if and dumps the
unit journal on failure so the real start error surfaces.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
container cp needs its source on the node running the docker CLI, so
role_path sources break on swarm workers. The joomla plugin dir now
ships as a controller-side tarball through container exec -i. A new
repository lint forbids docker/container cp in tracked .yml and .sh
files; the six legitimate same-node uses carry justified nochecks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New env handler exports the ca_trust_paths SPOT value (SOFTWARE_NAME read from group_vars) into .env; trust/linux.sh and trust/wsl2.sh consume the variable instead of carrying the path literal and fail loud when it is unset. Stale .env files need a regeneration (make dotenv) before the trust scripts run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a Makefile target plus helper script that removes container-owned
generated artefacts (build/install.stamp, tasks/groups/*-roles.yml)
which the host user cannot rm directly. The helper auto-starts a
stopped infinito compose container before deletion.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Path-style invocation in scripts/system/network/dns/common.sh failed
with `ImportError: attempted relative import with no known parent
package` because cli/meta/domains/__main__.py uses
`from . import PROJECT_ROOT`. The DNS host generator silently fell
back to a static 3-host list on every bootstrap run (visible in the
test-development CI logs).
Switch the caller to `PYTHONPATH=<repo> python3 -m cli.meta.domains`
so the package context resolves. Keeps the PROJECT_ROOT SPOT
(tests/lint/filesystem/python/test_project_root_import.py) intact —
no local redefinition.
Also: one-line YAML rewrap in compose/cache.override.yml (formatter).
Catches refactor drift across `.sh`, `Makefile` recursive calls, and
GitHub workflows. Includes fixes for the 5 bugs the new test surfaced
on first run: `make -C ROOT down`/`up` in the network stack refresh
helper (missed during the compose-* rename) and four
`make compose-deploymode=…` / `compose-deployapps=…` typos in
07/08/09_deploy_*.sh plus three docstring/echo references
(missing space after `compose-deploy`, introduced by the
deploy → compose-deploy sed-rename).
Four system scripts (wsl2-systemd-enable, wsl-dns-setup, tls-trust
linux/wsl2) sourced scripts/meta/env/{runtime,defaults}.sh — both
collapsed into load.sh by the SPOT refactor (80c396b8). Route them
through load.sh so INFINITO_IS_WSL2/INFINITO_CONTAINER resolve via
the generated .env.
Also rename the CoreDNS template placeholders to the INFINITO_*
namespace: \${IP4}/\${DOMAIN_RE} -> \${INFINITO_IP4}/\${INFINITO_DOMAIN_RE}.
env/ci.env carries the prefixed names since the env-prefix lint
(22cf6f38f), so envsubst was rendering empty answers — NXDOMAIN on
every CoreDNS query, breaking test-dns.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reason: ruff parses every `# noqa: <code>` marker as a flake8 directive
and warns on every non-flake8 rule key. Every catalog rule in
`docs/contributing/actions/testing/suppression.md` is project-specific
(kebab-case), so `# noqa:` markers fired the warning on every parse.
`# nocheck:` is the project synonym ruff ignores.
Scope:
- 41 `roles/*/meta/services.yml` + 1 task + 1 .cfg.j2 template: pure
noqa→nocheck swap for catalog rules. Mixed flake8 markers
(`# noqa: E402, X`) preserved verbatim.
- 5 production .py + 2 production scripts: same swap.
- All catalog-rule lints' docstrings updated; redundant "or # nocheck:"
parentheticals collapsed to a single nocheck form.
- `utils/annotations/suppress.py`: docstring rewritten to spell out
the new convention. Parser still accepts both keywords as synonyms.
- `docs/contributing/actions/testing/suppression.md`: convention
rewritten to mandate `nocheck:`; missing `domain-spot` and
`literal-protocol-lookup` rows added; "enforced by" pointer added.
New guard:
- `tests/lint/repository/test_noqa_only_ruff_codes.py`: meta-lint
failing any `# noqa: <code>` whose codes don't match `[A-Z]+\d+`
(real ruff/flake8 shape). Allowlists the impl, its unit test, the
catalog page, the direct-yaml lint, and itself.
- `tests/lint/repository/test_no_direct_yaml_calls.py`: hardened to
reject the `noqa:` form via inlined regex (no longer routed through
`utils.annotations.suppress`).
Tangential cleanups carried in this same commit:
- `utils/cache/yaml.py`, `utils/handler/yaml.py`: 6 `direct-yaml`
markers swapped.
- `utils/cache/base.py`: `from . import PROJECT_ROOT, ROLES_DIR` moved
up to the import block (E402).
- `cli/create/role/__init__.py`: reordered so PROJECT_ROOT is defined
before `__main__` is imported (also fixed a latent circular import).
- PROJECT_ROOT migration tail: 2 new `__init__.py` modules + 10 test
files swapped from `def repo_root()` to `from . import PROJECT_ROOT`.
- ~20 ruff auto-fix removals of unused `from pathlib import Path`.
Verification:
- `ruff check .` clean.
- 23 suppression-related tests pass.
- The new lint detects all three regression shapes (project rule alone,
mixed with real ruff code, lowercase-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Roles, playbooks, templates and scripts MUST consume the central
`NETWORK_PUBLIC_DNS_RESOLVERS` list from `group_vars/all/08_networks.yml`
instead of pasting `1.1.1.1` / `8.8.8.8` / `1.0.0.1` literally, so a
single edit propagates everywhere those resolvers are used.
- tests/lint/repository/test_no_hardcoded_dns_resolvers.py: scans the
repo for the literal IPs declared in NETWORK_PUBLIC_DNS_RESOLVERS,
fails on any occurrence not covered by `# noqa: hardcoded-dns-resolver`.
- roles/sys-svc-container/vars/main.yml: `SYS_SVC_DOCKER_AUTOMTU_TARGETS`
no longer hard-codes `1.1.1.1`; the list is now built from
`['registry-1.docker.io', 'github.com'] + NETWORK_PUBLIC_DNS_RESOLVERS`.
- roles/user/templates/bashrc.j2: `ip route get 8.8.8.8` ->
`ip route get {{ NETWORK_PUBLIC_DNS_RESOLVERS[0] }}`.
- group_vars/all/08_networks.yml: drop literal IP examples from the
resolver-block doc comment.
- compose/coredns/Corefile.tmpl, scripts/system/network/dns/setup/wsl.sh,
roles/sys-ctl-mtn-cert-deploy/SETUP.md: tag the legitimate literal
occurrences with `# noqa: hardcoded-dns-resolver`. These are
substitution points that cannot consume the variable (CoreDNS
forward directive, host-bootstrap shell fallback, Markdown example).
- docs/contributing/actions/testing/suppression.md: catalog the new
`hardcoded-dns-resolver` rule with position semantics and example.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings up the package-cache compose service alongside the existing
registry-cache, gated by the same `cache` profile. Bootstrap helper
(scripts/docker/cache/package.sh) auto-rotates the admin password from
the well-known default, accepts the Sonatype CE EULA (3.71+ blocks
artifact downloads with HTTP 403 until /v1/system/eula is POSTed —
disclaimer must be sent verbatim with its unicode quotes), enables
anonymous read, creates one default blobstore, and registers the MVP
proxy set: apt-debian, apt-ubuntu, pypi-proxy, npm-proxy, helm-bitnami,
raw-githubusercontent. The runner gets pip.conf / .npmrc /
sources.list snippets bind-mounted in (each defaulting to /dev/null
when the cache profile is inactive) so the deploy is byte-identical
with cache off.
Cache plumbing:
- scripts/meta/env/cache/{registry,package}.sh: per-cache env defaults,
hostname-derived admin password seed (uses `hostname` command, not
$HOSTNAME shell var, so the value is identical across interactive
shells, non-interactive subshells, and Python subprocesses).
- scripts/docker/cache/{registry-ca,package}.sh: container-side
bootstrap helpers, both moved under scripts/docker/cache/.
- scripts/system/cache/clean.sh + `make cache-clean`: wipes both
/var/cache/infinito/core/cache/{registry,package} dirs and stops
the cache containers.
- cli/deploy/development/compose.py: Profile gate exports the three
new INFINITO_PACKAGE_CACHE_* client-snippet paths and runs the
bootstrap helper after the package-cache reports healthy.
Compose service uses sonatype/nexus3:3.91.1-ubi pinned, runs as
root (UBI image refuses to chown a fresh bind-mount), and forces
NEXUS_SECURITY_RANDOMPASSWORD=false so the bootstrap can log in with
admin/admin123 before rotating; the random-password mode otherwise
auto-completes onboarding silently and leaves no readable secret on
disk for an automated bootstrap to pick up.
Tests: tests/unit/cli/deploy/development/test_compose.py grows two
new assertions (cache services listed in `up` args when local; build
tests pin CI=true to retain ci-only assertions).
Closes req-012.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
