Replace the single `make -j ... --output-sync=target` invocation with
a shell-level fan-out that times each target individually and reports
a status table at the end. Output for each target is buffered to a
log file and dumped under a ✅/❌ header as soon as that target
finishes (preserving the per-target grouping `--output-sync` gave us).
Non-zero exit codes propagate; the run aborts at the first failure
once all already-launched children drain.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make test` and `make lint` now delegate to a generic parallel-make
wrapper that caps `-j` at INFINITO_WORKER_CPU, syncs per-target
output, and prints wall-clock elapsed time. `make test` gains the
full `lint` sweep and drops the phantom `test-deploy` target (PHONY
without recipe); pre-commit hooks consolidate around a single
`make test` invocation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename INFINITO_PARALLEL -> INFINITO_WORKER_ENABLED and add two derived
keys so consumers no longer multiply by their own factor: the env
generator resolves the cascade once at `make dotenv` time and bakes
the integer counts into `.env`.
Chain:
- INFINITO_WORKER_ENABLED (operator toggle, default true)
- INFINITO_WORKER_CPU = cpu_count() when ENABLED truthy, else 1
- INFINITO_WORKER_FETCH = CPU * 20 for I/O-bound bulk fetches,
collapsing to 1 when CPU == 1
Handlers:
- utils/env/handlers/infinito_worker.py (the prior single key)
removed
- utils/env/handlers/infinito_worker_cpu.py + infinito_worker_fetch.py
added; registry orders them after `passthrough` so the cascade
reads its dependency from the env-builder state.
Consumers (now 1-liners):
- tests/external/repository/test_urls_reachable.py reads
INFINITO_WORKER_FETCH directly.
- utils/update/base.py::resolve_max_fetch_workers() reads it directly;
utils/update/{docker,repository}.py keep using that helper plus a
ThreadPoolExecutor for the actual bulk fetch (with `dict(pool.map(...))`
satisfying PERF403).
- scripts/lint/autoformat.sh + docs/contributing/tools/make.md
rename INFINITO_PARALLEL -> INFINITO_WORKER_ENABLED.
Tests:
- EXPECTED_BASELINE_KEYS in test_dotenv_generator updated to track the
three new keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
galaxy-importer's legacy-role mode was dropped upstream in 0.5.0
(CLI no longer accepts --legacy-role / --namespace), and the project
does not publish roles to Ansible Galaxy. The check enforces a
publishing schema that never gets exercised, against an abandoned API
that can only be kept alive via pinning galaxy-importer==0.4.39 +
setuptools<81 in an isolated venv. The cost has stopped buying
anything.
Removed:
- scripts/lint/galaxy.sh
- utils/install/lint/galaxy_importer.py
- install_isolated_pip_pkg() in utils/install/pip.py (only galaxy_importer
used it)
- Makefile lint-galaxy target + the entry in the `lint:` aggregate
- pre-commit make-lint-galaxy hook
- The galaxy_importer ensure() call in _install_ansible_tools
- Test assertion against cli.galaxy_importer in test_main.py
- Documentation rows mentioning lint-galaxy / galaxy-importer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refactor scripts/install/lint.sh (~870 LOC bash) into a Python package
under utils/install/. scripts/install/lint.sh becomes a 7-line wrapper
that delegates to `python3 -m utils.install.lint`.
Layout:
- utils/install/{primitives,pip,npm,github_release,system_pkg}.py:
channel-agnostic installers
- utils/install/lint/<tool>.py: one ensure() per tool (actionlint,
shfmt, mbake, ruff, ansible_lint, galaxy_importer, markdownlint_cli2,
eslint, shellcheck, ansible_commands, ansible_collections)
- utils/install/lint/__main__.py: CLI + group dispatch + stamp logic
INFINITO_LINT_RUNNER dispatch:
- New env var (host | docker, default docker), tracked in env/static.env
+ passthrough handler + drift test.
- scripts/install/wrapper.sh routes `make install-lint` to host or
container based on the runner.
- scripts/lint/wrapper.sh already dispatched; container branch now uses
bash (not --login) so /etc/profile does not reset PATH, and forwards
BASH_ENV so the .env keys land in the lint script's env.
Stamp logic:
- Per-interpreter stamp at build/install-lint-<sha256(sys.executable)>.stamp
so host venv and container venv track installs independently despite
bind-mounted build/.
- `--force` drops the stamp; group args bypass the stamp.
Tests + docs:
- 43 unit tests under tests/unit/utils/install/{,lint/} cover primitives,
pip/npm/github_release/system_pkg fallback chains, and the main CLI
(stamp logic, group dispatch, error path).
- utils/install/README.md + utils/install/lint/README.md document the
structure and the per-tool ensure() contract.
- docs/contributing/tools/make.md install-lint row updated to mention
the runner dispatch + per-env stamp + install-lint-force.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`export VAR=$(cmd)` masks the inner command's exit code with export's own.
Splitting the assignment lets a failing `dirname "$PYTHON"` propagate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new steps wrapped around the existing push/create flow:
1. Dedupe before push: hash the local diff vs origin/<base-branch> and
compare against gh pr diff of every open PR (any author, any branch)
targeting the same base. On a match, exit 0 without pushing and
without creating a PR. This is what stops the same image-version
bump from being opened repeatedly.
2. Supersede after create: close every other open PR whose branch
starts with UPDATE_BRANCH_PREFIX with a "Superseded by #N." comment.
The prefix filter keeps the bot from ever closing repository-refs,
skills, or human-authored PRs, even though dedupe itself sees them.
CRLF normalization on both sides of the hash compare so a runner that
checks out the patch with native-CRLF translation still produces a
stable hash.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the shell helpers under scripts/github/ into five subfolders
that match how the workflow files consume them:
runner/ host-runner hygiene (disk, swap, Docker daemon, diagnostics)
cancel/ run cancellation on PR-close and branch-delete events
resolve/ derivation of workflow inputs and outputs from repo state
update/ scheduled Update: Versions pipeline
release/ release-time gates
Drop the now-redundant subfolder prefix from each filename, e.g.
dump_runner_state.sh becomes dump_state.sh and cancel_pull_request_runs.sh
becomes pull_request_runs.sh. Each subfolder carries a README that
describes its scope per documentation.md rules; the top-level README
points at the subfolders without inventorying files.
Propagate the new paths through ten workflow files, three docs
(workflow.md, pipeline.md, secrets.md), and two tests. Fix a real
runtime bug in enlarge_swap.sh, which still referenced the
pre-rename dump_runner_state.sh path. Extend the inline-block-size
lint hint to describe the new subfolder convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workflow-driven update PRs opened with GITHUB_TOKEN do not start
further workflow runs, so the CI orchestrator never executed against
them. Each PR-opening job in update.yml now mints a short-lived
GitHub App installation token via actions/create-github-app-token@v2
and forwards APP_SLUG to open_update_pr.sh, which derives the
verified-commit identity from the App user.
The job-level permissions block drops to read-only since write access
is provided exclusively by the App token. Behaviour with the secrets
unset is fail-loud: the token step fails, the PR step is skipped,
and no PR with an unintended identity is ever created.
Requires the BOT_APP_ID and BOT_APP_PRIVATE_KEY repository secrets
documented in docs/contributing/tools/github/actions/secrets.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror scope.sh's is_markdown_or_rst skip-list in affected_roles.sh so a
PR that touches both a role and a doc file outside roles/<role>/ resolves
to that role only, not __ALL__. Covered by tests/integration/meta/workflows/test_diff_affected_roles.py.
Refines the explicit HTTPS default_server introduced in e9df1ae7b:
instead of aborting the TLS handshake (curl→000), complete TLS with
the primary-domain cert and return 421 Misdirected Request (RFC 9110
§15.5.20). When the cert SAN still covers the requested host but no
vhost is configured for it — the exact failure mode that
sys-svc-cln-domains produces between matrix-deploy variants — clients
now see a precise, RFC-correct signal instead of an opaque TLS error.
For hostnames not in any cert SAN (production with per-domain LE
certs and unrelated probes), curl still rejects the cert and reports
000, so the response is naturally dual-mode:
- SAN match + no vhost → 421 (deploy-incomplete signal)
- no SAN match → 000 (truly unknown host)
Updates scripts/tests/environment/08_deploy_performance.sh: the
matomo-only fresh-purged deploy keeps dashboard.infinito.example in
the shared CI cert's SAN list, so the assertion changes from
`assert_http_status 000` to `assert_http_status 421`. The matomo
assertion in 07_deploy_minimal.sh stays at 000 because matomo is
excluded from the cert SAN in that deploy's CA regen.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- new utils.inventory.bundle_apps resolver aggregates and dedupes role
groups from inventories/bundles/{servers,workstations}/<name>/inventory.yml
- new make targets deploy-bundles (fresh-purged) and redeploy-bundles
(reuse-kept), driven by BUNDLES=... env, wrapped by
scripts/tests/deploy/local/deploy/{bundles,redeploy-bundles}.sh
- unit tests for the resolver under tests/unit/utils/inventory/
- document the new targets in docs/contributing/tools/make.md,
docs/contributing/actions/testing.md, inventories/bundles/README.md
- collapse multi-line Makefile comments to one-liners for consistency
- disable nextcloud bbb plugin auto-enable (commented out)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-app nginx gateways used proxy_pass with a literal docker container
hostname. nginx OSS resolves that hostname once at startup; if the
upstream container later restarts and gets a new IP (e.g. a partial
compose-up after credential change), the gateway proxies onto a dead
address until nginx itself reloads — the 502 we hit on taiga after a
reuse-kept replaced the taiga-back container.
Switch every per-app docker-hostname proxy_pass to a variable form so
the embedded docker DNS re-resolves at request time:
set $upstream_<rolename>_<purpose> http://<host>:<port>;
proxy_pass $upstream_<rolename>_<purpose>;
The full `resolver` directive (TTL + ipv6 toggle inherited from
NETWORK_IPV6_ENABLED) lives in one SPOT — NETWORK_DOCKER_DNS_NGINX_DIRECTIVE
in group_vars/all/08_networks.yml — so cache TTL and ipv6 stay
consistent across the stack.
In taiga the upstream `set` declarations hoist to server scope because
taiga-back is reused across /api/, /oidc/ and /admin/; in nextcloud
and kix each upstream is used exactly once so the declaration stays
local. Locations whose original proxy_pass stripped the location
prefix (`proxy_pass http://.../;` on /media/, /whiteboard/,
/standalone-signaling/) keep that semantics via an explicit
`rewrite ^<loc>(.*)$ /\$1 break;`, because variable-proxy_pass drops
the implicit prefix-strip.
kix's static files/kix-proxy.default.conf moves to
templates/kix-proxy.default.conf.j2 so it can reference the new
constant; the deploy task switches from copy to template accordingly.
Drive-by: scripts/tests/e2e/rerun-spec.sh docstring refresh — the
"only overwrites tests/playwright.spec.js" claim went stale when the
rerunner started restaging every role-local playwright/*.js.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`roles/__pycache__/` is created whenever Python imports the `roles`
package (it has an `__init__.py`), then `find` picks it up and
galaxy-importer fails the run with `FAIL __pycache__`. Same root cause
as the test_mapping.py fix in 635a0808d.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The early `/opt/compose/${entity}` check in apps.sh skipped redirect-only
entities (cdn, css, asset, file — no compose stack, only nginx vhost
files) before nginx.sh could run, leaving their per-domain vhosts
behind across matrix-deploy variant transitions.
Concretely chess CI Run 25997888664: variant-0 deployed web-svc-css
which rendered css.infinito.example.conf; the inter-round purge for
variant-1 skipped css with `\!\!\! WARNING: /opt/compose/css not found`;
the vhost survived; hlth-csp's headless browser then probed
https://css.infinito.example/ and got ERR_CERT_COMMON_NAME_INVALID
(the surviving vhost falls back to the default_server cert).
Moves the compose-dir guard from the entity-collection loop into the
purge loop and runs nginx.sh unconditionally before the
compose-dependent purges. The else branch prints the skip reason so
operators retain the trace signal for redirect-only entities.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Inner-loop iteration on a deployed stack often needs a one-off sidecar
container against the same docker daemon the stack lives on — typical
cases are rerunning a Playwright project with a patched .env, retrying
an arbitrary CLI image against an app container, or smoke-checking a
release image without going through Ansible. Without a wrapper the
caller has to spell out `docker exec <host-container> docker run --rm
<flags...> <image> <cmd>` by hand, which is verbose enough that agents
tend to fall back to redeploying instead of validating in-place.
Add `make run IMAGE=<ref> [CMD=...] [RUN_FLAGS=...]` plus the underlying
`scripts/tests/deploy/local/exec/run.sh`. The script reuses
`scripts/meta/env/all.sh` to resolve the host container name (same SPOT
as `make exec`), forwards RUN_FLAGS verbatim, and only wraps CMD in
`sh -lc` so default-entrypoint images and shell one-liners both work.
Wire both iteration loops to mention the wrapper next to `make exec`:
the Role Loop's Inspect block gets a bullet pointing at it, and the
Playwright Spec Loop gains a short "Debugging in the nested docker
daemon" section that also reminds the agent to mirror any successful
hand-patch back into the role's playwright.env.j2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the no-op `docker restart` recovery with `POST
/service/rest/v1/repositories/{name}/invalidate-cache` against each
apt-format proxy repo (Debian, Ubuntu, …) discovered dynamically
from Nexus' REST API. Container restart never re-validated cached
content; the new path makes Nexus actually re-fetch from upstream
on the next request and keeps repo definitions, blobstore, and
admin credentials intact.
Co-located changes:
- fresh-purged-app.sh: source scripts/meta/env/all.sh so the cache
override env (incl. INFINITO_PACKAGE_CACHE_FRONTEND_IP) is set
for the deploy CLI — without this, apt traffic bypasses Nexus.
- utils/nexus/list_apt_proxy_repos.py: stdin JSON → stdout name list,
exit 1 on malformed input / no matches; used by the bash helper
for discovery (parser lives outside the heredoc for testability).
- tests/unit/utils/nexus/test_list_apt_proxy_repos.py: 10 cases
covering filter rules, empty/invalid input, non-dict entries,
missing/empty names, and a realistic live payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 08 (`deploy-fresh-purged-apps … FULL_CYCLE=true`) iterates matomo's
two variants and skips the final inter-round purge, so the host ends on
variant 1 (mariadb.shared=false → embedded `database` service). The
previous pin to VARIANT=0 forced a variant flip (shared=true) on a host
still wired for variant 1, which left the freshly-rendered matomo
compose.yml referencing `database` without an image and tripped the
`compose config --quiet` validation handler with:
service "database" has neither an image nor a build context specified:
invalid compose project
Pinning the reuse pass to VARIANT=1 redeploys onto the same variant the
matrix last left running, which is the intended `reuse of existing
state` semantic from variants.md ("the last state remains available for
inspection or follow-up specs").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`docker compose down --remove-orphans` only deletes networks that
compose itself created (those carrying
`com.docker.compose.network=default` + `com.docker.compose.project=<name>`
labels). When the per-entity default network ends up unlabeled or
project-mislabeled — typical after an inter-round matrix-deploy
purge, or when a `community.docker.docker_network` Ansible call
materialised the network before compose ran — compose deliberately
leaves it untouched, and a subsequent `compose up` aborts with
network <entity> was found but has incorrect label
com.docker.compose.network set to "" (expected: "default")
The matrix-deploy CI rounds for web-app-matrix, web-app-gitea and
others reliably reproduce this.
Add a new entity-keyed purge primitive
`scripts/container/purge/entity/network.sh <ENTITY>` that runs after
the compose / dir / nginx primitives. It is idempotent: no-op when
the network does not exist, skips removal while active container
endpoints prove the network belongs to a shared service still in
use (svc-db-mariadb, svc-db-postgres, svc-db-openldap, svc-ai-ollama,
…), and otherwise drops the orphan network with `docker network rm`.
`docker network rm` failures are logged and tolerated — the next
`compose up` will surface the same problem with an actionable error.
Wire the primitive into `scripts/container/purge/apps.sh` immediately
after the existing `nginx.sh` call, so every entity-keyed purge
collects its own default network on the same pass.
Finally, after the entity loop, run a single bare `docker network
prune -f` to sweep orphan networks whose name does NOT correspond
to a purged entity (left over from renamed roles, abandoned
matrix-deploy rounds, etc.). The sweep is bare on purpose — no
`|| true` guard — so a daemon-side prune failure aborts the purge
loudly via the script's `set -euo pipefail`. Performance: ~13× faster
than calling prune inside `network.sh` per entity (one ~200 ms
listing pass vs. N × 200 ms for N entities).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`INFINITO_PACKAGE_CACHE_MAX_AGE_MIN` was 129600 (90 days), 13× longer
than the 7-day `Valid-Until` window Debian and Ubuntu generate by
default in their apt `Release` files. Nexus held cached `Release`
manifests for 90 days, so once the upstream `Valid-Until` elapsed the
manifest stayed in the cache for another ~83 days and every Docker
build that ran `apt-get update` against the package-cache aborted with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
Drop the default to 8640 minutes (6 days). Stays one day below the
7-day `Valid-Until` so Nexus always re-validates the cached `Release`
against upstream before apt would see it expired. The single value
still applies to every proxy repo (apt, yum, pypi, npm, gem, go, helm,
raw-*), but the `Valid-Until` enforcement only exists in apt — for
the other repos the change merely makes the cache 84 days fresher
instead of staler, no regression.
Update both docs sites that document the default:
- docs/contributing/artefact/files/compose.yml.md (env-var table)
- docs/contributing/environment/cache.md (operator narrative)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the registry-cache or package-cache (Nexus / nginx frontend)
serves apt repository metadata whose `Valid-Until` window has elapsed,
Docker builds inside an Ansible-driven deploy abort with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
which surfaces as a generic non-zero exit from the wrapped deploy
command and forces the operator to manually wipe the caches before the
next attempt.
Add `scripts/tests/deploy/local/utils/cache-retry.sh` exposing
`deploy_with_cache_retry <label> -- <cmd> [args...]`. The helper runs
the wrapped command with combined stdout+stderr teed to a temp log,
captures the wrapped command's real exit code via PIPESTATUS (with a
disciplined `set +e` / restore-previous-state dance so the caller's
`set -e` does not abort the script before the helper can react), and
on non-zero exit grep's the captured log for the stale-cache
signature `Release file ... is expired` / `Valid-Until ... expired`.
If matched, the helper wipes both cache stacks via `make cache-clean`
(registry-cache + package-cache through
scripts/system/cache/clean.sh), runs `docker builder prune -af` and
`docker image prune -af` to drop cached build layers and unused base
images, and re-runs the wrapped command exactly once. Non-stale-cache
failures bubble up unchanged so the caller's existing error handling
is preserved. Live in the deploy/update script layer rather than the
Ansible handler because only the host-side scripts can reach the
registry-cache + package-cache containers and the host Docker engine.
Wire the helper into all five local deploy entrypoints:
- fresh-purged-app.sh (both `--full-cycle` and the sync-only branch)
- fresh-kept-app.sh
- fresh-kept-all.sh
- reuse-kept-app.sh (added SCRIPT_DIR/REPO_ROOT locally because the
script did not yet have them)
- reuse-kept-all.sh
CI deploy paths (scripts/tests/deploy/ci/*) are intentionally left
alone — CI runs do not share the developer's local cache stack and
they handle stale metadata differently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
09_deploy_reuse.sh ran `make deploy-reuse-kept-apps APPS=web-app-matomo`
without VARIANT, which makes reuse-kept-app.sh fall back to the
unsuffixed `${INVENTORY_DIR}` path. The matrix-init in step 08 only
writes `${INVENTORY_DIR}-0` / `-1` because matomo declares 2 variants
(see roles/web-app-matomo/meta/variants.yml; planner contract:
cli/administration/deploy/development/inventory/planner.py:107), so
the reuse pass always missed the inventory and aborted with exit 2.
Also surface sibling inventory dirs as suggestions when the lookup
misses, so the next caller hitting this gets a hint instead of a bare
ENOENT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matrix-deploy variant transitions left per-domain vhost files behind:
when a service dropped out of a round's include set, sys-svc-compose's
docker prune removed its container but the vhost under
<NGINX_DIR>/conf.d/servers/{http,https}/<domain>.conf survived, making
the next round's HTTPS probe see 502 instead of TLS-error 000 (5 failed
test-development jobs on run 25705903504).
The inter-round purge in cli.administration.deploy.development.deploy
already invokes scripts/tests/deploy/local/purge/entity.sh with the
previous round's full include — the missing piece was an entity-keyed
nginx primitive next to db/compose/dir. Add it:
- utils.roles.entity_apps.apps_for_entity: reverse of get_entity_name,
walks roles/*/vars/main.yml to map entity -> [application_id]
- utils.cleanup.nginx_vhosts: resolves entity -> apps -> canonical+alias
domains (via build_applications_from_roles + iter_app_domains so it
stays ansible-free) and removes the matching vhost files
- scripts/container/purge/entity/nginx.sh: thin bash wrapper,
symmetric to db/compose/dir
- scripts/container/purge/apps.sh: drive nginx.sh inside the entity
loop, alongside db/compose/dir
Tests: 14 new unit tests covering the reverse-lookup edge cases and
the purge iterator (existing-only / aliases / multi-entity / no-op CLI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matrix-variant deploys write `${INVENTORY_DIR}-N/devices.yml` only —
the unsuffixed path doesn't exist, so the old `cat ${INVENTORY_FILE}`
killed every distro job in test-environment CI. Walk the suffixed
glob inside the container and fall back to the unsuffixed path.
Inner CMD runs under `sh -lc` (dash on Debian) so stay strictly
POSIX: no `shopt nullglob`, no arrays, no `(( ))`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops `MAX_LINES` from 12 to 11 in `test_inline_script_block_size.py`,
catching the five blocks that were sitting exactly on the previous
threshold. Pure-bash blocks move to dedicated `.sh` files invoked via
`ansible.builtin.script:` or `run: bash …`; the keycloak heredoc with a
templated JSON payload splits into a `set_fact` plus a folded
`printf | kcadm … -f -` one-liner.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make install-lint` failed on every test-environment matrix runner
(manjarolinux/base, debian:bookworm, ubuntu:latest, fedora:latest,
quay.io/centos/centos:latest) with:
npm not found; install Node.js/npm first to get markdownlint-cli2.
make: *** [Makefile:227: install-lint] Error 1
The two npm-driven installers (`install_markdownlint_with_npm` and
`ensure_eslint`) bailed out as soon as `npm` was missing instead of
following the same auto-install pattern used elsewhere in the
script (e.g. `ensure_command shellcheck` / `shfmt` / `ruff`).
* Add an `npm:<manager>` mapping to `install_command`. pacman /
apt-get / dnf / yum get the candidate list `npm nodejs`; brew
gets `node`. `install_package_candidates` returns on the first
candidate that installs, which covers every matrix distro
(Arch's `npm` pulls `nodejs` as a hard dep; Debian/Ubuntu's `npm`
Depends: nodejs; Fedora/EL9+ ship `npm` as a direct package; brew's
`node` bundles npm).
* `install_markdownlint_with_npm` and `ensure_eslint` now call
`ensure_command npm` when `npm` is missing, before failing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires JavaScript linting into the project for the first time, scoped to
the Playwright specs and persona helpers under `roles/**/files/**/*.js`.
Setup
-----
- `package.json` + `package-lock.json` pin eslint v9, eslint-plugin-
playwright v2, @eslint/js, globals.
- `eslint.config.js` (flat config) loads `eslint:recommended` and
`playwright/flat/recommended`. Browser globals are injected so
`page.evaluate(() => window.foo)` callbacks lint cleanly. Project-
idiomatic patterns are explicitly turned off with rationale: the
service-gating contract (req 006/019) uses test.skip on disabled
services; persona helpers branch on isVisible/isServiceEnabled before
expect; some specs deliberately use waitForTimeout / networkidle for
OIDC settle. `no-empty` allows the empty-catch idiom; the autofixer
for `prefer-web-first-assertions` and `no-wait-for-navigation` is
unsafe in this codebase (broken diff in dashboard.js), so both are
off pending a hand-migration pass.
- `scripts/lint/javascript.sh` mirrors the markdown.sh shape (skip on
missing, exit 127 with "make install-lint" hint).
- `scripts/lint/autoformat.sh` gets a `run_eslint` worker (treats rc≤1
as OK, like markdownlint).
- `scripts/install/lint.sh` gets `ensure_eslint` (npm-local install in
the repo root, matching the markdownlint pattern), a `javascript`
group, and `install_javascript_tools` is wired into the `all` group.
- `Makefile` exposes `lint-javascript` and folds it into the aggregate
`lint` target.
- `.github/workflows/lint-javascript.yml` follows the lint-makefile.yml
shape (Node 20, npm cache, `npm ci`, `make lint-javascript`); the
ci-orchestrator wires it into the lint pipeline + lint-gate `needs:`.
- `.gitignore` adds `node_modules/`.
Real lint findings fixed in this commit
---------------------------------------
- `roles/web-app-decidim/files/playwright.spec.js` was using
`runBiberFlow` / `runAdminFlow` without ever requiring `./personas`
(would have failed at runtime). Adds the require.
- Same file: two nutzlose `\\'csrf-token\\'` escapes in a double-quoted
query selector — eslint flagged, fixed.
- `roles/web-app-moodle/files/playwright.spec.js`:
`expect(await locator.count()).toBe(0)` →
`await expect(locator).toHaveCount(0)` (Playwright auto-retrying
assertion; safe `prefer-to-have-count` autofix).
Lint baseline: 0 errors, 285 warnings (mostly `no-unused-vars` and
`playwright/expect-expect` — addressable as follow-up cleanup, not
blocking).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shellcheck flagged three `[[ cond ]] && B || C` chains in
scripts/lint/autoformat.sh (ruff, shellcheck, mbake worker tails) as
SC2015: if `B` exits non-zero, `C` runs even when the test was true.
Rewrites each as `if [[ cond ]]; then B; else C; fi` so the success
and failure branches stay disjoint.
Reason: `make lint-shellcheck` aborted on these three info-level
findings; production-profile lint runs treat them as failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The reference at scripts/lint/makefile.sh:4 pointed at
github.com/mavyhq/bake which 404s; pypi metadata (mbake project) lists
the canonical homepage as github.com/EbodShojaei/bake.
Reason: tests/external/repository/test_urls_reachable.py was failing
with HTTP 404 on the stale URL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces roles/test-e2e-playwright/files/personas/ — runGuestFlow,
runBiberFlow, runAdminFlow plus 10 utils (csp, dashboard, env, keycloak,
landing, logout, matomo, prometheus, interaction, index) — and wires
both the Ansible runner (run_one.yml) and the local rerun helper
(scripts/tests/e2e/rerun-spec.sh) to stage the tree alongside
service-gating.js so specs can `require("./personas")`.
First batch of role specs migrated to consume the shared library
(dashboard, keycloak, mailu, matomo, prometheus, web-svc-cdn,
web-svc-simpleicons) — each gets the spec.js persona scenarios plus
the matching meta/services.yml + playwright.env.j2 entries; matomo is
marked role-closed in req 019. dashboard/keycloak/mailu TODO.md
updated; docs/requirements/019 reflects matomo closure.
Reason: req 019 Rule 3 mandates that every role's guest/biber/
administrator flow runs the same canonical steps; centralising them
in one shared library keeps per-role drift from re-introducing the
stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.
In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.
The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.
Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Package renames (dynamic discovery picks up new names automatically):
- cli/build/role_include/ -> cli/build/include/
- cli/build/roles_list/ -> cli/build/list/
- cli/meta/roles/lifecycle_filter/ -> cli/meta/roles/lifecycle/
Domain logic moved out of cli/ (no CLI surface, just a helper class):
- cli/meta/roles/applications/resolution/services/{resolver,errors}.py
-> utils/roles/applications/services/{resolver,errors}.py
- drop dead services/yaml_utils.py + its only test
Bug fixes:
- cli/meta/{callorder -> roles/callorder} dropped its hard-coded
parents[3] PROJECT_ROOT (broke after the directory move and resolved
to <repo>/cli/); it now reuses utils.PROJECT_ROOT.
Help renderer (cli/core/help.py + colors.py):
- DIM color on the description column to separate it from the command name
- Blank line between the long command name and its wrapped description
- Italic invocation under the category heading + leading blank line
- New Style.ITALIC
Doc updates: path references in docs/requirements/00{8,10}*.md plus
soft-wrap fixes in the touched regions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* .pre-commit-config.yaml: add 6 scoped lint hooks (lint-action,
lint-ansible, lint-python, lint-shellcheck, lint-markdown, lint-galaxy)
with `files:` patterns so a markdown-only commit no longer triggers
ansible-lint or galaxy-importer. Aggregate make-test, make-test-lint
(md-only branch) and pre-push make-test-external are kept.
* Makefile: drop `lint` from the `make test` dependency chain. Per-file-
type lint stages now live exclusively in pre-commit; running them again
from `make test` would just duplicate work.
* scripts/lint/autoformat.sh: add markdownlint-cli2 --fix and
ansible-lint --fix passes alongside ruff/shfmt/shellcheck.
* roles/svc-db-openldap/meta/main.yml: collapse the wrapped description
into a single line (still under Galaxy's 255-char limit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
ensure_* functions (pip / npm with sudo / --user / --break-system-packages
fallbacks). install_ansible_tools now also pulls ansible-lint and
galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
role, and treats `bare-name dependency` errors as success since our
roles/ tree is not a Galaxy collection layout. All other Galaxy schema
errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
`make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
now installs ansible-core + ansible-lint; ci-orchestrator wires both
new lint workflows into lint-gate.
Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
meta/main.yml plus _CANONICAL_COMPANY in
tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.
Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
renamed to "## User data resolution" (D3 self-label rule).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop four one-shot or now-redundant tools from cli/fix/:
* vars_main_files — bulk-created vars/main.yml with application_id
per role-prefix during the SPOT migration. The schema is now
enforced declaratively by tests/integration/roles/applications/id/
test_matches_role_name.py and the role-mapping SPOT lint.
* move_unnecessary_dependencies — moved meta dependencies to guarded
include_role on first sweep. Permanent enforcement lives in
tests/integration/roles/meta/dependencies/test_unnecessary.py.
* tabs — replaced literal tab characters with two-space indentation
in YAML files. Autoformat covers new occurrences.
* ini_py — generic helper that filled missing __init__.py files; new
packages now ship them from the start.
Move the remaining (still-actively-used) docker version updater out
from under cli/fix/ and into cli/contributing/, where the recurring
maintenance tooling lives:
* cli/fix/docker_image_versions → cli/contributing/docker/update_versions
Update the corresponding wiring:
* scripts/github/update_docker_image_versions.sh now calls
python -m cli.contributing.docker.update_versions
(the entry point invoked daily by .github/workflows/update.yml).
* docs/requirements/008-role-meta-layout.md references the new path.
* PROJECT_ROOT in __init__.py drops one parents[] level (the package
sits one folder deeper now).
cli/fix/ is fully removed; new contributor-facing tooling belongs
under cli/contributing/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move utils/roles/applications/files.py to utils/roles/mapping.py and add a
data-driven role-type detector in utils/roles/type.py. Each entry in
ROLE_FILES carries per-role-type allowed/mandatory flags plus marker entries
that drive get_role_types(); a role MAY belong to several types (e.g. an
application that also ships a system service).
Add cli/meta/roles/mapping with files / types / markers subcommands so
contributors can query the schema in text or JSON. Move
cli/meta/applications/ to cli/meta/roles/applications/ and adjust PROJECT_ROOT
parents indices accordingly. Move docs/contributing/design/services/ to
docs/contributing/design/role/services/ and document the mapping in
docs/contributing/design/role/mapping.md per documentation.md.
Replace the narrow test_app_only_files with a generalist
tests/integration/roles/meta/test_mapping.py that walks every role and every
ROLE_FILES entry and reports forbidden files, missing mandatory files,
forbidden entries, and missing mandatory entries in one structured fail
report. meta/info.yml and meta/users.yml are now allowed across every role
type. Create the seven mandatory files surfaced by the generalist test
(README.md for desk-keepassxc, desk-obs, web-svc-cdn following readme_md.md,
plus tasks/main.yml stubs for svc-db-memcached, svc-db-redis, sys-svc-proxy,
web-app-socialhome).
Audit and rewrite touched markdown for documentation.md compliance: replace
em-dashes with fluent sentences, drop SPOT self-labels, fix all broken
relative links introduced by the services/ move.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
