Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.
Helper
utils/annotations/suppress.py exposes is_suppressed_at,
is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
(case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
`<\!-- ... -->` comments, and combine multiple kebab-case rules
comma-separated. 17 unit tests cover the grammar in isolation.
Refactored consumers
- tests/external/repository/test_urls_reachable.py (rule: url)
- tests/external/docker/test_image_versions.py via
utils/docker/version_updater.py (rule: docker-version)
- tests/lint/repository/test_no_direct_yaml_calls.py (rule:
direct-yaml)
- tests/lint/ansible/test_service_shared_consistency.py (rule:
shared)
- tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
email)
- tests/lint/repository/test_python_file_size.py (rule: file-size)
- tests/lint/ansible/test_run_once_tags.py +
tests/integration/roles/run_once/test_schema.py (rules: run-once,
run-once-suffix)
Marker migration
12 role tasks/main.yml files: `# run_once_<key>: deactivated`
-> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
drops a malformed inline `noqa: intentional legacy pin` from a TODO
comment so the new parser does not pick it up as a stray rule.
Caching wins
Test files that previously loaded YAML / read files via raw
yaml.safe_load + Path.read_text now route through utils.cache.yaml
(load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
- test_service_shared_consistency.py
- test_web_app_email_integration.py
- tests/integration/roles/run_once/test_schema.py
- utils/docker/version_updater.py
This both speeds up repeated reads and stays consistent with the
direct-yaml lint policy.
Documentation
New SPOT page at docs/contributing/actions/testing/suppression.md
with grammar, position semantics, the catalog of rules, examples,
and an "adding a new rule" checklist. lint.md and external.md drop
their per-rule tables and link to the new page. All touched docs
conform to docs/contributing/documentation.md (no em-dash thought
breaks, no SPOT self-labels, RFC-2119 keywords used consistently).
Settings sort tooling
scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
and is wired into scripts/lint/autoformat.sh. The tooling fixes the
ASCII-ascending requirement enforced by
tests/lint/repository/test_claude_settings_sorted.py; the test's
failure message now points operators at the script. .claude/settings.json
is committed in sorted form.
Two design docs under docs/contributing/design/:
- variants.md (renamed from matrix-deploy.md): explains the
folder-per-round wrapper that consumes per-role meta/variants.yml
and iterates variants at deploy time. Headings normalised to
sentence-case; the "legacy unsuffixed inventory path" wording
rewritten to drop the historical-context phrasing forbidden by
documentation.md.
- inventory.md (new): explains how a single round's inventory is
assembled out of meta/services.yml plus the variant payloads,
which inclusion mechanisms exist (operator --apps, run_after walk,
shared-service auto-include via strict `is True`), what lives in
the inventory file vs. host_vars/<host>.yml, and when Jinja
actually gets rendered (only at play runtime, via
utils.cache.applications._render_with_templar). Worked example
documents the WordPress + Discourse wiring as a reference case.
Cross-links updated wherever they pointed at matrix-deploy.md or
its concept name as a file:
- docs/agents/action/iteration/role.md
- docs/contributing/actions/deploy.md
- docs/contributing/artefact/files/plugins/lookup/applications.md
- docs/contributing/artefact/files/role/variants.md
- scripts/tests/deploy/local/deploy/reuse-kept-app.sh (path comment)
Plain-text occurrences of "matrix-deploy" as a concept term inside
Python comments, log messages, and test docstrings stay as-is; only
file-link references are migrated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.
Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
variant twice and ran the async pass against a host left in a
different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
as a runtime extra-var and required cross-stage coordination.
How:
- `roles/<role>/meta/variants.yml` (file format docs at
docs/contributing/artefact/files/role/variants.md): YAML list of
deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
variant 0 (= legacy payload). 6 roles get their dev overrides moved
here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
drops the retired `_active_variants` runtime selector. Loader always
returns variant 0 as the default; non-zero variants reach
`get_merged_applications` as inventory-level `applications.<app>`
overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
`plan_dev_inventory_matrix` (pure planner shared between init and
deploy), `build_dev_inventory_matrix` (materialises one folder per
round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
cleanup (only apps whose variant changed AND that are about to
re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
sync + async per round so the async re-deploy targets exactly the
host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
bash callers can inject per-run context into in-container helpers
without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
(Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
switch to a single init + single deploy `--full-cycle`. The
`make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
`INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
their in-container heredocs to dedicated helpers in
`scripts/tests/deploy/local/utils/`, removing the bash-in-bash
quoting hell.
Tests:
- `tests/unit/utils/test_variants.py` (renamed from
`test_application_variants.py`): loader edge cases, variant-zero
default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
per-round deploy filter, cleanup decisions, `--variant` pinning,
`--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
(new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.
Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
(folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
declaration" rule; `applications.md` renamed its SoT header for
consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expose INFINITO_MEM_LIMIT/INFINITO_MEMSWAP_LIMIT/INFINITO_CPUS on the
infinito service so operators can cap local runs to GitHub-hosted runner
specs and reproduce OOM failures like #162. Document the compose.yml as
a dev/CI-only artefact under docs/contributing/artefact/files/ and link
to it from the CI debugging guide.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Split docs/agents/action/iteration.md into role.md, playwright.md,
workflow.md, and a scope-focused README. Move Certificate Authority
and Inspect rules into role.md. Rewrite playwright.md as a machine-
oriented state machine with explicit preconditions, procedure, exit
gates, and escape path.
Extract Playwright spec content MUSTs into the new contrib SPOT
docs/contributing/artefact/files/role/playwright.specs.js.md. Reduce
the agent-side spec and env.j2 pages to procedure only. Align the
framework SPOT and the test-e2e-playwright role README with the
central templates/package.json.j2 + files/playwright.config.js model
and images.playwright.version as single source of truth.
Sweep em dashes and sentence-case role headings across all touched
docs per documentation.md and readme_md.md rules.
Reorders all documented and scripted `make` invocations from the
`VAR=value make target` shell-env prefix form to the make-native
`make target VAR=value` form. GNU Make auto-exports both command-line
and inherited shell-env variables to recipe shells, so the change is
purely about command-line shape: only the make-suffix form starts with
`make` and is therefore matched by the `Bash(make*)` permission entry
in .claude/settings.json, avoiding unnecessary approval prompts for
agent runs.
Adds an explicit CLAUDE.md rule with ✅/❌ examples and the underlying
reason (permission prefix match, not Makefile semantics).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Change the URL reachability test to fail on 4xx, non-500 5xx, and
request errors (DNS, connection). Only HTTP 500 now emits a warning
annotation, distinguishing temporary upstream breakage from dead links.
Add a unit test for the probe classification logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep live registry lookups out of the default make test flow.
Add a dedicated external suite, workflow, and docs so the check stays opt-in but still gates code CI when enabled.
Add tests/lint/repository/test_markdown_links.py — a hard-fail lint
test that scans all git-tracked markdown files for broken relative and
repo-root-absolute links. Skips external URLs, anchor-only links, fenced
code blocks, and placeholder targets.
Apply documentation.md rules to all staged files: add emojis after
headlines in docs files, remove emojis from role README headings,
standardize Credits sections to the canonical format, remove empty
sections and orphaned sentences referring to missing documentation files,
fix raw URLs and incorrect link text across role READMEs and docs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move emojis after headline text (CONTRIBUTING.md, actions/, design/, tools/, tests/)
- Replace em dashes with colons, semicolons, or complete sentences throughout
- Add missing emojis to all headlines in exceptions.md, principles.md, services.md, shell.md, setup.md, pull-request.md, commit.md, act.md, docker.md, pr/README.md
- Update H1 titles to reflect renamed directories (actions/, design/)
- Fix broken relative links in workflow.md after directory moves
- Fix full-path link texts in services.md (style.css.md, javascript.js.md)
- Fix broken principles.md link to lint.md after testing/ → actions/testing/ move
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
