Commit Graph
12 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 71888d86af refactor(ci): prefix workflows by trigger
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.

Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.

Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5a223f407e refactor(ci): consolidate lint and test workflows into make-backed pairs
Replace the ten lint-*.yml workflows and the four test-code-*.yml
workflows with lint.yml (make lint fanning out every check, plus the
hadolint SARIF job) and test.yml (make test running unit, integration,
lint and external suites in parallel on the runner host). Drop the
lint-gate and test-code-gate jobs; code-quality-gate now needs lint,
test and security-codeql directly. Dependency diagram, workflows table
and pipeline docs follow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 19:53:20 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a578952c3 feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.

Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
  BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
  PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
  WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
  promoted IMAGE_TAG from GHA-only to general passthrough

compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions

Tooling:
- scripts/install/all.sh owns the stamp-file install logic
  (Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
  8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env

Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
  filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator

Docs:
- variables.md is now a pointer; per-variable docs live next to
  the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 98457f98fd refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.

utils/
  update/
    base.py        # semver primitives shared by both backends
    docker.py      # OCI registry lookups + version: line rewriter
    repository.py  # git ls-remote walker + ref: line rewriter

cli/contributing/update/
  docker/           # python -m cli.contributing.update.docker
  repository/       # python -m cli.contributing.update.repository

scripts/github/update/
  image_versions.sh
  repository_refs.sh

Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.

Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbach 837d0c5335 test(suppression): unify # noqa / # nocheck marker grammar
Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.

Helper
  utils/annotations/suppress.py exposes is_suppressed_at,
  is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
  suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
  (case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
  `<\!-- ... -->` comments, and combine multiple kebab-case rules
  comma-separated. 17 unit tests cover the grammar in isolation.

Refactored consumers
  - tests/external/repository/test_urls_reachable.py (rule: url)
  - tests/external/docker/test_image_versions.py via
    utils/docker/version_updater.py (rule: docker-version)
  - tests/lint/repository/test_no_direct_yaml_calls.py (rule:
    direct-yaml)
  - tests/lint/ansible/test_service_shared_consistency.py (rule:
    shared)
  - tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
    email)
  - tests/lint/repository/test_python_file_size.py (rule: file-size)
  - tests/lint/ansible/test_run_once_tags.py +
    tests/integration/roles/run_once/test_schema.py (rules: run-once,
    run-once-suffix)

Marker migration
  12 role tasks/main.yml files: `# run_once_<key>: deactivated`
  -> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
  drops a malformed inline `noqa: intentional legacy pin` from a TODO
  comment so the new parser does not pick it up as a stray rule.

Caching wins
  Test files that previously loaded YAML / read files via raw
  yaml.safe_load + Path.read_text now route through utils.cache.yaml
  (load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
    - test_service_shared_consistency.py
    - test_web_app_email_integration.py
    - tests/integration/roles/run_once/test_schema.py
    - utils/docker/version_updater.py
  This both speeds up repeated reads and stays consistent with the
  direct-yaml lint policy.

Documentation
  New SPOT page at docs/contributing/actions/testing/suppression.md
  with grammar, position semantics, the catalog of rules, examples,
  and an "adding a new rule" checklist. lint.md and external.md drop
  their per-rule tables and link to the new page. All touched docs
  conform to docs/contributing/documentation.md (no em-dash thought
  breaks, no SPOT self-labels, RFC-2119 keywords used consistently).

Settings sort tooling
  scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
  and is wired into scripts/lint/autoformat.sh. The tooling fixes the
  ASCII-ascending requirement enforced by
  tests/lint/repository/test_claude_settings_sorted.py; the test's
  failure message now points operators at the script. .claude/settings.json
  is committed in sorted form.
2026-04-30 11:54:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
Environment TestandClaude Opus 4.6 a2275e6991 docs(make): pass variables after target to match Bash(make*) permission
Reorders all documented and scripted `make` invocations from the
`VAR=value make target` shell-env prefix form to the make-native
`make target VAR=value` form. GNU Make auto-exports both command-line
and inherited shell-env variables to recipe shells, so the change is
purely about command-line shape: only the make-suffix form starts with
`make` and is therefore matched by the `Bash(make*)` permission entry
in .claude/settings.json, avoiding unnecessary approval prompts for
agent runs.

Adds an explicit CLAUDE.md rule with ✅/❌ examples and the underlying
reason (permission prefix match, not Makefile semantics).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 12:28:44 +02:00
Environment TestandClaude Sonnet 4.6 750c110f45 fix(docs): apply documentation rules, fix URLs, and remove orphaned references
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see  #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 00:38:42 +02:00
Environment TestandClaude Sonnet 4.6 2b90239282 test(external): fail on client/server errors instead of warning for dead links
Change the URL reachability test to fail on 4xx, non-500 5xx, and
request errors (DNS, connection). Only HTTP 500 now emits a warning
annotation, distinguishing temporary upstream breakage from dead links.
Add a unit test for the probe classification logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 23:23:43 +02:00
Environment Test 78a69130c9 test(external): add live url reachability scan 2026-04-10 23:01:25 +02:00
Environment Test 51341629d6 docs(external): clarify suite behavior 2026-04-10 22:42:23 +02:00
Environment Test 212cc39052 test(external): split live registry checks
Keep live registry lookups out of the default make test flow.

Add a dedicated external suite, workflow, and docs so the check stays opt-in but still gates code CI when enabled.
2026-04-10 22:31:42 +02:00