Commit Graph
396 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 4085c2344b chore: clear repo-lint failures surfaced by make test
Reorder the clean-container-owned / clean-pycache-dirs / clean-stale-nfs Makefile targets into ascending order, set the executable bit on the swarm NFS cleanup scripts, and opt the CI-logs analyze and download-test file reads out of the cached-read lint (a runtime read of a user-supplied log, and a temp fixture written inside the same test).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:08 +02:00
kevinveenbirkenbach c7726c0e00 Prevent wedged swarm NFS cleanup 2026-06-26 17:01:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 76fac1fbfe fix(lint): filesystem-lint compliance for default.env, Makefile, clean_all.sh
Pre-existing test-lint failures surfaced while greening the suite:
- default.env: move the INFINITO_SWARM_* block into ASC order (it sat before
  INFINITO_LIMIT/LINT instead of after INFINITO_SUBNET) and trim the
  INFINITO_SWARM_TEST_LABEL comment under the 140-char cap.
- Makefile: order act-swarm-clean before act-swarm-down and fold the
  act-swarm-clean comment block to one description line plus Note: markers.
- scripts/tests/deploy/swarm/clean_all.sh: add the executable bit.
- tests/unit/plugins/filter/test_ruby.py: autoformat reflow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 18:31:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8547a800ed refactor(ci): unify deploy tests to compose+swarm, retire INFINITO_DEPLOY_TYPE
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.

Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.

Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bc1184dd10 refactor(complexity-cli): split modules, add base/siblings + --unique
Split the single complexity __main__.py into graph (discovery + BFS),
model (ComplexityRow + scoring), render (table/json/string) and cli
(argparse + main); __main__ stays a thin entry point.

Add a 'base' column (sha1 of the role's name unioned with its embedded
services, sorted) so roles in the same mutual-embedding cluster share a
key, and a 'siblings' column listing the rest of that cluster. --unique
keeps one role per base. roundtrip's default app list now passes
--unique to skip redundant same-cluster deploys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3a55877a39 refactor(swarm-test): SPOT topology naming in default.env + add act-swarm-clean
Hoist the swarm-test node, network and NFS-export-base names out of the shell
harness (00_topology.sh) into default.env (INFINITO_SWARM_*), so the shell
harness and the Python harness (extend_inventory, write_extras) read one source
instead of each hardcoding the suffixes. 00_topology.sh, extend_inventory.py,
write_extras.py and 01/09/11 now read the SPOT; passthrough.py exports the keys
into the generated .env. 00_topology.sh reads default.env directly (not .env)
because the workflow sources it before make dotenv runs.

Rename topology.sh -> 00_topology.sh (sorts ahead of 01_/02_, the order it is
sourced in) and update all 10 references.

Add scripts/tests/deploy/swarm/clean_all.sh + make act-swarm-clean to reclaim
wedged act-swarm state between runs, label-scoped via INFINITO_SWARM_TEST_LABEL
so it never touches unrelated containers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 18:31:34 +02:00
Alejandro Roman 187a954350 fix CI failures after rebase 2026-06-24 18:30:58 +02:00
Alejandro Roman 0153b5cac1 fix CI failures after rebase 2026-06-24 18:30:58 +02:00
Alejandro Roman 16f0f9d5a1 refactor(svc-runner): fix test issues after rebase 2026-06-24 18:30:41 +02:00
Alejandro Roman 764a1adf30 feat(svc-runners): add the logic to have x numbers of runners self hosted ad connected to GIthub, allowing to have 20 runners from gh + x self hosted 2026-06-24 18:30:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5dacc90f70 feat(test): default roundtrip apps to the complexity-ranked list
make roundtrip with no apps= now iterates every application, most-complex first (complexity --sort total --order desc --format string). Also marks roundtrip.sh executable and moves the roundtrip target into alphabetical order.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 20:46:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 669ef045fb feat(test): add roundtrip target to validate roles across deploy modes
make roundtrip apps="<app> ..." iterates each role through every deploy mode in order (compose, then swarm), stopping at the first failure. Per-step logs under TMPDIR; each validated swarm cluster is released unless keep=true. The mode sequence is overridable (modes="compose swarm") so kubernetes can be appended later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 19:42:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d2daa08677 feat(test): make SWARM_NAME mandatory for act-swarm cluster names
Every swarm-test cluster is named so parallel clusters get distinct container and network names. topology.sh and extend_inventory require SWARM_NAME; act-swarm-zombie defaults it to the app id (name= overrides), the CI workflow derives it from the matrix app, and act-swarm-down/shell/playwright require name=. The bare swarm-mgr-01 names are gone (node refs now carry the cluster prefix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 19:29:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 761450363b feat(make): add act-runner-image target for Docker-28/29 act runs
Recent Docker tightened docker cp tar extraction, so act aborts at job setup when the stock catthehacker runner image's /var/run symlink is unpacked into. Add a make act-runner-image target that builds local/act-runner-fixed (stock image with /var/run removed); pass it via ACT_PLATFORM_IMAGE. Document the symptom + fix in iteration/workflow.md and cross-reference from playwright.md and README.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 12:16:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 821ff4c386 feat(act-swarm): add act-swarm-playwright helper to rerun specs on the live cluster
Mirror compose-playwright for the act-swarm DinD cluster: copy the working-tree's modified files into the node's frozen bootstrap copy (the nodes are not bind-mounted), then rerun via rerun-spec.sh. rerun-spec.sh gains an opt-in TEST_E2E_PLAYWRIGHT_NETWORK_HOST that switches it to --network host (mirroring the deploy-time swarm branch in 02_run_one.yml) so cross-app canonical domains are reachable on the node; the compose path is unchanged. Document the all-green-before-redeploy gate in playwright.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4f69444bc0 fix(swarm-test): force host app-discovery runner in act-swarm-zombie
act auto-loads the repo .env (INFINITO_APP_DISCOVERY_RUNNER=docker), which overrode the swarm workflow env: host. The discover job then ran 'docker compose exec infinito' (not running in the act runner), emitted an empty matrix, and aborted the whole swarm run before any deploy. Pin host via ACT_ENV (act --env outranks the auto-loaded --env-file) so discovery uses the venv python. Local act harness only; real GitHub Actions does not auto-load .env.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:05:09 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9cee64430a feat(act-swarm-zombie): disable= plumbing via ACT_ENV semicolon
* Makefile: `make act-swarm-zombie app=<id> disable=<keys>` now
  forwards the `disable=` value into the act run via ACT_ENV. The
  swarm provision CLI inside the workflow reads it through
  `apply_services_disabled_from_env` and removes the named provider
  roles from the test inventory + sets their `enabled: false /
  shared: false` overrides in host_vars. Matches the iteration spec
  in docs/agents/action/iteration/role.md.
* scripts/tests/deploy/act/workflow.sh: switch the ACT_ENV pair
  separator from comma to semicolon so values can themselves carry
  commas (the disable= list is comma-separated). Only one in-tree
  caller (the Makefile target above) currently uses ACT_ENV with
  multiple pairs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 13:34:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d22f05353e feat(make/act-swarm): zombie target + topology SPOT
Adds `act-swarm-zombie app=<id>` which runs the swarm matrix-app test and leaves
the DinD cluster + lab network alive afterwards (INFINITO_KEEP_SWARM_NODES=true
forwarded via the new ACT_ENV passthrough in workflow.sh) so failures can be
post-mortem inspected with `make act-swarm-exec node=… cmd=…`,
`make act-swarm-shell node=…`, and finally released with `make act-swarm-down`.

Consolidates the swarm-test topology (manager / worker / nfs-server names, lab
network, NFS export base) into scripts/tests/deploy/swarm/topology.sh as the
single source of truth: 13_cleanup.sh and shell_node.sh source it directly,
the GitHub Actions workflow ingests it into $GITHUB_ENV via a setup step,
replacing the duplicate env: block entries.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbach 64a0c55b14 chore(makefile): sort act-debug between act-app and act-workflow
Test gate enforces alphabetical ordering of target definitions.
2026-06-14 19:08:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e293bd921 fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".

Fix:

- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
  full_cycle, variant) from `passthrough.STATIC_KEYS` and from
  `default.env`. They are per-invocation; .env is the persistent SPOT
  and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
  `os.environ.get("KEY", "")` so the missing-key path stops at "no
  filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
  (lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
  disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
  full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
  INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
  name in every doc; routing-knob env vars now use the same name so
  the cmdline form, the recipe env-prefix, and the consumer reads
  all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
  knobs whose alias differs from the env name (`type` →
  INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
  auto-exports cmdline variables into the recipe's shell env, so the
  identical-name `$(if $(apps),apps="$(apps)")` lines were pure
  noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
  `extra_env` per `docker exec` call and forwards `disable` via
  `-e`, so the container env never needs the value frozen at
  compose-up time.
- Update the 3 actual leak call sites
  (07/08/09_deploy_*.sh) to the make-alias form
  (`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
  the deliberate make/env parity, not a typo.

Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.

Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 17:47:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 25d709785c feat(make): add make requirements-archive target
Installs kpmx and runs `pkgmgr archive docs/requirements` in one
shot. The requirements-completeness lint failure message now points
contributors at this target instead of asking them to invoke pkgmgr
directly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 09:13:02 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ff84378bfa feat(make): add clean-container-owned target
Adds a Makefile target plus helper script that removes container-owned
generated artefacts (build/install.stamp, tasks/groups/*-roles.yml)
which the host user cannot rm directly. The helper auto-starts a
stopped infinito compose container before deletion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 490f0919ab refactor(make): replace INFINITO_CMD with cmd parameter
INFINITO_CMD is a per-invocation argument, not a config value — drop it
from default.env, the passthrough STATIC_KEYS, and the dotenv-generator
test, and rename to lowercase `cmd` in Makefile targets, exec scripts,
internal callers, and user-facing docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 05:31:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3b1e6bc9ef fix(deploy): default INFINITO_DEBUG=true via default.env + expose debug= on compose-deploy
The deleted `deploy-reuse-kept-apps` Makefile target hardcoded
`INFINITO_DEBUG=true` inline; the new `compose-deploy` recipe dropped
that without a replacement, so `apps/update/selection.sh` (strict
`${INFINITO_DEBUG:?}` guard) crashed in the test-development CI flow.

Make default.env the single source of truth: pin INFINITO_DEBUG=true so
BASH_ENV → load.sh propagates a valid value through every Make recipe,
and expose `debug=` on compose-deploy alongside the other short
aliases so callers can override per invocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7ad7b5e76c feat(lint): add lint-playwright for per-role spec parse checks
Each role's Playwright spec is staged the same way `test-e2e-playwright` does at deploy time (central config + personas tree + service-gating) and `npx playwright test --list` runs against it. `--list` skips the browser launch and any test body, so this catches refactor regressions like missing helpers, broken exports, or syntax errors in seconds instead of waiting for a full deploy. Stub env values keyed off the variable-name suffix satisfy the few specs that build URLs or call `.replace()` at module-load time.

Parallelism caps at `INFINITO_WORKER_CPU` (same env knob as `scripts/make/parallel.sh`); output is replayed in `SPEC_FILES` order after all workers drain so stdout stays deterministic. `@playwright/test` ships as a devDep via the existing eslint `npm ci`. Workflow wires into `ci-orchestrator.yml` behind the `lint-gate`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 10:24:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c36929090d refactor(make): apply comment-block schema across Makefile targets
Splits mixed-content target descriptions into a single description line
plus `Note:` follow-ups, and dresses the targets that carry real
parameters (`compose-deploy`, `compose-exec`, `compose-inner-run`) with
explicit `Usage:` / `Example:` / `Param <name>:` lines. Adds a
`variant=<idx>` Make var alias to `compose-deploy` so multi-variant
pinning works without dropping to `INFINITO_VARIANT=` env prefixes.

Removes redundant inline `compose-deploy` / `compose-inner-run` param
enumerations from the docs and replaces them with
`make help target=<name>` pointers, so the help banner stays the SPOT.

Also fixes SC2016 in `scripts/make/help.sh` (literal backticks in the
help banner) with a scoped suppression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 91fc422f19 feat(make): add compose-playwright target + schema-aware help
Adds `make compose-playwright role=<role> [pw="..."] [keep=true]` as the
canonical inner-loop entry point for rerunning a role-local Playwright
spec against the live stack — replaces the raw
`scripts/tests/e2e/rerun-spec.sh` invocation across docs.

Defines a small comment-block schema for Makefile targets (description
line on top, followed by `Usage:` / `Example:` / `Note:` / `Param <name>:`
markers). Lint enforces it; `make help` shows the description per target
in the short list and `make help target=<name>` prints the full block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 605f29a2f8 refactor(make): rename deploy to compose-deploy
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.

Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2e42ade0d2 refactor(make): rename container-* purges/refresh into the compose-* family
Move the three container-* targets into the compose- domain prefix to
match their actual scope (compose stack maintenance) and to make `make
help` group them with the other compose lifecycle commands:

- container-purge-entity       -> compose-entity-purge
- container-purge-system       -> compose-system-purge
- container-refresh-inventory  -> compose-inventory-refresh

Update the prereq chain (compose-system-purge -> compose-entity-purge),
the local-deploy testing doc table, and reorder the block so the
compose- family stays alphabetically sorted between compose-down and
compose-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 fb3d7c4e17 feat(make): default to help when invoked without a target
Set `.DEFAULT_GOAL := help` so bare `make` prints the target index
instead of falling through to the alphabetically-first recipe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 914926f78b refactor(make): sort targets alphabetically, satisfy ruff PERF401
Reorder every target block in the Makefile so the definitions land in
strict ASC order — the order broke during the cascading domain-prefix
renames (install-agent, security-apparmor-*, etc.) and now matches what
test_makefile_targets_sorted_ascending requires.

Rewrite the two for-loops in test_phony_declarations as list-comprehension /
list.extend forms so ruff PERF401 stops complaining and the file passes
autoformat.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 81ff9ede5a refactor(make): inline .PHONY declarations + lint guard
Place a single `.PHONY: <target>` line directly above each target's
comment block instead of clustering all phony declarations at the top
of the Makefile. The `help.sh` parser already resets its comment state
on lines beginning with `.`, so the PHONY entry must sit ABOVE the
comment (not between comment and target) to keep `make help` accurate.

Add the previously-missing PHONY entries for `dotenv` and `dotenv-force`
so every task target carries one.

Introduce `tests/lint/filesystem/makefile/test_phony_declarations.py`
to enforce two invariants going forward:
- every target has a matching `.PHONY: <target>` above its comment, and
- every `.PHONY: <target>` references an actually-defined target (no orphans).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 356e3366be feat(make): consolidate local deploy into single make deploy router
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.

Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.

Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.

Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 20:22:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0f276c856d refactor(scripts): restructure local deploy scripts into apps/ verb subtree
Split the local deploy scripts into a verb-based hierarchy under
scripts/tests/deploy/local/deploy/apps/{initialize,reinstall,update}/
plus scripts/tests/deploy/local/deploy/bundles/, with each verb folder
exposing all.sh (full discovery) and selection.sh (one or more apps via
INFINITO_APPS). The Makefile targets retain their existing names and now
dispatch to the new paths. The reinstall and bundles flows take over the
down + up cycle that used to be a Make prerequisite, and bundles/fresh.sh
now correctly delegates to apps/reinstall/selection.sh (the previous
delegate path was broken after the move). Each new directory carries a
README.md documenting scope, entry points, and when to use which verb.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:54:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f743f7b33f feat(console): add interactive make console REPL for the CLI
Wrap `python -m cli` in a host-side REPL launched via `make console`. The
loop normalizes input (strips leading `infinito`/`cli`, treats bare `help`
as `--help`), ignores Ctrl+C so accidental input cancels don't kill the
session, and exits only on `exit`/`quit`/`:q` or Ctrl+D. Includes a banner
with help/web/docs/license/author hints. Drops the now-redundant
`infinito --help` footer from `make help`. Adds 22 unit tests covering
normalization, subprocess dispatch, and main-loop control flow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 17:05:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 12a56e449d refactor(env): move env/default.env to repo-root default.env and drop env/
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.

Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
  default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
  switched from globbing env/*.env to targeting default.env directly;
  test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
  STATIC_ENV + every docstring/error message

Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:22:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f20278b4d5 refactor(env): drop env.development marker and its compose layering
The env.development marker file existed for two purposes that no longer
justify the wiring cost: a per-machine compose --env-file override layer
appended after auto-loaded .env, and a fallback signal for
detect_runtime() to return "dev" when the marker exists. With every
INFINITO_* default now collapsed into env/default.env -> .env,
persistent per-machine overrides are best expressed as
\`export INFINITO_FOO=…\` before invoking make (load.sh honors
caller-set values via setdefault). The marker fallback in
detect_runtime was redundant with the explicit RUNTIME=dev that every
dev-flow callsite already passes (fresh-*.sh, reset/inventory.sh).

Removed:
- Makefile mark-development target + .PHONY entry
- env.development from .gitignore / .dockerignore
- env_local conditional in compose.up()
- env_development conditional in down._compose_run()
- env.development conditional in scripts/meta/resolve/apps.sh and
  scripts/meta/resolve/diff/affected_roles.sh
- env.development marker branch + the now-unused project_root parameter
  from cli/meta/runtime/detect_runtime; __main__.py caller follows
- "Optional Development Marker" section in docs/contributing/environment/
  setup.md and the marker reference in docs/administration/installation.md

Also drops two RUNTIME=='dev' guards in roles/web-app-bigbluebutton that
relied on the auto-detect path:
- the compose build etherpad task in tasks/02_compose.yml
- the freeswitch iptables-REJECT entrypoint override in
  templates/compose.override.yml.j2

With auto-detection of "dev" gone the guards would otherwise only fire
on explicit RUNTIME=dev runs, silently regressing the local-dev path
operators rely on; both behaviors now apply unconditionally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:22:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e4e861c261 refactor(cli): move ports/networks suggesters under cli/contributing/network and add diagnose subcommand
Relocates `cli/meta/{ports,networks/suggest}` to
`cli/contributing/network/{ports,address/suggest}` so port and subnet
allocators live alongside the rest of the contributor-side tooling
(mirror, update, changelog) instead of `meta`. Mirrors the tests
into `tests/unit/cli/contributing/network/` and renames the contract
docs to `docs/contributing/tools/network/{port,address}.md`. Prog
names update from `cli meta ports suggest` / `cli meta networks
suggest` to `cli contributing network ports suggest` / `cli
contributing network address suggest`; the corresponding lint
references and `utils.meta.scan` docstring follow.

Adds a third subcommand: `cli contributing network diagnose`
(stdlib-only DNS / TCP / TLS / PMTU probe against the standard
infrastructure hosts plus interface MTU, `/etc/resolv.conf`,
`/etc/hosts`, proxy env, CA-bundle summaries). Split across `cli.py`
(orchestration), `config.py` (SPOT for hosts / per-distro install
commands), `format.py` (output + subprocess helpers), `tools.py`
(distro-aware auto-install of `iputils-ping` + `iproute2`),
`report.py` (system-info dumps + IPv6-default-route detection), and
`probes.py` (per-host probes). Exposed as `make debug-network`.

Companion `docs/contributing/actions/debugging/network.md` absorbs
the diagnose-tool walkthrough (incl. how to read MTU-induced TLS
handshake timeouts) so the merged page covers DNS / MTU / IPv6 /
connectivity end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c6291320ce feat(cleanup): add make clean-pycache-only-dirs for orphan __pycache__ parents
After moving or deleting a Python module, the surrounding directory
often ends up holding nothing but a stale `__pycache__/`. `git clean`
does not remove the parent because the parent is still tracked, so
these orphans stay around indefinitely (or until ansible-lint /
test-init checks start complaining).

Adds `utils.cleanup.pycache_only_dirs` (walked post-order so emptying
a child can collapse a parent on the next pass through the sorted
list), skips `.git`, `.venv`, `venv`, `node_modules`, and tolerates
permission errors from container-root-owned bytecode. Exposed as
`make clean-pycache-only-dirs`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2710971d9a fix(makefile): make lint-ansible depend on setup
The role-include files under `tasks/groups/*-roles.yml` are generated
by `make setup` and tracked as `.gitignore`d artefacts. CI checks out
the repo and runs `make lint-ansible` before `make setup`, so
ansible-lint statically resolves the literal include in
`tasks/stages/02_workstation.yml:7` against a missing
`tasks/groups/desk-roles.yml` and surfaces an un-ignored
`load-failure[filenotfounderror]` warning that, under the `min`
profile, exits with status 2.

Making `lint-ansible` depend on `setup` materialises the include
files before the lint walks them, so the literal path resolves
normally and no per-file ignore entry is needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5cbf9ccb3a feat(make): add make help listing targets via comment-derived descriptions
Adds scripts/make/help.sh which parses the Makefile, joins comment
lines preceding each target as its description, and tails the listing
with a pointer to `infinito --help` (or `python -m cli --help` when
the console script is absent) plus a link to https://docs.infinito.nexus.
The CLI's global help now references `make help` and the docs site.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 01:25:51 +02:00
kevinveenbirkenbachandClaude Opus 4.7 22cf6f38f1 refactor(env+make): rename static.env→default.env + sort + lint gates
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.

Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.

Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.

Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py        (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py   (# nocheck: markers exempt)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8a1bb42485 feat(make): test-signed pre-push gate + race-free dockerignore
`make test-signed` checks `git log -1 --pretty=%G?`; `N` (no
signature attached) aborts and the pre-push hook gates pushes
of unsigned tips. Sign locally via `git-sign-push` (already a
dev dep) or `git commit -S`.

Move the `.git` entry from a Makefile `echo … >> .dockerignore`
appendage into `.gitignore` itself (no-op for Git, required for
Docker). `make dockerignore` collapses to a single `cat .gitignore
> .dockerignore` which converges on the same content under any
parallel interleaving — previously 5 concurrent `make setup`
invocations from `make test` could append 3-5 stacked `.git`
lines and make pre-commit revert on "files modified by hook".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19f9735909 feat(test/external): bound runtime + share parallel update pipeline
The URL-reachability probe now bounds its wall-clock end-to-end:
allow_redirects=False (a redirect chain previously stacked
timeouts), a 5h30m global deadline that marks in-flight probes
as warnings on expiry, and live `[done/total]` progress every 50
probes. Submission order is shuffled so same-host URLs no longer
batch at the front and self-DoS one origin.

The Docker image-version test drops its bespoke sequential
fetch loop and calls find_outdated_updates() from utils.update.docker,
so it now shares the INFINITO_WORKER_FETCH-sized worker pool with
the repository-refs check.

`scripts/tests/code/wrapper.sh` propagates BASH_ENV into the
container exec — without it INFINITO_WORKER_FETCH was unset
inside the container and the URL probe collapsed to 1 worker.
`make test` gains test-external; `make dotenv-force` regenerates
.env with a stripped environment so stale BASH_ENV-preloaded
INFINITO_* values cannot pin old numbers via setdefault.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 38f3e2fc5a feat(make): parallel test/lint via scripts/make/parallel.sh wrapper
`make test` and `make lint` now delegate to a generic parallel-make
wrapper that caps `-j` at INFINITO_WORKER_CPU, syncs per-target
output, and prints wall-clock elapsed time. `make test` gains the
full `lint` sweep and drops the phantom `test-deploy` target (PHONY
without recipe); pre-commit hooks consolidate around a single
`make test` invocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 770531395d chore(lint): drop galaxy-importer lint
galaxy-importer's legacy-role mode was dropped upstream in 0.5.0
(CLI no longer accepts --legacy-role / --namespace), and the project
does not publish roles to Ansible Galaxy. The check enforces a
publishing schema that never gets exercised, against an abandoned API
that can only be kept alive via pinning galaxy-importer==0.4.39 +
setuptools<81 in an isolated venv. The cost has stopped buying
anything.

Removed:
- scripts/lint/galaxy.sh
- utils/install/lint/galaxy_importer.py
- install_isolated_pip_pkg() in utils/install/pip.py (only galaxy_importer
  used it)
- Makefile lint-galaxy target + the entry in the `lint:` aggregate
- pre-commit make-lint-galaxy hook
- The galaxy_importer ensure() call in _install_ansible_tools
- Test assertion against cli.galaxy_importer in test_main.py
- Documentation rows mentioning lint-galaxy / galaxy-importer

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 92f11a6701 feat(install/lint): Python-ize lint install + host/docker dispatcher
Refactor scripts/install/lint.sh (~870 LOC bash) into a Python package
under utils/install/. scripts/install/lint.sh becomes a 7-line wrapper
that delegates to `python3 -m utils.install.lint`.

Layout:
- utils/install/{primitives,pip,npm,github_release,system_pkg}.py:
  channel-agnostic installers
- utils/install/lint/<tool>.py: one ensure() per tool (actionlint,
  shfmt, mbake, ruff, ansible_lint, galaxy_importer, markdownlint_cli2,
  eslint, shellcheck, ansible_commands, ansible_collections)
- utils/install/lint/__main__.py: CLI + group dispatch + stamp logic

INFINITO_LINT_RUNNER dispatch:
- New env var (host | docker, default docker), tracked in env/static.env
  + passthrough handler + drift test.
- scripts/install/wrapper.sh routes `make install-lint` to host or
  container based on the runner.
- scripts/lint/wrapper.sh already dispatched; container branch now uses
  bash (not --login) so /etc/profile does not reset PATH, and forwards
  BASH_ENV so the .env keys land in the lint script's env.

Stamp logic:
- Per-interpreter stamp at build/install-lint-<sha256(sys.executable)>.stamp
  so host venv and container venv track installs independently despite
  bind-mounted build/.
- `--force` drops the stamp; group args bypass the stamp.

Tests + docs:
- 43 unit tests under tests/unit/utils/install/{,lint/} cover primitives,
  pip/npm/github_release/system_pkg fallback chains, and the main CLI
  (stamp logic, group dispatch, error path).
- utils/install/README.md + utils/install/lint/README.md document the
  structure and the per-tool ensure() contract.
- docs/contributing/tools/make.md install-lint row updated to mention
  the runner dispatch + per-env stamp + install-lint-force.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a578952c3 feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.

Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
  BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
  PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
  WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
  promoted IMAGE_TAG from GHA-only to general passthrough

compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions

Tooling:
- scripts/install/all.sh owns the stamp-file install logic
  (Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
  8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env

Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
  filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator

Docs:
- variables.md is now a pointer; per-variable docs live next to
  the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbach 162fffc337 Merge remote-tracking branch 'origin/main' into feature/web-app-kix 2026-05-19 18:17:13 +02:00