Commit Graph
298 Commits
Author SHA1 Message Date
Alejandro Roman dcb07255c0 fix: INFINITO_REPO_ROOT must be empty in default.env; hardcode container path in dedicated.sh 2026-06-24 18:31:03 +02:00
Alejandro Roman cdff6c1323 fix: remove inline INFINITO_* defaults from compose.yml and dedicated.sh 2026-06-24 18:31:02 +02:00
Alejandro Roman 0153b5cac1 fix CI failures after rebase 2026-06-24 18:30:58 +02:00
Alejandro Roman c038d147a3 fix runner issue in DinD + addeed docs for CLI runners 2026-06-24 18:30:46 +02:00
Alejandro Roman dd4fb1c5d8 test-e2e-cli structure 2026-06-24 18:30:46 +02:00
Alejandro Roman 3e06ee738f fix cleaner + revert changes ot of scope 2026-06-24 18:30:46 +02:00
Alejandro Roman 8fd23c6062 fix cleaner 2026-06-24 18:30:45 +02:00
Alejandro Roman 7dfba97555 add retries for self hosted slower servers 2026-06-24 18:30:44 +02:00
Alejandro Roman aff4a7d158 refactor(svc-runner): fix comments on PR 2026-06-24 18:30:43 +02:00
Alejandro Roman e6866faa08 refactor(svc-runner): modify test 2026-06-24 18:30:42 +02:00
Alejandro Roman 83975bd8d4 refactor(svc-runner): fix new test to pass CI 2026-06-24 18:30:41 +02:00
Alejandro Roman fecdb8cc0a refactor(svc-runner): fix test for runenr 2026-06-24 18:30:41 +02:00
Alejandro Roman 958409cb85 fix(svc-runners): fix prune not working 2026-06-24 18:30:40 +02:00
Alejandro Roman b42a493865 fix(svc-runners): volume pruen before deploy to aovid access denied 2026-06-24 18:30:40 +02:00
Alejandro Roman 1ea5345205 fix(svc-runners):add more time for slow specs server 2026-06-24 18:30:39 +02:00
Alejandro Roman f2998b8f8c fix(svc-runners):fix -T issue 2026-06-24 18:30:39 +02:00
Alejandro Roman c8bc1692dd fix(svc-runners):fix multiplier for sel hosted 2026-06-24 18:30:38 +02:00
Alejandro Roman d45d894feb fix(svc-runners): multiply retries for slower self hosted runners only 2026-06-24 18:30:38 +02:00
Alejandro Roman 5567a996c3 fix(svc-runners): cleaning unsued contaienrs, imprve CPU use and cache 2026-06-24 18:30:37 +02:00
Alejandro Roman a61e5a9ac0 fix(svc-runners): fix issues with tmpfs access for self hosted 2026-06-24 18:30:36 +02:00
Alejandro Roman 5698d9e708 fix(svc-runners): improve runner speed, use RAm isntead of disk 2026-06-24 18:30:35 +02:00
Alejandro Roman 8cab0f04da fix(svc-runners): isolate parallel runners (ports, inventory, docker volume, dns) 2026-06-24 18:30:34 +02:00
Alejandro Roman 7d28cbfbf3 feat(svc-runners): fix CI conflicts 2026-06-24 18:30:33 +02:00
Alejandro Roman 2bd4c07e0b feat(svc-runners): fix container finisher conflicts 2026-06-24 18:30:32 +02:00
Alejandro Roman bcf9e97fda feat(svc-runners): fix container conflicts 2026-06-24 18:30:32 +02:00
kevinveenbirkenbach 85c1ed1a6e Merge branch 'main' into feature/web-app-zammad
# Conflicts:
#	CHANGELOG.md
#	packaging/arch/PKGBUILD
#	packaging/debian/changelog
#	packaging/fedora/infinito-nexus.spec
#	pyproject.toml
#	roles/sys-svc-compose/README.md
#	scripts/docker/entry.sh
#	scripts/github/update/image_versions.sh
#	scripts/github/update/repository_refs.sh
2026-05-29 11:18:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e293bd921 fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".

Fix:

- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
  full_cycle, variant) from `passthrough.STATIC_KEYS` and from
  `default.env`. They are per-invocation; .env is the persistent SPOT
  and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
  `os.environ.get("KEY", "")` so the missing-key path stops at "no
  filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
  (lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
  disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
  full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
  INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
  name in every doc; routing-knob env vars now use the same name so
  the cmdline form, the recipe env-prefix, and the consumer reads
  all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
  knobs whose alias differs from the env name (`type` →
  INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
  auto-exports cmdline variables into the recipe's shell env, so the
  identical-name `$(if $(apps),apps="$(apps)")` lines were pure
  noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
  `extra_env` per `docker exec` call and forwards `disable` via
  `-e`, so the container env never needs the value frozen at
  compose-up time.
- Update the 3 actual leak call sites
  (07/08/09_deploy_*.sh) to the make-alias form
  (`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
  the deliberate make/env parity, not a typo.

Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.

Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 17:47:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8ab37347b1 feat(test-e2e-playwright): add host.docker.internal:host-gateway to sidecar runs
Gives Playwright sidecars a stable name for the docker host so role specs
can reach services whose port is bound on the host (e.g. Mailu's SMTP
submission on 465/587 in the Zammad email-channel test) without
attaching to that service's docker network.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:53:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 39ef28ea9e fix(scripts/tests/env/common): drop single-quoted printf that breaks compose-exec wrap
Inner `'==> %s\n'` collided with Make's `cmd='$(cmd)'` recipe wrap, mis-tokenizing the line and exiting 2 in every test-development variant.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 490f0919ab refactor(make): replace INFINITO_CMD with cmd parameter
INFINITO_CMD is a per-invocation argument, not a config value — drop it
from default.env, the passthrough STATIC_KEYS, and the dotenv-generator
test, and rename to lowercase `cmd` in Makefile targets, exec scripts,
internal callers, and user-facing docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 05:31:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c3ff093df0 refactor(scripts/deploy/local): mirror host wrappers with container/ subtree
Renames the three in-container deploy helpers out of `utils/` (where
they lived next to unrelated bash libs) into a `container/` subtree
that mirrors the host-side `apps/` layout one-to-one:

  utils/fresh-kept-all-init-and-deploy.sh  -> container/initialize/all.sh
  utils/reuse-kept-all-deploy.sh           -> container/update/all.sh
  utils/reuse-kept-app-deploy.sh           -> container/update/selection.sh

The `fresh-/reuse-/kept-` prefixes were overloaded ("kept" was
universal, "fresh/reuse" duplicated the verb already encoded in
initialize/update). New names use the same verbs + scope as the host
wrappers, so the host/container halves of each flow share a name.

Updates the three host wrappers under apps/{initialize,update}/ to
point at the new in-container paths and modernizes their
cache-retry keys (`fresh-kept-all` -> `initialize-all`, `reuse-all` ->
`update-all`, `reuse-<app>` -> `update-<app>`).

Adds READMEs for container/, container/initialize/, container/update/
following the project documentation conventions (H1 + trailing emoji,
RFC 2119 keywords, env-var tables, cross-links to the matching host
wrappers) and extends deploy/README.md with a pointer to the new
subtree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 54f88275ce feat(cli/console): stateful navigation, ls, and modular split
Reworks the REPL into a stateful shell that tracks a category path: typing a bare category name cd's into it, `/` returns to root, `..` walks one level up (chained as `../..` etc.), path tokens with slashes or a leading `..` navigate, `infinito [path]` always jumps absolute, and `ls` renders the current folder with a 🗂️/⚙️ emoji column, name, and a 30-char-truncated description (folders sorted before commands). Executable leaves (sub-packages with `__main__.py`) run instead of cd-ing into them. The prompt prints `🖥️  infinito <space-separated path>> `; the screen clears (incl. scrollback via `\033[3J`) before each command run, and the run echoes the resolved invocation in the fresh frame so the user sees what is about to execute.

Splits `repl.py` (~250 lines) into focused modules: `constants.py` (sentinels), `navigation.py` (path/category resolve + prompt + normalize), `ls.py` (folder rendering + truncate), `runner.py` (subprocess dispatch + clear_screen), `banner.py` (startup banner). `repl.py` is now a thin `main()` loop with backward-compat re-exports so external consumers keep the `repl.<symbol>` surface. Mirrors the split on the test side into `test_navigation.py`, `test_runner.py`, `test_ls.py`, and a `test_repl.py` that owns only main-loop integration with a shared `_MainLoopHarness` mixin.

Adds `scripts/tests/environment/11_console.sh` as a CI smoke that feeds a navigation sequence through the REPL and asserts every prompt shape plus the `ls` emoji output, so a regression in the prompt format, the path resolver, or the folder rendering is caught by the environment-test orchestrator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 13:06:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b5c3f33b56 fix(ci): surface in-container ansible log on the runner for role-runtime summary
The CI `📊 Top role runtimes` step parses `$ANSIBLE_LOG_PATH` on the
runner, but ansible-playbook actually runs inside the dev container via
`docker compose exec` — and the env var was never forwarded there, so
the log file never materialised. Even when set, the log would have
lived inside the container, invisible to the runner-side parser.

Forwards `ANSIBLE_LOG_PATH` through `compose.exec`'s `extra_env` so
ansible-playbook writes the log at the configured path inside the
container, then `docker cp`s it back to the runner from the cleanup
trap before the stack is torn down.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbach 5545cbe9c7 fix(test-deploy): run entry-bootstrap with --compile -- true
`entry.sh true` skips `run_local_build` because `true` is treated as
the exec target rather than a flag, so the in-container reinstall step
never ran. `--compile -- true` triggers the build first and then runs
`true` as a no-op exec target.
2026-05-21 22:58:28 +02:00
kevinveenbirkenbach 664457b739 test(lint): verify every make <target> reference resolves
Catches refactor drift across `.sh`, `Makefile` recursive calls, and
GitHub workflows. Includes fixes for the 5 bugs the new test surfaced
on first run: `make -C ROOT down`/`up` in the network stack refresh
helper (missed during the compose-* rename) and four
`make compose-deploymode=…` / `compose-deployapps=…` typos in
07/08/09_deploy_*.sh plus three docstring/echo references
(missing space after `compose-deploy`, introduced by the
deploy → compose-deploy sed-rename).
2026-05-21 22:58:27 +02:00
kevinveenbirkenbachandClaude Opus 4.7 605f29a2f8 refactor(make): rename deploy to compose-deploy
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.

Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 356e3366be feat(make): consolidate local deploy into single make deploy router
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.

Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.

Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.

Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 20:22:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0f276c856d refactor(scripts): restructure local deploy scripts into apps/ verb subtree
Split the local deploy scripts into a verb-based hierarchy under
scripts/tests/deploy/local/deploy/apps/{initialize,reinstall,update}/
plus scripts/tests/deploy/local/deploy/bundles/, with each verb folder
exposing all.sh (full discovery) and selection.sh (one or more apps via
INFINITO_APPS). The Makefile targets retain their existing names and now
dispatch to the new paths. The reinstall and bundles flows take over the
down + up cycle that used to be a Make prerequisite, and bundles/fresh.sh
now correctly delegates to apps/reinstall/selection.sh (the previous
delegate path was broken after the move). Each new directory carries a
README.md documenting scope, entry points, and when to use which verb.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:54:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e909d05b4a fix(env): collapse NIX_CONFIG to access-tokens line in the handler
\`utils/env/writer.py\` rejects multi-line values because bash \`source\`
and docker-compose \`--env-file\` cannot round-trip them
(introduced by a3782bf8d). \`scripts/tests/code/run.sh\` simultaneously
appended \`accept-flake-config = true\` to NIX_CONFIG before invoking
\`make setup\`, producing exactly the multi-line shape the writer
refuses. The test-environment workflow tripped this on every run
where \`06_test.sh\` → \`make test\` → \`scripts/tests/code/run.sh\`
fired (see actions/runs/26227822536/job/77180908867).

Fix the producer side, not the writer's layering:

* utils/env/handlers/nix_config.py: pull only the \`access-tokens\`
  line out of NIX_CONFIG and write that single line. Other Nix
  settings (\`accept-flake-config\`, \`experimental-features\`, …)
  already live in /etc/nix/nix.conf via the Dockerfile inside the
  dev container; host operators set them once in
  ~/.config/nix/nix.conf.

* scripts/tests/code/run.sh: drop the NIX_CONFIG mutation block.
  It was a no-op for in-container test runs (the container's
  /etc/nix/nix.conf already has accept-flake-config) and an active
  bug for the env-pipeline (multi-line value crashed make dotenv).

The writer keeps its multi-line guard unchanged — the contract is
that .env carries single-line values, and producers must respect it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:04:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a0620f7bb fix(inventory): drop provision-CLI args that shadowed group_vars defaults
Root cause of the DinD DNS-check failure on test-deploy-server
(networks.internet.ip4 falling through to 127.0.0.1 despite
INFINITO_IP4=172.30.0.10 reaching every link of the env pipeline):

  cli/administration/inventory/provision/cli.py declared --ip4 with a
  hardcoded default of "127.0.0.1" and ensure_host_vars_file wrote that
  value unconditionally into host_vars/<host>.yml. group_vars'
  defaults_networks.internet.ip4 resolves "172.30.0.10" via
  \`lookup('env', 'INFINITO_IP4')\`, but tasks/stages/01_constructor.yml
  merges \`defaults_networks | combine(networks | default({}, true))\` so
  host_vars wins. INFINITO_IP4 looked broken end-to-end; the breakage
  actually lived in the provision-CLI host_vars overlay.

  INFINITO_DNS_IP and INFINITO_DOMAIN propagated correctly because
  the provision CLI had no --dns and the --primary-domain write was
  None-gated and never invoked.

While here, --ssl-disabled and --primary-domain are removed too:
neither had any effect in the dev flow. provision/cli.py applies
\`--vars-file inventories/development/default.yml\` AFTER
ensure_host_vars_file via apply_vars_overrides_from_file, whose
_deep_update_commented_map unconditionally overwrites
(\`target[key] = value\`). The dev vars-file always set
\`TLS_ENABLED: true\`, so \`--ssl-disabled\` (which forced
\`TLS_ENABLED: false\` in host_vars) was systematically reverted to
true. Dev/CI ran with TLS enabled the whole time. Inventory stays at
\`TLS_ENABLED: true\` to preserve that effective behaviour.

Removed from cli/administration/inventory/provision/cli.py:
- --ip4, --ip6: defaulted to 127.0.0.1 / ::1, shadowed group_vars
- --primary-domain: never set by any caller, None-gated write
- --ssl-disabled: always re-overwritten by --vars-file later in the
  same provision pass

Removed from cli/administration/inventory/provision/host_vars.py the
matching writes in ensure_host_vars_file; the function now only sets
\`ansible_connection: local\` for localhost-class hosts. Defaults for
DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6} live in
group_vars (env-driven) and the inventory vars-file (deployment-wide
flags); host_vars stops duplicating them.

Callers cleaned up:
- cli/administration/deploy/development/inventory/builder.py
- cli/administration/deploy/container/command.py
- scripts/tests/deploy/local/utils/fresh-kept-all-init-and-deploy.sh
- scripts/tests/deploy/local/reset/inventory.sh

Tests + docs follow:
- tests/unit/cli/administration/inventory/provision/test_host_vars.py
  now asserts host_vars no longer carries baked DOMAIN_PRIMARY /
  TLS_ENABLED / networks defaults
- docs/administration/deploy.md drops --ssl-disabled from the
  provision example
- docs/contributing/design/inventory.md re-scopes the host_vars
  column: the DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6}
  values arrive via --vars / --vars-file overlays, otherwise group_vars
  provides the env-driven default

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 17:05:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 13c93ad6d5 refactor(compose): drop the ci profile and its --profile ci arg everywhere
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.

Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
  registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
  docker-compose command builder; down.py drops its now-unused
  Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
  meta/resolve/diff/affected_roles): plain \`docker compose\` instead
  of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
  reflect the missing --profile pair

Docs:
- docs/contributing/artefact/files/compose.yml.md: the
  \"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
  \`docker compose ... --profile ci config\` debug snippet drops the
  flag; while touching the file the two em-dash clause separators in
  the reproduction-profiles shell block are rewritten to colons
  (documentation.md em-dash rule), and one list-item in
  compose.yml.md is collapsed to one sentence per line for
  consistency with its siblings

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:22:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 12a56e449d refactor(env): move env/default.env to repo-root default.env and drop env/
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.

Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
  default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
  switched from globbing env/*.env to targeting default.env directly;
  test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
  STATIC_ENV + every docstring/error message

Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:22:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 07a9440ac7 refactor(env): fold ci.env into default.env so INFINITO_IP4/DNS/DOMAIN reach Ansible
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.

Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.

Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.

All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3fe215aaa6 fix(test-runner): source load.sh + prefer project venv over ambient VIRTUAL_ENV
`make test-unit` (docker runner) reached run.sh inside the container via
`bash --login`, which never sources the project's env-loader. PYTHON stayed
unset, so the test invocation fell back to the system python3 — which has
none of the runtime deps (ruamel, ansible, bcrypt, requests, certifi,
humanfriendly), producing 132+ ModuleNotFoundError errors.

run.sh now sources scripts/meta/env/load.sh itself (idempotent via
INFINITO_ENV_LOADED), so the env layer applies regardless of how the shell
was spawned.

While load.sh's detect.sh (now scripts/meta/env/python.sh) correctly walks
/opt/venvs and ~/.venvs to find the venv, an ambient VIRTUAL_ENV from a
different project (e.g. /root/.venvs/pkgmgr in pkgmgr's container) would
hijack the resolution. python.sh now scans the project-named venv candidates
first and only falls back to VIRTUAL_ENV when no project venv exists.

Also moves scripts/meta/python/detect.sh to scripts/meta/env/python.sh so the
env-related helpers live under one directory; updates the load.sh source
pragma and the lint-test path that scans it for INFINITO_* registrations.

`make test-unit` goes from 1262 tests / 132 errors to 1999 tests / 0 errors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a3782bf8d3 fix(env): break NIX_CONFIG bootstrap-loop
Two independent bugs were compounding into a corrupted NIX_CONFIG that
broke every local image rebuild:

1. scripts/tests/code/run.sh non-idempotently appended
   "accept-flake-config = true" to whatever NIX_CONFIG it inherited.
   On the next source of .env (which carried the previous value back),
   the line accumulated -- "true\naccept-flake-config = true\n..." --
   until Nix rejected the Boolean. Fix: pipe through `awk '\!seen[$0]++'`
   to dedup lines while preserving order.

2. utils/env/writer.py silently re-encoded literal newlines as `\n`
   escape sequences inside double-quoted values. docker-compose's
   --env-file loader does not decode them and neither does bash
   `source`, so any multi-line value round-tripped into garbage on the
   next read. Fix: raise ValueError instead -- producers must collapse
   to a single line before handing off to the env-builder.

Test fixtures for the writer adjusted to assert the new raise
semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 03:10:14 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a77a570409 refactor(env): consolidate INFINITO_* namespace and centralize path/image SPOTs
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:

* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
  env var → defaulted to infinito.localhost, which never resolved
  because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
  INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
  (env-builder, role default, run_one.yml docker exec, playwright.config.js,
  lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
  the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
  compose forwards it as build-arg, volume mount path, and container env;
  every Python CLI + shell helper + CI workflow now reads it strictly
  (os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
  /opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
  INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
  utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
  container) through compose.yml's environment block.

Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 02:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9d94f18129 fix(tests/environment): tolerate missing python3 in load_repo_env
00_orchestrator.sh sources common.sh in a vanilla container before
01_install.sh has run; the eager load_repo_env then sources load.sh ->
detect.sh which hard-fails per the new SPOT contract. Skip the load
silently when python3 is absent — the post-install loop iteration
re-runs load_repo_env and picks up the freshly installed interpreter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:56:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 80c396b839 refactor(env+install/spot): collapse python/venv resolution into shell SPOT
Pulls VENV/PYTHON/PIP/PYTHONPATH out of the dotenv handler pipeline
into scripts/meta/python/detect.sh. load.sh now sources detect.sh and
prepends ${VENV}/bin to PATH above the INFINITO_ENV_LOADED guard, so
every BASH_ENV re-entry re-resolves PYTHON the moment a venv
materialises mid-chain — the same fix as b79ed2869 but at the SPOT,
which retires the reload-trick from ansible.sh, python.sh and
pre-commit.sh.

Also renames INVENTORY_VARS_FILE → INFINITO_INVENTORY_VARS_FILE under
the new env-prefix lint, drops the 5 dead INFINITO_ENV_*_LOADED unsets
left over from the multi-loader era, and adds a lint guarding the
INFINITO_ prefix invariant on env/*.env keys.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:39:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 eb4bcd23a8 refactor(env/spot): forbid .get() defaults + cache namespace rename
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
  Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
  Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
  a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
  Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
  empty-string fallback (which silently masks an unset key). The
  nocheck check walks every physical line the call spans so an
  autoformat-split call still honours the marker.

Cache namespace flattens to a single prefix:
  INFINITO_PACKAGE_CACHE_*  -> INFINITO_CACHE_PACKAGE_*
  INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.

SPOT-strict rewrites:
  - 19 sites: `something.get("INFINITO_X", "")` -> subscript
    `something["INFINITO_X"]` (loud KeyError when env loader did
    not run; all keys live in env/default.env).
  - 2 handler-internal bootstrap reads suppressed with
    `# nocheck: handler-bootstrap-read`.
  - 30 runtime-only / driver-injected / test-fixture references
    annotated with `# nocheck: <reason>` per-line.
  - Unit tests that previously removed SPOT-guaranteed keys via
    `{k: v ... if k \!= KEY}` or `clear=True` switch to setting
    the key to "" explicitly, matching the new contract.

Side fixes:
  - INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
    shell-side defaults in common.sh to env/default.env and
    passthrough.STATIC_KEYS.
  - dns_systemd_is_operational() drops the dead
    INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
    variable was never set anywhere).
  - INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
    to `os.environ[KEY]`.
  - scripts/tests/code/wrapper.sh exec args move into a bash
    array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
    own element line (SC2215 fix).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00