Commit Graph
24 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 1079f48feb fix(env): derive the image and container names from the distro under test
A compose deploy job labelled centos or arch built its stack image from
pkgmgr-debian and ran it as infinito_nexus_debian (#794). distros.sh
exports INFINITO_DISTRO per iteration and sources load.sh again, but the
loader returned early because INFINITO_ENV_LOADED and the .env mtime were
unchanged. A full reload would not have helped either: the debian values
were already exported, so the restore step kept them as caller intent.

load.sh now records the distro it loaded for. Another INFINITO_DISTRO
defeats the early return and drops INFINITO_CONTAINER,
INFINITO_PARENT_IMAGE and INFINITO_IMAGE, the three keys the env handlers
derive from the distro. When the requested distro also differs from the
one .env was generated for, the loader sources a set that
`python -m cli.meta.env --output` generates for it in a private temp
directory and leaves .env untouched, so a local multi-distro run does not
switch the checkout's distro. A caller value set for the loaded distro
still wins.

Verified: tests/unit/python/scripts/meta/env/test_load.py and the new
distros.sh case pass. Against main's loader both fail with the symptom of
the issue: INFINITO_DISTRO=centos next to infinito_nexus_debian and
pkgmgr-debian:stable, and distros.sh handing debian, arch and centos the
same debian parent. shellcheck is clean.

Consequence: CI jobs labelled arch, ubuntu, fedora or centos now build
and deploy that distro, so a failure that shows up there was masked
before.

feature/store-i18n carries 3f973eb137, which resolves the parent image
for scripts/image/push.sh on its own, and 05e4dbafcb, which inserts the
INFINITO_INSTANCE slot check into the block this commit restructures. The
merge keeps the slot check ahead of the distro check.

Closes #794

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:30:48 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d0ae004eb0 feat(env): fork-identity vars + CI-redeploy aliases
INFINITO_FORK_REPOSITORY_NAME (default.env, static) plus two handlers: INFINITO_FORK_ACCOUNT defaults to the local OS username (getpass.getuser()), and INFINITO_FORK_REPOSITORY_URL composes https://github.com/<account>/<name>. Registered in ORDERED_HANDLERS with the account handler before the URL handler; documented in docs/contributing/environment/variables.md.

Two operator aliases build on it: i8cire (make quality + git-sign-push + retrigger the failed swarm CI jobs of a run) and i8puci (the same without the quality gate). Both take a run id or full URL: a bare id is expanded against INFINITO_FORK_REPOSITORY_URL, a URL is passed through, and a missing INFINITO_FORK_REPOSITORY_URL fails fast.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 12a56e449d refactor(env): move env/default.env to repo-root default.env and drop env/
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.

Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
  default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
  switched from globbing env/*.env to targeting default.env directly;
  test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
  STATIC_ENV + every docstring/error message

Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:22:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 07a9440ac7 refactor(env): fold ci.env into default.env so INFINITO_IP4/DNS/DOMAIN reach Ansible
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.

Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.

Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.

All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 14:55:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 eb4bcd23a8 refactor(env/spot): forbid .get() defaults + cache namespace rename
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
  Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
  Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
  a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
  Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
  empty-string fallback (which silently masks an unset key). The
  nocheck check walks every physical line the call spans so an
  autoformat-split call still honours the marker.

Cache namespace flattens to a single prefix:
  INFINITO_PACKAGE_CACHE_*  -> INFINITO_CACHE_PACKAGE_*
  INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.

SPOT-strict rewrites:
  - 19 sites: `something.get("INFINITO_X", "")` -> subscript
    `something["INFINITO_X"]` (loud KeyError when env loader did
    not run; all keys live in env/default.env).
  - 2 handler-internal bootstrap reads suppressed with
    `# nocheck: handler-bootstrap-read`.
  - 30 runtime-only / driver-injected / test-fixture references
    annotated with `# nocheck: <reason>` per-line.
  - Unit tests that previously removed SPOT-guaranteed keys via
    `{k: v ... if k \!= KEY}` or `clear=True` switch to setting
    the key to "" explicitly, matching the new contract.

Side fixes:
  - INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
    shell-side defaults in common.sh to env/default.env and
    passthrough.STATIC_KEYS.
  - dns_systemd_is_operational() drops the dead
    INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
    variable was never set anywhere).
  - INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
    to `os.environ[KEY]`.
  - scripts/tests/code/wrapper.sh exec args move into a bash
    array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
    own element line (SC2215 fix).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 22cf6f38f1 refactor(env+make): rename static.env→default.env + sort + lint gates
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.

Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.

Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.

Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py        (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py   (# nocheck: markers exempt)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a578952c3 feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.

Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
  BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
  PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
  WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
  promoted IMAGE_TAG from GHA-only to general passthrough

compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions

Tooling:
- scripts/install/all.sh owns the stamp-file install logic
  (Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
  8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env

Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
  filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator

Docs:
- variables.md is now a pointer; per-variable docs live next to
  the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a26bfc4e94 refactor(cli): reorg admin/contributing tree + per-folder help system
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):

- cli/deploy             -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault              -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror             -> cli/contributing/mirror

The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".

Help system redesign in cli/core/{discovery,help,app}.py:

- Discovery now distinguishes runnable commands (directories with
  __main__.py) from category folders (directories that only group
  further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
  as the title and its first paragraph as the one-line description in
  the help output.
- "infinito" / "infinito --help" prints only the top-level entries
  (one level deep). "infinito <category>" navigates into a folder and
  shows its sub-folders and commands one level deep, with the README
  title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
  prefix recursively. N caps the depth (1-based); omit N for an
  unbounded walk. The output uses Unicode tree connectors, an emoji
  per entry (folder for categories, gear for commands), a globally
  aligned description column, and multi-line wrapped descriptions
  with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
  ("infinito meta callorder") rather than the bare leaf name, so the
  output is copy-pasteable.

Convention doc + 17 CLI READMEs:

- New docs/contributing/artefact/files/cli/readme_md.md describes the
  H1+paragraph layout the help system reads, and how it differs from
  role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
  and their immediate sub-folders, plus cli/meta/roles/applications
  and its resolution sub-folder. Each follows the "title with trailing
  emoji + single descriptive paragraph" convention.

Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.

Fixes uncovered by the move:

- import unittest.mock added to four dedicated-deploy tests so
  unittest.mock.patch resolves under stdlib unittest discovery
  (Python 3.13 no longer surfaces unittest.mock through "import
  unittest" alone).
- parents[N] index in the new
  cli/administration/inventory/validate/__init__.py corrected from
  3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
  substring assertions updated to match the new validator path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:17:09 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a080dc4f98 refactor(cli): inventory tools under administration/, drop role scaffolder + dead SSO indirection
Move:
- cli/build/inventory/full      -> cli/administration/inventory/devices
- cli/create/inventory          -> cli/administration/inventory/provision
- cli/create/credentials        -> cli/administration/inventory/credentials
- cli/encrypt/inventory         -> cli/administration/inventory/encrypt

Delete:
- cli/create/role + templates/roles/ (stale scaffolder)
- roles/web-app-baserow/templates/javascript.js.j2 + dead sso_warning snippet
- js_application_name across akaunting/baserow/opencloud/opentalk/taiga,
  javascript service disabled for baserow + taiga

Wire-through:
- argv lists in cli/deploy/{container,development}/* + 2 shell scripts
  now invoke 'infinito administration inventory provision'
- subprocess module names + docs (contributing/* + administration/deploy.md)
- discovery resolver test + folder-item-limit whitelist updated
- ruff fixes in devices/command.py (RET504 inline + Path.open over open)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 18:41:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 eb45f0ffaf feat(cache): inner-build override + TTL env-var + SPOT docs
Extend the local cache stack with three independent capabilities:

1. Inner-`dockerd` build coverage for HTTP-only package mirrors.
   `roles/sys-svc-compose/files/compose.py` (the runner's `compose`
   wrapper) auto-generates a transient `compose.cache.override.yml`
   per app when `INFINITO_PACKAGE_CACHE_FRONTEND_IP` is exported. The
   override pins `build.extra_hosts` for `deb.debian.org`,
   `archive.ubuntu.com`, and `security.ubuntu.com` to the frontend.
   `pull.py` now delegates to the wrapper so build/pull operations see
   the same `-f` set the wrapper auto-detects. nginx in the frontend
   gains a port-80 listener with path rewrites for those hostnames,
   and `package.sh` registers two new Nexus proxies
   (`apt-debian-security`, `apt-ubuntu-security`) for the security
   suites whose upstream URL prefix differs from the main archive.

   Alpine is intentionally excluded from the inner-build extra_hosts:
   3.18+ defaults to HTTPS, and inner-`dockerd` build containers do
   not import the frontend CA. Routing apk through the frontend would
   trip TLS verification.

2. General cache freshness env-var. `scripts/meta/env/cache/package.sh`
   exports `INFINITO_PACKAGE_CACHE_MAX_AGE_MIN` (default `129600`,
   i.e. 90 days). The bootstrap helper consumes it for every Nexus
   proxy's `contentMaxAge`, `metadataMaxAge`, and
   `negativeCache.timeToLive`, replacing the previous 1440-minute
   literal. Lower the value when chasing fast-moving upstreams.

3. SPOT cache documentation at `docs/contributing/environment/cache.md`.
   `compose.yml.md`, `variables.md`, the per-service READMEs, and
   `requirements/012-...` now link to it instead of restating the
   stack's behaviour. Net documentation lines drop substantially while
   the cache page becomes the single user-facing reference.

Also: code-comment cleanup per `docs/contributing/documentation.md`
guidelines across all touched .py / .sh / .yml files. Multi-paragraph
docstrings, banner separators, and restate-the-obvious comments are
removed; one-line WHY comments remain where the rationale is not
already visible in the code.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 01:08:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b34131ca1d feat(package-cache): add Sonatype Nexus 3 OSS pull-through cache (req-012)
Brings up the package-cache compose service alongside the existing
registry-cache, gated by the same `cache` profile. Bootstrap helper
(scripts/docker/cache/package.sh) auto-rotates the admin password from
the well-known default, accepts the Sonatype CE EULA (3.71+ blocks
artifact downloads with HTTP 403 until /v1/system/eula is POSTed —
disclaimer must be sent verbatim with its unicode quotes), enables
anonymous read, creates one default blobstore, and registers the MVP
proxy set: apt-debian, apt-ubuntu, pypi-proxy, npm-proxy, helm-bitnami,
raw-githubusercontent. The runner gets pip.conf / .npmrc /
sources.list snippets bind-mounted in (each defaulting to /dev/null
when the cache profile is inactive) so the deploy is byte-identical
with cache off.

Cache plumbing:
- scripts/meta/env/cache/{registry,package}.sh: per-cache env defaults,
  hostname-derived admin password seed (uses `hostname` command, not
  $HOSTNAME shell var, so the value is identical across interactive
  shells, non-interactive subshells, and Python subprocesses).
- scripts/docker/cache/{registry-ca,package}.sh: container-side
  bootstrap helpers, both moved under scripts/docker/cache/.
- scripts/system/cache/clean.sh + `make cache-clean`: wipes both
  /var/cache/infinito/core/cache/{registry,package} dirs and stops
  the cache containers.
- cli/deploy/development/compose.py: Profile gate exports the three
  new INFINITO_PACKAGE_CACHE_* client-snippet paths and runs the
  bootstrap helper after the package-cache reports healthy.

Compose service uses sonatype/nexus3:3.91.1-ubi pinned, runs as
root (UBI image refuses to chown a fresh bind-mount), and forces
NEXUS_SECURITY_RANDOMPASSWORD=false so the bootstrap can log in with
admin/admin123 before rotating; the random-password mode otherwise
auto-completes onboarding silently and leaves no readable secret on
disk for an automated bootstrap to pick up.

Tests: tests/unit/cli/deploy/development/test_compose.py grows two
new assertions (cache services listed in `up` args when local; build
tests pin CI=true to retain ci-only assertions).

Closes req-012.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:11:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9af3055d80 feat(registry-cache): move env defaults into registry_cache.sh, mandate in compose.yml
The three INFINITO_REGISTRY_CACHE_* variables previously carried their
defaults in compose.yml as `${VAR:-...}`, which baked /tmp paths and
a static 32g cache size into the YAML. Move the defaults into a new
shell module that is sourced via `scripts/meta/env/all.sh` and tighten
compose.yml to require the variables strictly via `${VAR:?...}`.

Why
   * Adapt cache size to the host. Hard-coding 32g hurts hosts where
     `/tmp` is a small tmpfs (typical Linux: ~half of RAM); the cache
     would either be capped above the partition or starve other tmpfs
     users. Computing the default as half the free disk space at the
     cache path leaves a usable budget for the rest of the system on
     small hosts and grows on large ones.
   * Single source of values. With defaults in shell, the compose YAML
     no longer carries duplicated tunables; `compose.yml` just consumes
     what the env layer provides. Same pattern as the existing
     INFINITO_CONTAINER strict consumption.

Changes
* New scripts/meta/env/registry_cache.sh exports the three vars.
  CACHE_MAX_SIZE = `df --output=avail -B1G "$path" / 2`, walking up to
  the closest existing ancestor when the cache path itself does not
  exist yet (it is created on first start by `bind:
  create_host_path: true`). Minimum 1g, fallback 2g if `df` fails.
* scripts/meta/env/all.sh sources the new module so the Makefile-driven
  flow picks it up automatically.
* compose.yml: three `${VAR:-default}` -> `${VAR:?Source scripts/meta/env/registry_cache.sh ...}`
  so the absence of the env layer fails fast with a precise message.
* Stale block comment above the registry-cache service (CI-only,
  /tmp-defaults, etc.) collapsed to a one-line pointer to the README,
  per documentation.md "default to no comments / explain only the
  non-obvious why".
* docs/contributing/environment/variables.md and
  docs/contributing/artefact/files/compose.yml.md describe the new
  defaults source and the half-free-disk computation.

Verified
   `bash -c 'source scripts/meta/env/all.sh && docker compose --profile ci --profile cache config'`
   resolves the three vars (CACHE_MAX_SIZE: 3g on a host with 7G free
   in /tmp). Without sourcing, compose fails with the exact
   "Source scripts/meta/env/registry_cache.sh ..." message.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 00:38:58 +02:00
kevinveenbirkenbachandClaude Opus 4.7 94936f198b feat(compose): gate registry-cache to local dev only via Profile class
Move the registry-cache pull-through proxy out of the always-on `ci`
profile into its own `cache` profile, and select it based on the runtime
environment instead of statically.

Why
   The proxy only earns its keep when image pulls amortize across many
   runs on the same host. That holds on developer machines (cross-run
   image dedup, persistent host disk) but not on GitHub-hosted runners
   (fresh disk per job), where the proxy startup overhead is net loss
   without the corresponding payoff.

How
* Compose: split `registry-cache` into its own `cache` profile;
  `infinito.depends_on.registry-cache` gets `required: false` so Compose
  does not auto-start the proxy via the dependency when the profile is
  inactive; the systemd drop-in bind-mount source is gated by the new
  `INFINITO_REGISTRY_CACHE_PROXY_CONF` env var with `/dev/null` default,
  so dockerd loads an empty drop-in (no `HTTP_PROXY`, no `ExecStartPre`)
  when the proxy is not running and pulls images directly.
* New `cli/deploy/development/profile.py` `Profile` class is the SPOT
  for which compose profiles activate. KISS + SRP: `is_ci()` reads the
  three CI signals (`GITHUB_ACTIONS`, `RUNNING_ON_GITHUB`, `CI`),
  `registry_cache_active()` is its strict inverse, and `args()` returns
  the matching `--profile ...` flag list.
* `Compose` and `down.py` consume the same `Profile` so up/down cannot
  drift on which services are part of the stack.
* Documentation under `compose/registry-cache/README.md`,
  `docs/contributing/artefact/files/compose.yml.md` and
  `docs/contributing/environment/variables.md` describes the new toggle
  and the `INFINITO_REGISTRY_CACHE_PROXY_CONF` env var. Pre-existing
  documentation.md violations in those files (one SPOT self-label, one
  em dash, two path-as-link-text instances) corrected in the same pass
  per the fix-nearby rule.
* Tests: 14 new `Profile` unit tests in `test_profile.py` plus two
  scenario tests in `test_compose.py` that lock down the
  `--profile ci` (CI runner) vs `--profile ci --profile cache` (local)
  argument shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 00:25:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1dd8d3f5c0 feat(compose): add pull-through registry-cache + reorg compose/ subdirs
Introduces a `registry-cache` service (rpardini/docker-registry-proxy) in
the `ci` profile that MITM-caches every outbound docker registry pull
made by `infinito`'s inner dockerd, drastically cutting bandwidth on
re-deploys, parallel CI shards, and local fresh-purges.

Wiring (all bind-mounted via compose.yml, no Dockerfile or entry.sh
staging, so edits hot-reload on next container start):
- CA bundle at /opt/registry-cache-ca (read-only),
- systemd drop-in at /etc/systemd/system/docker.service.d/registry-cache-proxy.conf,
- registry-cache-ca.sh at /usr/local/bin/registry-cache-ca.sh.

The drop-in registers registry-cache-ca.sh as ExecStartPre. Compose
gates `infinito` on `registry-cache` `service_healthy` so the CA is on
disk before dockerd boots; the install script supports both Debian
(`update-ca-certificates`) and Arch/Fedora (`update-ca-trust extract`)
trust stores, detected by command availability.

Also reorganizes compose/ into per-service subdirectories
(compose/coredns/, compose/registry-cache/) with README.md per directory
following the documentation guidelines, adds scripts/docker/README.md,
and nests the registry-cache host paths under /tmp/infinito/core/
(mirror/ for blobs, ca/ for the MITM CA bundle).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 18:14:11 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f2021b07d refactor: collapse INFINITO_DISTRO/INFINITO_CONTAINER to a single SPOT and align all consumers
Follow-up to d002796a5. The formula `infinito_nexus_${INFINITO_DISTRO}` is
now defined in exactly one place — `scripts/meta/env/defaults.sh:18`,
exported outside the load-once guard so it tracks INFINITO_DISTRO across
matrix iterations (caller re-sources defaults.sh after distro changes).
Every other site reads INFINITO_CONTAINER strictly from env; nothing
re-derives the formula, no lazy/colon-dash fallbacks, no per-call defaults.

Production code aligned:

- cli/deploy/development/common.py: add resolve_container() (strict env
  reader, raises SystemExit on unset). make_compose() calls it as an
  early-fail guard so a missing defaults.sh source surfaces here instead
  of as a confusing compose.yml interpolation error later.
- cli/deploy/development/deploy.py: stop re-deriving from compose.distro;
  use resolve_container().
- compose.yml: container_name reads ${INFINITO_CONTAINER:?...} strictly,
  so callers that bypass defaults.sh fail fast with a clear hint.
- scripts/meta/resolve/image/local.sh: drop ${INFINITO_DISTRO:-debian}
  fallback; require INFINITO_DISTRO via :? error.
- scripts/tests/deploy/ci/dedicated.sh: drop local _container fallback.
- scripts/tests/deploy/local/deploy/fresh-kept-app.sh: drop redundant
  ${INFINITO_DISTRO:-debian} reassignment after sourcing env/all.sh
  (defaults.sh already defaults the value).
- scripts/tests/deploy/local/deploy/fresh-purged-app.sh: drop
  target_container fallback; INFINITO_CONTAINER is already in env.
- scripts/tests/deploy/local/exec/container.sh: drop the lazy
  ${INFINITO_CONTAINER:=infinito_nexus_...} default; defaults.sh source
  in env/all.sh already guarantees the value.
- scripts/tests/dns.sh: source defaults.sh so callers only need to set
  INFINITO_DISTRO (the implicit dependency was previously a footgun).
- scripts/system/tls/trust/{wsl2,linux}.sh: same — source defaults.sh,
  drop the ${INFINITO_CONTAINER:-infinito_nexus_debian} fallback in
  linux.sh.
- .github/workflows/test-dns.yml: drop redundant
  INFINITO_CONTAINER: infinito_nexus_${{ matrix.distro }} from per-step
  env blocks (3 sites). The env now flows from INFINITO_DISTRO via the
  defaults.sh sourced inside the scripts.
- tests/unit/cli/deploy/development/test_matrix_deploy.py: replace the
  module-level os.environ.setdefault('INFINITO_CONTAINER', 'infinito_nexus_arch')
  shim — which itself encoded the formula — with a module-level patch of
  resolve_container that returns an opaque test-fixture string. Drop the
  now-unused distro="arch" field from the _args() Namespace fixture
  (deploy.handler stopped consuming it after the --distro arg removal).

Documentation aligned:

- docs/contributing/environment/variables.md: rewrite the INFINITO_CONTAINER
  row to spell out the SPOT contract — set automatically by defaults.sh,
  never re-derived elsewhere, callers re-source defaults.sh after
  changing INFINITO_DISTRO mid-process.
- docs/contributing/artefact/files/compose.yml.md: replace the
  INFINITO_DISTRO row (which described the now-removed
  `infinito_nexus_${INFINITO_DISTRO:-debian}` interpolation) with an
  INFINITO_CONTAINER row noting the strict ${INFINITO_CONTAINER:?...} read.

Verified:
  - Formula derivation: 1 production hit (the SPOT in defaults.sh:18).
  - INFINITO_CONTAINER fallback patterns (:-/:=): 0.
  - --distro CLI arg refs: 0 (one comment explaining the removal).
  - make test green: 1553 unit + 168 integration tests, no skipped suites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 12:21:35 +02:00
kevinveenbirkenbach 9cbb12d540 Propagate host MTU to the local compose network 2026-03-31 21:00:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 a50c90c733 fix(deploy): resolve SERVICES_DISABLED service names to app IDs and verify inventory
- deploy.py: use find_provider_roles() to map service names (e.g. 'matomo')
  to provider app IDs (e.g. 'web-app-matomo') before filtering deploy_ids;
  forward SERVICES_DISABLED into the container via extra_env
- container.sh: rename INSPECT_CMD to CMD for consistency; fix indentation
- Makefile: expose CMD variable on exec target for non-interactive use
- environment.sh: source scripts/meta/env/all.sh for INVENTORY_DIR; add
  inventory verification after SERVICES_DISABLED deploy using make exec
- variables.md: document CMD variable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 01:26:44 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 9d863f1ef4 feat(inventory): add SERVICES_DISABLED env var support and environment test flow
- Add services_disabler.py: parse SERVICES_DISABLED, disable services in host_vars,
  remove provider roles from inventory
- Wire apply_services_disabled_from_env into cli/create/inventory/cli.py after deploy
- Add unit tests for all services_disabler functions
- Update hardware.md and variables.md with SERVICES_DISABLED documentation
- Update environment.sh: deploy dashboard with SERVICES_DISABLED=matomo first,
  assert dashboard reachable + matomo not reachable, then deploy matomo and assert
  reachable; extract assert_http_status / assert_http_status_not helpers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 15:00:24 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 06a688c59a refactor(deploy): rename APP to APPS throughout the project
Makes clear that multiple space- or comma-separated app ids are
supported. Renames the env var, CLI flag (--app → --apps), and all
references in scripts, workflows, and documentation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 11:41:31 +02:00
kevinveenbirkenbach 6d98ca4cd0 refactor: normalize local deploy layout 2026-03-28 06:20:06 +01:00
kevinveenbirkenbach 67fadb57b8 refactor: normalize inventory filename to devices 2026-03-28 05:40:13 +01:00
kevinveenbirkenbach 49e3d5cd9a docs: add environment variable reference 2026-03-28 01:46:29 +01:00