Commit Graph
39 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Fable 5 407b8a8a1f fix(e2e): use the tls lookup instead of the unregistered urlsplit filter; autoformat
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:05:59 +02:00
kevinveenbirkenbachandClaude Fable 5 c505ce7466 test(lint): forbid unmarked '|| true' in shell scripts; grandfather the corpus
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:39:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c3ff093df0 refactor(scripts/deploy/local): mirror host wrappers with container/ subtree
Renames the three in-container deploy helpers out of `utils/` (where
they lived next to unrelated bash libs) into a `container/` subtree
that mirrors the host-side `apps/` layout one-to-one:

  utils/fresh-kept-all-init-and-deploy.sh  -> container/initialize/all.sh
  utils/reuse-kept-all-deploy.sh           -> container/update/all.sh
  utils/reuse-kept-app-deploy.sh           -> container/update/selection.sh

The `fresh-/reuse-/kept-` prefixes were overloaded ("kept" was
universal, "fresh/reuse" duplicated the verb already encoded in
initialize/update). New names use the same verbs + scope as the host
wrappers, so the host/container halves of each flow share a name.

Updates the three host wrappers under apps/{initialize,update}/ to
point at the new in-container paths and modernizes their
cache-retry keys (`fresh-kept-all` -> `initialize-all`, `reuse-all` ->
`update-all`, `reuse-<app>` -> `update-<app>`).

Adds READMEs for container/, container/initialize/, container/update/
following the project documentation conventions (H1 + trailing emoji,
RFC 2119 keywords, env-var tables, cross-links to the matching host
wrappers) and extends deploy/README.md with a pointer to the new
subtree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbach 5545cbe9c7 fix(test-deploy): run entry-bootstrap with --compile -- true
`entry.sh true` skips `run_local_build` because `true` is treated as
the exec target rather than a flag, so the in-container reinstall step
never ran. `--compile -- true` triggers the build first and then runs
`true` as a no-op exec target.
2026-05-21 22:58:28 +02:00
kevinveenbirkenbach 664457b739 test(lint): verify every make <target> reference resolves
Catches refactor drift across `.sh`, `Makefile` recursive calls, and
GitHub workflows. Includes fixes for the 5 bugs the new test surfaced
on first run: `make -C ROOT down`/`up` in the network stack refresh
helper (missed during the compose-* rename) and four
`make compose-deploymode=…` / `compose-deployapps=…` typos in
07/08/09_deploy_*.sh plus three docstring/echo references
(missing space after `compose-deploy`, introduced by the
deploy → compose-deploy sed-rename).
2026-05-21 22:58:27 +02:00
kevinveenbirkenbachandClaude Opus 4.7 605f29a2f8 refactor(make): rename deploy to compose-deploy
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.

Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 356e3366be feat(make): consolidate local deploy into single make deploy router
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.

Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.

Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.

Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 20:22:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a0620f7bb fix(inventory): drop provision-CLI args that shadowed group_vars defaults
Root cause of the DinD DNS-check failure on test-deploy-server
(networks.internet.ip4 falling through to 127.0.0.1 despite
INFINITO_IP4=172.30.0.10 reaching every link of the env pipeline):

  cli/administration/inventory/provision/cli.py declared --ip4 with a
  hardcoded default of "127.0.0.1" and ensure_host_vars_file wrote that
  value unconditionally into host_vars/<host>.yml. group_vars'
  defaults_networks.internet.ip4 resolves "172.30.0.10" via
  \`lookup('env', 'INFINITO_IP4')\`, but tasks/stages/01_constructor.yml
  merges \`defaults_networks | combine(networks | default({}, true))\` so
  host_vars wins. INFINITO_IP4 looked broken end-to-end; the breakage
  actually lived in the provision-CLI host_vars overlay.

  INFINITO_DNS_IP and INFINITO_DOMAIN propagated correctly because
  the provision CLI had no --dns and the --primary-domain write was
  None-gated and never invoked.

While here, --ssl-disabled and --primary-domain are removed too:
neither had any effect in the dev flow. provision/cli.py applies
\`--vars-file inventories/development/default.yml\` AFTER
ensure_host_vars_file via apply_vars_overrides_from_file, whose
_deep_update_commented_map unconditionally overwrites
(\`target[key] = value\`). The dev vars-file always set
\`TLS_ENABLED: true\`, so \`--ssl-disabled\` (which forced
\`TLS_ENABLED: false\` in host_vars) was systematically reverted to
true. Dev/CI ran with TLS enabled the whole time. Inventory stays at
\`TLS_ENABLED: true\` to preserve that effective behaviour.

Removed from cli/administration/inventory/provision/cli.py:
- --ip4, --ip6: defaulted to 127.0.0.1 / ::1, shadowed group_vars
- --primary-domain: never set by any caller, None-gated write
- --ssl-disabled: always re-overwritten by --vars-file later in the
  same provision pass

Removed from cli/administration/inventory/provision/host_vars.py the
matching writes in ensure_host_vars_file; the function now only sets
\`ansible_connection: local\` for localhost-class hosts. Defaults for
DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6} live in
group_vars (env-driven) and the inventory vars-file (deployment-wide
flags); host_vars stops duplicating them.

Callers cleaned up:
- cli/administration/deploy/development/inventory/builder.py
- cli/administration/deploy/container/command.py
- scripts/tests/deploy/local/utils/fresh-kept-all-init-and-deploy.sh
- scripts/tests/deploy/local/reset/inventory.sh

Tests + docs follow:
- tests/unit/cli/administration/inventory/provision/test_host_vars.py
  now asserts host_vars no longer carries baked DOMAIN_PRIMARY /
  TLS_ENABLED / networks defaults
- docs/administration/deploy.md drops --ssl-disabled from the
  provision example
- docs/contributing/design/inventory.md re-scopes the host_vars
  column: the DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6}
  values arrive via --vars / --vars-file overlays, otherwise group_vars
  provides the env-driven default

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 17:05:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a77a570409 refactor(env): consolidate INFINITO_* namespace and centralize path/image SPOTs
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:

* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
  env var → defaulted to infinito.localhost, which never resolved
  because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
  INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
  (env-builder, role default, run_one.yml docker exec, playwright.config.js,
  lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
  the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
  compose forwards it as build-arg, volume mount path, and container env;
  every Python CLI + shell helper + CI workflow now reads it strictly
  (os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
  /opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
  INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
  utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
  container) through compose.yml's environment block.

Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 02:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 80c396b839 refactor(env+install/spot): collapse python/venv resolution into shell SPOT
Pulls VENV/PYTHON/PIP/PYTHONPATH out of the dotenv handler pipeline
into scripts/meta/python/detect.sh. load.sh now sources detect.sh and
prepends ${VENV}/bin to PATH above the INFINITO_ENV_LOADED guard, so
every BASH_ENV re-entry re-resolves PYTHON the moment a venv
materialises mid-chain — the same fix as b79ed2869 but at the SPOT,
which retires the reload-trick from ansible.sh, python.sh and
pre-commit.sh.

Also renames INVENTORY_VARS_FILE → INFINITO_INVENTORY_VARS_FILE under
the new env-prefix lint, drops the 5 dead INFINITO_ENV_*_LOADED unsets
left over from the multi-loader era, and adds a lint guarding the
INFINITO_ prefix invariant on env/*.env keys.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 00:39:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 eb4bcd23a8 refactor(env/spot): forbid .get() defaults + cache namespace rename
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
  Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
  Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
  a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
  Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
  empty-string fallback (which silently masks an unset key). The
  nocheck check walks every physical line the call spans so an
  autoformat-split call still honours the marker.

Cache namespace flattens to a single prefix:
  INFINITO_PACKAGE_CACHE_*  -> INFINITO_CACHE_PACKAGE_*
  INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.

SPOT-strict rewrites:
  - 19 sites: `something.get("INFINITO_X", "")` -> subscript
    `something["INFINITO_X"]` (loud KeyError when env loader did
    not run; all keys live in env/default.env).
  - 2 handler-internal bootstrap reads suppressed with
    `# nocheck: handler-bootstrap-read`.
  - 30 runtime-only / driver-injected / test-fixture references
    annotated with `# nocheck: <reason>` per-line.
  - Unit tests that previously removed SPOT-guaranteed keys via
    `{k: v ... if k \!= KEY}` or `clear=True` switch to setting
    the key to "" explicitly, matching the new contract.

Side fixes:
  - INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
    shell-side defaults in common.sh to env/default.env and
    passthrough.STATIC_KEYS.
  - dns_systemd_is_operational() drops the dead
    INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
    variable was never set anywhere).
  - INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
    to `os.environ[KEY]`.
  - scripts/tests/code/wrapper.sh exec args move into a bash
    array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
    own element line (SC2215 fix).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 22:26:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a578952c3 feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.

Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
  BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
  PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
  WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
  promoted IMAGE_TAG from GHA-only to general passthrough

compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions

Tooling:
- scripts/install/all.sh owns the stamp-file install logic
  (Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
  8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env

Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
  filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator

Docs:
- variables.md is now a pointer; per-variable docs live next to
  the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e2136a3bc7 fix(cache-retry): truncate sanitized label to 80 chars before mktemp template
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 01:09:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5fd55072c7 fix(scripts/tests/deploy/local/utils/cache-retry): switch stale-apt recovery to Nexus invalidate-cache
Replace the no-op `docker restart` recovery with `POST
/service/rest/v1/repositories/{name}/invalidate-cache` against each
apt-format proxy repo (Debian, Ubuntu, …) discovered dynamically
from Nexus' REST API. Container restart never re-validated cached
content; the new path makes Nexus actually re-fetch from upstream
on the next request and keeps repo definitions, blobstore, and
admin credentials intact.

Co-located changes:
- fresh-purged-app.sh: source scripts/meta/env/all.sh so the cache
  override env (incl. INFINITO_PACKAGE_CACHE_FRONTEND_IP) is set
  for the deploy CLI — without this, apt traffic bypasses Nexus.
- utils/nexus/list_apt_proxy_repos.py: stdin JSON → stdout name list,
  exit 1 on malformed input / no matches; used by the bash helper
  for discovery (parser lives outside the heredoc for testability).
- tests/unit/utils/nexus/test_list_apt_proxy_repos.py: 10 cases
  covering filter rules, empty/invalid input, non-dict entries,
  missing/empty names, and a realistic live payload.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 00:44:47 +02:00
kevinveenbirkenbachandClaude Opus 4.7 51a2a9113c feat(deploy/local): auto-clear caches + retry on stale apt Release file
When the registry-cache or package-cache (Nexus / nginx frontend)
serves apt repository metadata whose `Valid-Until` window has elapsed,
Docker builds inside an Ansible-driven deploy abort with
  E: Release file for http://deb.debian.org/debian/dists/.../InRelease
     is expired (invalid since 5h 30min 54s).
which surfaces as a generic non-zero exit from the wrapped deploy
command and forces the operator to manually wipe the caches before the
next attempt.

Add `scripts/tests/deploy/local/utils/cache-retry.sh` exposing
`deploy_with_cache_retry <label> -- <cmd> [args...]`. The helper runs
the wrapped command with combined stdout+stderr teed to a temp log,
captures the wrapped command's real exit code via PIPESTATUS (with a
disciplined `set +e` / restore-previous-state dance so the caller's
`set -e` does not abort the script before the helper can react), and
on non-zero exit grep's the captured log for the stale-cache
signature `Release file ... is expired` / `Valid-Until ... expired`.

If matched, the helper wipes both cache stacks via `make cache-clean`
(registry-cache + package-cache through
scripts/system/cache/clean.sh), runs `docker builder prune -af` and
`docker image prune -af` to drop cached build layers and unused base
images, and re-runs the wrapped command exactly once. Non-stale-cache
failures bubble up unchanged so the caller's existing error handling
is preserved. Live in the deploy/update script layer rather than the
Ansible handler because only the host-side scripts can reach the
registry-cache + package-cache containers and the host Docker engine.

Wire the helper into all five local deploy entrypoints:
- fresh-purged-app.sh (both `--full-cycle` and the sync-only branch)
- fresh-kept-app.sh
- fresh-kept-all.sh
- reuse-kept-app.sh (added SCRIPT_DIR/REPO_ROOT locally because the
  script did not yet have them)
- reuse-kept-all.sh

CI deploy paths (scripts/tests/deploy/ci/*) are intentionally left
alone — CI runs do not share the developer's local cache stack and
they handle stale metadata differently.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 11:45:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ffea45b163 fix(test-env): pin matomo reuse-pass to VARIANT=0 + suggest sibling inventories on miss
09_deploy_reuse.sh ran `make deploy-reuse-kept-apps APPS=web-app-matomo`
without VARIANT, which makes reuse-kept-app.sh fall back to the
unsuffixed `${INVENTORY_DIR}` path. The matrix-init in step 08 only
writes `${INVENTORY_DIR}-0` / `-1` because matomo declares 2 variants
(see roles/web-app-matomo/meta/variants.yml; planner contract:
cli/administration/deploy/development/inventory/planner.py:107), so
the reuse pass always missed the inventory and aborted with exit 2.

Also surface sibling inventory dirs as suggestions when the lookup
misses, so the next caller hitting this gets a hint instead of a bare
ENOENT.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 11:45:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a26bfc4e94 refactor(cli): reorg admin/contributing tree + per-folder help system
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):

- cli/deploy             -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault              -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror             -> cli/contributing/mirror

The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".

Help system redesign in cli/core/{discovery,help,app}.py:

- Discovery now distinguishes runnable commands (directories with
  __main__.py) from category folders (directories that only group
  further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
  as the title and its first paragraph as the one-line description in
  the help output.
- "infinito" / "infinito --help" prints only the top-level entries
  (one level deep). "infinito <category>" navigates into a folder and
  shows its sub-folders and commands one level deep, with the README
  title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
  prefix recursively. N caps the depth (1-based); omit N for an
  unbounded walk. The output uses Unicode tree connectors, an emoji
  per entry (folder for categories, gear for commands), a globally
  aligned description column, and multi-line wrapped descriptions
  with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
  ("infinito meta callorder") rather than the bare leaf name, so the
  output is copy-pasteable.

Convention doc + 17 CLI READMEs:

- New docs/contributing/artefact/files/cli/readme_md.md describes the
  H1+paragraph layout the help system reads, and how it differs from
  role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
  and their immediate sub-folders, plus cli/meta/roles/applications
  and its resolution sub-folder. Each follows the "title with trailing
  emoji + single descriptive paragraph" convention.

Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.

Fixes uncovered by the move:

- import unittest.mock added to four dedicated-deploy tests so
  unittest.mock.patch resolves under stdlib unittest discovery
  (Python 3.13 no longer surfaces unittest.mock through "import
  unittest" alone).
- parents[N] index in the new
  cli/administration/inventory/validate/__init__.py corrected from
  3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
  substring assertions updated to match the new validator path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:17:09 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a080dc4f98 refactor(cli): inventory tools under administration/, drop role scaffolder + dead SSO indirection
Move:
- cli/build/inventory/full      -> cli/administration/inventory/devices
- cli/create/inventory          -> cli/administration/inventory/provision
- cli/create/credentials        -> cli/administration/inventory/credentials
- cli/encrypt/inventory         -> cli/administration/inventory/encrypt

Delete:
- cli/create/role + templates/roles/ (stale scaffolder)
- roles/web-app-baserow/templates/javascript.js.j2 + dead sso_warning snippet
- js_application_name across akaunting/baserow/opencloud/opentalk/taiga,
  javascript service disabled for baserow + taiga

Wire-through:
- argv lists in cli/deploy/{container,development}/* + 2 shell scripts
  now invoke 'infinito administration inventory provision'
- subprocess module names + docs (contributing/* + administration/deploy.md)
- discovery resolver test + folder-item-limit whitelist updated
- ruff fixes in devices/command.py (RET504 inline + Path.open over open)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 18:41:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d97ed7a9e8 feat(deploy): per-role matrix-deploy variants with folder-per-round model
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.

Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
  variant twice and ran the async pass against a host left in a
  different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
  as a runtime extra-var and required cross-stage coordination.

How:
- `roles/<role>/meta/variants.yml` (file format docs at
  docs/contributing/artefact/files/role/variants.md): YAML list of
  deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
  variant 0 (= legacy payload). 6 roles get their dev overrides moved
  here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
  drops the retired `_active_variants` runtime selector. Loader always
  returns variant 0 as the default; non-zero variants reach
  `get_merged_applications` as inventory-level `applications.<app>`
  overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
  `plan_dev_inventory_matrix` (pure planner shared between init and
  deploy), `build_dev_inventory_matrix` (materialises one folder per
  round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
  paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
  cleanup (only apps whose variant changed AND that are about to
  re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
  round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
  sync + async per round so the async re-deploy targets exactly the
  host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
  bash callers can inject per-run context into in-container helpers
  without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
  (Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
  test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
  switch to a single init + single deploy `--full-cycle`. The
  `make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
  `INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
  their in-container heredocs to dedicated helpers in
  `scripts/tests/deploy/local/utils/`, removing the bash-in-bash
  quoting hell.

Tests:
- `tests/unit/utils/test_variants.py` (renamed from
  `test_application_variants.py`): loader edge cases, variant-zero
  default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
  variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
  per-round deploy filter, cleanup decisions, `--variant` pinning,
  `--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
  (new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.

Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
  (folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
  flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
  admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
  recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
  declaration" rule; `applications.md` renamed its SoT header for
  consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:47:48 +02:00
kevinveenbirkenbach 6d98ca4cd0 refactor: normalize local deploy layout 2026-03-28 06:20:06 +01:00
kevinveenbirkenbach 67fadb57b8 refactor: normalize inventory filename to devices 2026-03-28 05:40:13 +01:00
kevinveenbirkenbach bee5b2a382 chore: make local deploy lib executable 2026-03-28 02:03:04 +01:00
kevinveenbirkenbach d072ffa664 refactor: share local deploy bool helper 2026-03-27 19:23:45 +01:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach c1a4660a8b refactor(env): split meta env into modular scripts
replace env.sh usage with scripts/meta/env/all.sh, add env directory README, and switch purge entity python resolution to scripts/meta/env/python.sh
2026-03-20 22:19:50 +01:00
kevinveenbirkenbach aded0c3bdd test(deploy): remove interactive flags from local docker exec 2026-03-20 16:36:32 +01:00
kevinveenbirkenbach 3b11d7f469 feat(packaging): add distro packages and install via package metadata 2026-03-19 02:53:15 +01:00
kevinveenbirkenbach 38b1d08ef7 Improve distro compatibility and dev environment testing 2026-03-18 20:00:33 +01:00
kevinveenbirkenbach a715970506 Refactor lint tooling and shell scripts 2026-03-18 12:37:31 +01:00
kevinveenbirkenbach b849976f2d fix(local-deploy): preserve container PATH for infinito CLI 2026-02-22 17:34:48 +01:00
kevinveenbirkenbach 3e9de04704 refactor: move server inventories into bundles 2026-02-13 10:14:17 +01:00
kevinveenbirkenbach 1bb67164e0 ci: add whitelist filtering for deploy test matrix
* Replace ONLY_APP with space-separated WHITELIST across CI and local/act deploy tests
* Rename build-test-matrix.sh to scripts/meta/resolve/apps.sh and apply optional whitelist filter
* Add manual CI entry workflow to run orchestrator with distros/whitelist inputs

https://chatgpt.com/share/698c406c-0968-800f-ae84-1bcca023363c
2026-02-11 09:40:00 +01:00
kevinveenbirkenbach 7c3e942fd5 Removed INCLUDE_RE 2026-02-10 14:53:31 +01:00
kevinveenbirkenbach 558ec41151 Removed EXCLUDE_RE 2026-02-10 14:51:42 +01:00
kevinveenbirkenbach e8f62dfa7c Solved reset bug 2026-02-06 18:02:24 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach a5bbba1eb3 Added inventory to cleanup 2026-02-04 10:07:02 +01:00
kevinveenbirkenbach fe40b48cab tests(local): add non-destructive dedicated deploy and refactor local utils
- Add local dedicated_distro.sh implementing the same two-pass deploy logic
  as CI (ASYNC_ENABLED=false/true) without destructive cleanup
- Switch test-local-dedicated to pure ENV-based interface (TEST_DEPLOY_TYPE, APP)
- Refactor local deploy helpers into utils/ structure:
  - move reset.sh to utils/reset.sh
  - split cleanup into targeted purge helpers (entity, web, lib)
- Update Makefile targets to reflect new structure and workflows

https://chatgpt.com/share/6982804e-1f60-800f-809e-629d0fe62de3
2026-02-04 00:10:00 +01:00