Commit Graph
154 Commits
Author SHA1 Message Date
kevinveenbirkenbach 3941f692de Merge branch 'feature/swarm-nfs' of github.com:kevinveenbirkenbach/infinito-nexus-core into feature/svc-net-tor 2026-08-01 03:13:36 +02:00
kevinveenbirkenbachandClaude Opus 5 0bed81c048 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Twelve conflicts, resolved by intent rather than by side.

Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.

web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').

handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.

svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.

Six follow-on failures the merge produced, fixed at the root:

- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
  design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
  the prefix lint forbids; the pair now lives in utils/github/variant/ as
  bundles.py and bundle_limits.py, with the -m entry point, the ruff
  per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
  three bare timeout literals; both route through gotoOnion and resolveTimeout
  now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
  distros that ship docker without it, the loop device is already detached when
  the probe failed before attaching, and modprobe zfs may fail because the
  /dev/zfs check decides.

make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 03:00:02 +02:00
kevinveenbirkenbachandClaude Opus 5 6c816e893e refactor(github): group the variant modules and drop the blind-except
variant_bundle_limits.py next to variant_bundles.py formed the prefix group
'variant', which the structure lint reads as a folder in disguise. It stayed
green locally because that lint asks git ls-files: the new file was untracked
through every quality run and only became visible once committed. CI saw it
immediately - the sole failure of run 30675284663.

Both move to utils/github/variant/ as bundles.py and limits.py, and the test
mirrors along.

The BLE001 per-file ignore is removed rather than repointed. parse_size raises
humanfriendly.InvalidSize for a malformed string and TypeError only for a
non-string, and the value reaching it is always str - so the catch-all was
never needed. Measured, not assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 02:46:36 +02:00
kevinveenbirkenbachandClaude Opus 5 82a5eb83f2 fix(env): keep the bundle defaults on the bare bootstrap layer
Registering variant_bundle/max_storage in ORDERED_HANDLERS was right - the key
had never reached a generated .env - but the handler imported its default from
utils.github.variant_bundles, and that module pulls humanfriendly and, through
utils.cache.applications, yaml. The .env generator imports every registered
handler and has to run on the bare bootstrap python, so one constant import
took it down.

Move both defaults into a stdlib-only module. This also ends a duplication the
handler layer had already been carrying: size.py spelled out '3' while
variant_bundles held the same 3 - the very thing max_storage tried to avoid by
importing, at the cost of the layer boundary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 02:13:31 +02:00
kevinveenbirkenbachandClaude Opus 5 b0b22fa452 test(env): fail when a handler is missing from ORDERED_HANDLERS
build_env() iterates an explicit registry, so a handler module that nobody
imports never runs and its key is silently absent from the generated .env -
the consumer reads an empty value and behaves as if the feature were off.
Nothing errors, which is what makes the omission survive review.

The check parses the registry with ast rather than importing it, because one
handler pulls a third-party dependency at import time and a registry check
must not depend on the environment it describes.

It immediately found variant_bundle/max_storage.py unregistered, so
INFINITO_VARIANT_BUNDLE_MAX_STORAGE never reached any .env although
utils/github/variant_bundles.py reads it; registering it lifts the generated
file from 129 to 130 keys.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 02:13:28 +02:00
kevinveenbirkenbachandClaude Opus 5 8e19f635be fix(diagnostics): read the local-dumps directory from the env instead of a literal
container.py hardcoded /tmp/infinito-rescue-diagnostics, a second copy of
DIR_RESCUE_DIAGNOSTICS from group_vars/all/05_paths.yml. Move the value to
a generated env key so the paths SPOT stays single: a new handler seeds
INFINITO_RESCUE_LOCAL_DUMPS_DIR from read_group_path(), recurse() forwards
it into nested runtimes, and an unset key now prints why the in-play role
dumps are missing rather than silently collecting nothing.

Registers the handler in ORDERED_HANDLERS - the registry is explicit, not
discovered - and moves rescue_diagnostics_dir.py into the new rescue/
package so the two siblings satisfy the prefix-group lint.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 02:13:27 +02:00
kevinveenbirkenbach 3f84626883 Merge branch 'feature/swarm-nfs' of github.com:kevinveenbirkenbach/infinito-nexus-core into feature/svc-net-tor 2026-07-29 16:30:43 +02:00
kevinveenbirkenbachandClaude Opus 5 61f3fe05fb refactor(ci): extract the per-distro loop SPOT
The serial distro loop (shuffle, time budget, fast-fail skip, exit on the first failing distro) lived inline in scripts/tests/deploy/ci/all.sh and was therefore reachable only from the compose and host deploys. Swarm and guide have to iterate distros with the same semantics, so the loop moves into scripts/tests/deploy/distros.sh and takes the per-distro command as "$@".

ci/all.sh keeps the app-level preamble and execs the loop with the new ci/one.sh, which resolves the CI image and dispatches to the per-app override or dedicated.sh.

Image resolution leaves the loop as well: scripts/meta/resolve/image/ci.sh renders the reference through cli.meta.ci.image_ref instead of respelling ghcr.io/... locally, which meta/distros.yml declares as the single source.

The two CI time knobs are declared in default.env instead of being repeated as literals, so no caller has to invent a budget.

Three paths are dropped as dead: the empty budget that disabled the limit, the failed counter that could only ever print 0, and the branch for an unset deadline.

Both new scripts are pinned by contract tests. Those tests clear BASH_ENV because the test container exports it as the env loader, which would otherwise re-inject every stripped key and make the assertions pass for the wrong reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 21:06:46 +02:00
kevinveenbirkenbachandClaude Opus 5 79ba511bb7 fix(env): keep the .env generator stdlib-only
The env generator runs on the bare bootstrap python, before any
dependency is installed. utils/distros.py reached PyYAML through
utils.cache.yaml, so `python -m cli.meta.env` died with
ModuleNotFoundError on every dev-runtime image that ships python3
without PyYAML.

utils/yaml_bootstrap.load_block reads the block-YAML subset the SPOT
files use with stdlib only. It raises BootstrapYamlError on anything it
cannot parse identically to yaml.safe_load, so meta/distros.yml fails
loud instead of drifting.

The per-line regex lint meant to guard this is gone: one hop of
indirection defeated it and it stayed green while the invariant was
broken. It is replaced by a test that runs the real generator with
every non-stdlib import blocked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:47:01 +02:00
kevinveenbirkenbach 6c20fd4e30 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	cli/administration/inventory/provision/cli.py
#	roles/sys-svc-mail-msmtp/templates/msmtprc.conf.j2
2026-07-28 16:19:34 +02:00
kevinveenbirkenbachandClaude Opus 5 8aba198aec feat(worktree): isolated per-branch checkouts sharing one cache stack
Parallel checkouts collided on the compose subnet, host ports and container
names, so a quality run in a worktree died at compose-up. Slot each checkout
via custom.env and layer two overrides on top: the shared cache network in
place of a second cache stack, and the shared .git directory without which
git aborts inside the container.

Both overlays gate on the resource they need, not on the slot number, so a
stray INFINITO_INSTANCE cannot make compose demand a value nobody set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:15:59 +02:00
kevinveenbirkenbachandClaude Opus 5 ebd7d2e72b refactor(meta): centralize category/distro SPOTs
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.

meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.

Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.

tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 16:15:56 +02:00
kevinveenbirkenbach 71c9f10dfd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-07-28 11:22:47 +02:00
kevinveenbirkenbachandClaude Opus 5 e409674718 refactor(meta): centralize category/distro SPOTs
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.

meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.

Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.

tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 11:19:30 +02:00
kevinveenbirkenbachandClaude Opus 5 2d283ed39a feat(ci): resolve image refs from services.yml SPOT
moby/buildkit was hardcoded in four places across two workflows and catthehacker/ubuntu in two act scripts, none of them reachable by the image mirror: utils/docker/image/discovery.py only scans roles/**/meta/services.yml. So the one image the CI build cannot proceed without was pulled straight from Docker Hub, unmirrored, which is what broke the fedora build in run 30160077828.

Declare both as mirror-only services on svc-runner, following the test-e2e-playwright precedent, so the existing mirror pipeline picks them up unchanged. New scripts/meta/resolve/image/service_ref.sh reads image+version back out of the declaration and is the single source for consumers outside Ansible; its mode argument is mandatory (upstream never touches the network, mirror prefers the GHCR tag and falls back to upstream). The fallback is why the mirror job can keep running in parallel with the build instead of gating it.

ACT_PLATFORM_IMAGE moves to an env handler so act consumers read the declared value via BASH_ENV, and workflow.sh now demands it instead of defaulting to a literal. Operator overrides still win through eb.setdefault.

Verified: service_ref.sh in all four paths (upstream, mirror hit, missing mode, invalid mode); the mirror path matches GHCRProvider.image_base() character for character and resolves against the live registry anonymously; ACT_PLATFORM_IMAGE resolves through the real BASH_ENV mechanism; make quality green on all four targets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:53:33 +02:00
Alejandro Roman 871b00670a Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-27 11:25:42 +02:00
kevinveenbirkenbach 0e64865297 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 5 5318df2e9d feat(ci): resolve image refs from services.yml SPOT
moby/buildkit was hardcoded in four places across two workflows and catthehacker/ubuntu in two act scripts, none of them reachable by the image mirror: utils/docker/image/discovery.py only scans roles/**/meta/services.yml. So the one image the CI build cannot proceed without was pulled straight from Docker Hub, unmirrored, which is what broke the fedora build in run 30160077828.

Declare both as mirror-only services on svc-runner, following the test-e2e-playwright precedent, so the existing mirror pipeline picks them up unchanged. New scripts/meta/resolve/image/service_ref.sh reads image+version back out of the declaration and is the single source for consumers outside Ansible; its mode argument is mandatory (upstream never touches the network, mirror prefers the GHCR tag and falls back to upstream). The fallback is why the mirror job can keep running in parallel with the build instead of gating it.

ACT_PLATFORM_IMAGE moves to an env handler so act consumers read the declared value via BASH_ENV, and workflow.sh now demands it instead of defaulting to a literal. Operator overrides still win through eb.setdefault.

Verified: service_ref.sh in all four paths (upstream, mirror hit, missing mode, invalid mode); the mirror path matches GHCRProvider.image_base() character for character and resolves against the live registry anonymously; ACT_PLATFORM_IMAGE resolves through the real BASH_ENV mechanism; make quality green on all four targets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:37 +02:00
kevinveenbirkenbach 1a0af0ce09 Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/sys-ctl-mtn-cert-deploy/tasks/main.yml
#	roles/sys-svc-compose-ca/tasks/main.yml
#	roles/web-app-dashboard/templates/javascript/oidc.js.j2
#	roles/web-app-mastodon/tasks/01_overrides.yml
#	roles/web-app-peertube/templates/compose.yml.j2
#	scripts/github/guide/compose_deploy.sh
#	scripts/github/guide/host_deploy.sh
#	scripts/make/alias.sh
#	tests/lint/ansible/roles/test_service_core_first_task_run_once.py
2026-07-24 04:30:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d9a53a0545 chore(lint): pin ruff to 0.16.0 and resolve the rules that version enforces
The ruff toolchain was unpinned, so host venvs, the lint container and CI
installed different releases over time. That drift is why lint-python only
failed on CI: locally ruff 0.15.x preview-gated ISC004/FURB192 and (falsely)
fired S310 on urllib.request.Request, while CI's 0.16.0 has ISC004/FURB192
stable and drops the S310-on-Request false positive.

Pin ruff==0.16.0 in pyproject.toml's dev extras (Dependabot keeps it current)
and have utils.install.lint.ruff read that pin and reinstall whenever the
found version drifts, so every environment resolves the same ruff. Then fix
what 0.16.0 flags: parenthesise the implicit string concatenations (ISC004),
use min() over sorted()[0] (FURB192), drop the now-unused S310 noqa on the
Request() builders (RUF100), and delete the narration comments that the
comment lint surfaces in the touched files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 23:08:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ceb6e65e49 refactor(ci): single-source the rescue-diagnostics base dir from default.env
INFINITO_RESCUE_DIAGNOSTICS_BASE joins default.env as the only literal
of the runner-side collection root and is wired through the passthrough
handler into the generated .env. The four deploy workflows load it with
one grep bootstrap step after checkout instead of hardcoding the path in
their env blocks; upload paths and per-app DIR derivations reference the
variable. dedicated.sh sources it from .env alongside the existing
diagnostics keys and hard-requires it, no script-level default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
Alejandro Roman 3c69ec4051 Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-23 12:13:04 +02:00
kevinveenbirkenbachandClaude Fable 5 2584de09ef Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Conflicts: keep both new passthrough env vars; adopt the swarm-nfs
ollama manager-ops split while porting our model-pull retry hardening
into it. Strip three narration comments the stricter comment lint
rejects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 02:09:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 120096bccf fix(ci): drop compose-covered roles from the host line of the same run
When a run deploys both single-node modes, the host discovery redeployed roles the compose line already covers (e.g. web-svc-html in both matrices). The single-node workflow now signals a co-running compose line via INFINITO_HOST_EXCLUDE_COMPOSE (a run-context registry key like INFINITO_DEPLOY_MODE), and the host query excludes the compose line's actually-triggered selection before its own budget cut, so freed host slots go to roles compose does not cover. Priority and regular lines each filter against their own compose counterpart through the identical whitelist/blacklist pass-through; host-only runs stay unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 01:38:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7d21876d3a chore(quality): settle lint follow-ups of the swarm-topology and matomo commits
Sort the new INFINITO_SWARM_* IP/subnet keys alphabetically in default.env, wire them through the static passthrough handler so make dotenv materialises them, and shorten the matomo DB-reset task name under the 80-char limit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:50:17 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
Alejandro Roman 1915ec115b Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-22 12:16:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 43d23ef696 refactor(agents): reorganize operator workflows into i8- skills
Move the operator cheatsheet's copy-paste prompts into per-workflow i8- skills
that each route to their authoritative procedure doc, and reduce the cheatsheet
to a table index plus a usage paragraph, enforced by a new lint. Add skills for
develop, test, quality, refactor, the debug and requirement flows, and
commit / pull-request / push-trigger-pull; rename the iteration skills to
i8-iterate-* and rename the index to i8-help. Wire the active-listening,
dialectic, and robot skills into the flows for clarification, uncertain root
causes, and autonomous execution. Drop the Infinito-specific agent aliases
(alias.md, INFINITO_ALIAS_MD, the make alias agent section, and the two alias
tests); terminal aliases stay.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:44 +02:00
Alejandro Roman ac101d2efa Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-21 10:00:17 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 327fb7a2d2 fix(ci): stop materialising INFINITO_TEST_TYPE into the shared .env
INFINITO_TEST_TYPE is a per-invocation label the Makefile sets differently
per parallel test target (external/integration/lint/unit), not a persistent
default. Registering it as a static passthrough key wrote a per-target-varying
value into the shared .env that make test regenerates concurrently, which
tears the file for the env-file schema lint. Drop it from default.env and the
passthrough handler and suppress the run.sh reference with the
makefile-supplied nocheck that lines 15 and 27 already use, hoisting the junit
path into a variable so the continuation line can carry it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 03:49:24 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3b5cd133d1 fix(ci): register INFINITO_TEST_TYPE as a static passthrough key
run.sh reads INFINITO_TEST_TYPE for the per-target junit report filename
but the key was undeclared, failing the used-keys, dotenv-generator and
alphabetical-sort lints. Add the empty default to default.env (after
INFINITO_TEST_RUNNER) and wire it through the static passthrough handler
so make dotenv materialises it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 02:39:28 +02:00
kevinveenbirkenbachandClaude Fable 5 9bce39009c feat(ci): derive deploy-matrix slots from the workflow chain
cli.meta.ci.slots statically and conservatively counts every job the
CI run spawns outside the deploy matrices (reusable workflows opened
recursively, static matrices multiplied out, dynamic ones estimated,
worst entry-*.yml overhead included) and splits the remaining
256-job run budget between compose/swarm/host by the _SHARES weights,
with a per-mode floor and _SLOT_OVERRIDES pins. INFINITO_MAX_JOBS
defaults to 'auto', resolved per mode in apps.sh; the host workflow
drops its hardcoded cap. --matrix renders the budget as a table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Fable 5 e32df7f24e feat(ci): single-node SPOT + priority deploy line
test-deploy-single-node.yml bundles the compose and host deploy tests
behind one reusable workflow; the orchestrator gains a priority input
that deploys focus roles first per mode and runs the remaining roles
only after every priority job succeeded, excluded via the new
INFINITO_BLACKLIST discovery filter in apps.sh. entry-manual exposes
the priority field and treats priority-only dispatches as roles-only.
Workflow dependency diagram added under .github/workflows/README.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:00 +02:00
Alejandro Roman ab25684714 adrees all commens and fix CI usses on prev run 2026-07-16 19:19:23 +02:00
kevinveenbirkenbachandClaude Opus 4.8 84fc309699 fix(lint): resolve make quality lint-test failures
Surfaced by make quality: alias comments on i8matiso/i8mativaso/i8s; self-path nocheck + drop two narration comments in load.sh (venv note kept in the header); shorten the host deploy job name to <=80 chars; move the legend rationale from an inline comment into the docstring. Also move utils/env/handlers/infinito/fork_{account,repository_url}.py into a fork/ subfolder (shared-prefix rule) and update the handler imports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d0ae004eb0 feat(env): fork-identity vars + CI-redeploy aliases
INFINITO_FORK_REPOSITORY_NAME (default.env, static) plus two handlers: INFINITO_FORK_ACCOUNT defaults to the local OS username (getpass.getuser()), and INFINITO_FORK_REPOSITORY_URL composes https://github.com/<account>/<name>. Registered in ORDERED_HANDLERS with the account handler before the URL handler; documented in docs/contributing/environment/variables.md.

Two operator aliases build on it: i8cire (make quality + git-sign-push + retrigger the failed swarm CI jobs of a run) and i8puci (the same without the quality gate). Both take a run id or full URL: a bare id is expanded against INFINITO_FORK_REPOSITORY_URL, a URL is passed through, and a missing INFINITO_FORK_REPOSITORY_URL fails fast.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:16 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc0799883f refactor(structure): move prefix-grouped files into named subfolders
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:

- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
  nested a second level); registry imports and the four handler-dir
  globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
  select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
  includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
  filters() dict keys, so Jinja usage is unaffected; ansible discovers
  the one-level-deep subdir)

Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ab3accba3a refactor(paths): derive var-lib subpaths everywhere from the paths SPOT
Add DIR_BACKUPS to utils.paths and INFINITO_DIR_BACKUPS /
INFINITO_DIR_SECRETS env handlers, derive the recover-console constants
(including NFS_EXPORT_STATE) from utils.paths and the nfs export SPOT,
and switch every remaining self-built /var/lib/infinito subpath - the
secrets recover targets, the swarm extras backup source, the users
token-store candidate chain, the CI recover drill, the swarm NFS
diagnostics and the playwright reports dir - to those constants. Tests
consume the env constants directly; narration comments in the touched
files are dropped per the comment lint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ac09f25750 fix(cache): metadata/negative TTL so unknown packages refresh
Split the single cache freshness window into content (immutable artifacts: wheels, debs, rpms) and metadata/negative-cache (indexes, not-found lookups). Content keeps the long 6-day window; metadata and negativeCache.timeToLive default to 0 so Nexus always revalidates indexes, and a newly published version shows at once instead of being masked by a stale cached index.

ensure_proxy_repo now PUT-updates an existing proxy (POST only creates), so the split applies to already-provisioned repos. Covers every ecosystem proxy (apt, pypi, npm, rubygems, packagist, alpine, go), not just python.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 02:47:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5541b132d1 feat(backup): recover drill between compose deploy passes
Add scripts/tests/deploy/ci/recover_drill.sh: seed a backup root (nfs, volume, secrets, each with a marker) plus a docker volume, then run recover full <root> localhost to restore every backup-stored type into its live target and verify the markers. Runs between PASS 1 and PASS 2 of the compose development deploy.

Hook it via _maybe_recover_drill in cli.administration.deploy.development, gated by INFINITO_RECOVER_DRILL (default true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 02:47:40 +02:00
kevinveenbirkenbachandClaude Fable 5 a3c9db91bb test(swarm): backup+restore DR drill exercising the full svc-bkp-* chain
Add routine/backup/: base.sh drives a marker seeded into the live NFS
volume and the manager secrets through the whole chain (volume+secrets+nfs
backup units -> remote-2-local pull -> local-2-device LUKS 'USB') and back
via every recover.py (device -> local root -> nfs export, docker volume and
host secrets), asserting each marker survives. Each per-host step is one
in-node routine run over docker exec. extend_inventory maps the volume/
secrets/nfs backup roles onto their hosts so the units install; write_extras
generates the pull keypair; the INFINITO_SWARM_BACKUP_* keys are added to
default.env + the passthrough handler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5 1e85bea1d9 feat(env): publish INFINITO_PLAYWRIGHT_STAGE_BASE_DIR from the role vars SPOT
New handler reads TEST_E2E_PLAYWRIGHT_STAGE_BASE_DIR from
roles/test-e2e-playwright/vars/main.yml (via ROLE_FILE_VARS_MAIN, plain
read_text line parse because env handlers bootstrap fresh hosts before
PyYAML exists) and publishes it as INFINITO_PLAYWRIGHT_STAGE_BASE_DIR.
rerun-spec.sh consumes the handler value and falls back to parsing the
same SPOT on DiD nodes without a generated .env, dropping its /tmp path
copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:52 +02:00
kevinveenbirkenbachandClaude Fable 5 bb5f569d56 fix(env): fall back to sysfs minimum MTU when no default route is readable
Agent sandboxes run in a network namespace with an empty /proc/net/route
while /sys/class/net still exposes the host NICs, so dotenv generated the
static 1500 fallback although the host uplink is 1400 and every TLS
handshake from the DinD stack blackholed. Take the minimum MTU across UP
physical interfaces (device symlink present) before resorting to the
static default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 094c02a69d feat(diagnostics): single SPOT rescue script with CI-uploaded snapshots
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.

The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.

Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f08d71534a feat(tooling): operator alias, shortcut, skill, cheat and onboard system
Add the i8* terminal aliases (make install-alias) and ai8* agent
conversation shortcuts (docs/.../agents/alias.md), move skill
distribution to the external INFINITO_SKILLS_REPOSITORY (make
install-skills), and add make cheat, make alias and make onboard. Introduce
custom.env operator overrides (win over defaults and handlers) and pin
INFINITO_ALIAS_MD / INFINITO_ALIAS_REPOSITORY / INFINITO_SKILLS_REPOSITORY
in default.env. Drop the in-repo skills-lock.json and its update-skills CI
job. AGENTS.md instructs agents to suggest the matching cli/prompt alias
when an operator writes something an alias covers. Lints enforce alias
sorting, i8/ai8 prefixes, no consonant clusters, full make-target
coverage, and terminal/agent shortcut non-overlap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 99ed66d716 fix(svc-runner): publish no dev-stack ports in nested DooD deploys
- compose/noports.override.yml drops every published port via !override
- compose_file_args appends it when INFINITO_PUBLISH_PORTS=false
- runner instances set the flag; default true keeps other flows unchanged
- central yaml loader + ansible-lint config accept compose merge tags

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8519642373 refactor(storage): SPOT the nfs export base and client version in their roles
storage.nfs shrinks to the deployment-specific server address: export_base moves into svc-storage-nfs-server's services.yml (the server dictates where exports root) and the mount protocol version into svc-storage-nfs-client's services.yml (the client owns how it mounts). Both are read through utils.storage.nfs loaders by the nfs lookups, the env handler and 00_topology, eliminating the unrendered-variables class for these values; the group_vars assert shrinks accordingly and the swarm test extras stop restating role defaults.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:54:32 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9f8aa6e9b6 refactor(env): SPOT the install scripts and derive the nfs export base from group_vars
INFINITO_PYTHON_INSTALL_SCRIPT and INFINITO_DOCKER_CLI_INSTALL_SCRIPT join the default.env SPOT: the Dockerfile consumes them as required args (:? in the RUN as last defense), all three image builders (compose.yml, build.sh, push.sh) pass them hard-required, the shell consumers read the SPOT via default.env grep and the Makefile via BASH_ENV. INFINITO_SWARM_NFS_EXPORT_BASE is no longer a static default: a new env handler derives it from the deploy SPOT group_vars/all/15_storage.yml, 00_topology greps that file (and the STATE_SUBDIR constant) directly, and the swarm test extras stop restating the group_vars value. Stale hadolint directives are restored and recognised; leftover narration comments in the touched files are dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00