Commit Graph
152 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 088c8be14f fix(ci): pull workspace images through the GHCR mirror like the deploy jobs
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.

The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.

call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 22:40:13 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 dbdf333e16 docs(workflow-loop): name what the local act loop cannot prove
The workspace suite diverges from a runner in two ways that make whole classes of defect unreproducible locally. INFINITO_DOCKER_VOLUME is a named volume locally and /mnt/docker on GHA (utils/env/handlers/infinito/docker_volume.py); compose down -v removes the former and cannot remove the latter, so nested Docker state is destroyed between tracks locally and carried forward in CI. And act sets ACT=true, so detect_gha_act resolves INFINITO_RUNNING_ON_GITHUB=false where a runner resolves true.

Both facts were already documented in isolation; their consequence was not, and it is the one that matters when someone reports a fix as verified because the local loop went green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:03 +02:00
kevinveenbirkenbachandClaude Opus 5 48845db979 refactor(profile): split runtime capabilities from runtime identity
One boolean answered five unrelated questions: whether the pull-through cache stack runs, whether image pulls take the GHCR mirror, whether the docker data root may be wiped, plus runtime naming and inventory resolution. The workspace is the environment where those answers diverge — it simulates a developer machine on a real runner — so getting one of them meant misreporting where the suite runs and silently paying for the other four.

is_ci() and runs_on_github() stay facts about the machine. cache_stack_enabled, image_mirror_enabled and docker_root_ephemeral become decisions that default off those facts and can be declared explicitly, so an environment states what it wants. Every default reproduces the previous condition exactly, including the distinction the old mirror and wipe gates drew between a real runner and act.

73ae40585 got the cache stack by flipping the identity flag and lost the docker-root wipe with it; the workspace now declares all three capabilities instead. Pairs with 4c068d1cf, which makes the resulting stray bind sources harmless rather than fatal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:03 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 4b28f0b271 docs(test-e2e-cli): name the path the framework actually asserts
run_one.yml stats roles/<role>/files/test/test.sh and fails when it is absent, but the runner guide and the nfs-server TODO both said files/test.sh. The TODO was the costlier of the two: following it would have produced a script the asset check never finds. It now also names templates/test.env.j2, without which discover_cli_roles never picks the role up at all and the test stays silently unrun.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:34 +02:00
kevinveenbirkenbachandClaude Opus 5 ef5ea7cb55 refactor(roles): name the credential file meta/secrets.yml
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.

62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.

Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 12:08:06 +02:00
kevinveenbirkenbach efaa9b1c49 Merge remote-tracking branch 'origin/main' into feature/ai-agent-employees
# Conflicts:
#	roles/web-app-flowise/meta/services.yml
2026-08-10 00:14:48 +02:00
kevinveenbirkenbachandClaude Opus 5 71888d86af refactor(ci): prefix workflows by trigger
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.

Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.

Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbachandClaude Opus 5 b41413389a fix(swarm): reclaim the lab subnet this worktree owns before a run
The lab subnet is pinned per worktree slot while lab network names carry
the cluster id, so a cluster kept from one app blocked the next run of
another with 'invalid pool request: Pool overlaps'. The pre-run teardown
only cleared the same cluster id, which never matched. Matching on the
IPAM subnet instead clears exactly the blocker and leaves a sibling
worktree's cluster alone; swarm-clean stays the host-wide tool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 00:14:50 +02:00
kevinveenbirkenbachandClaude Opus 5 f482b979bd refactor(ci): rename env/guide jobs, drop roles_only
test-environment.yml and test-guide.yml become test-workspace.yml and
test-instructions.yml. Orchestrator job ids, the distros-SPOT matrix
lint, the act example, the diagnostics slug and the rescue artefact
names follow.

Replace the roles_only input with per-job workspace/instructions
tri-state inputs (auto|true|false). 'auto' derives the former
roles_only skip straight from whitelist: both jobs run on a global
scope (whitelist empty or __ALL__) and are skipped on a role-scoped
run, which keeps the PR behaviour identical because the diff resolver
never emits an empty whitelist. entry-manual exposes both as choice
dropdowns defaulting to false, so a manual run opts in explicitly.
The now redundant roles_only output is dropped from
pr_affected_roles.sh and subset_roles.py.

Branch-protection required-status-check names change along with the
job renames.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 12:55:04 +02:00
kevinveenbirkenbachandClaude Opus 5 3a4b1f22ed refactor(ci): derive the dev runtime image matrix from the SPOT
The five images in test-environment.yml were a verbatim copy of the
dev_runtime_image field of every distro in meta/distros.yml. A prepare
job now reads them through utils.distros and publishes a JSON array that
the matrix consumes with fromJson, so adding or dropping a distro no
longer needs a workflow edit. The resolver runs on the bare interpreter
because utils/distros.py is stdlib-only.

An empty list aborts the resolver: GitHub skips a job with an empty
matrix silently, which would leave the run green with nothing tested.

The lint could no longer compare literals, so it walks the wiring
instead: axis expression to needs edge to job output to step id, then
runs that step body the way the runner does and compares what it writes
to GITHUB_OUTPUT against the SPOT. The Act example in the iteration docs
was the last remaining literal and now reads the SPOT too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:47:02 +02:00
kevinveenbirkenbachandClaude Opus 5 a35f0a02a2 docs(svc-storage-nfs-server): correct the stale flavor description
Both iteration guides and the nfs_flavor lookup docstring still described
the flavor as RUNTIME conditional, selecting ganesha only for dev and act
and kernel everywhere else. Since the per-variant switching was dropped,
meta/services.yml pins ganesha for every runtime and kernel is a
per-inventory override.

State that, name meta/services.yml as the single point of truth, and
record which os families ship ganesha.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:25:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cd4d0e3e28 refactor(cli): move cli/deploy/runner under cli/administration/deploy
The runner command joins its siblings (dedicated, development) under
cli/administration/deploy; cli/deploy is gone. Makefile, docs and the
requirement links now point at the new path, PROJECT_ROOT climbs one
level further, and the unit tests moved along.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f08d71534a feat(tooling): operator alias, shortcut, skill, cheat and onboard system
Add the i8* terminal aliases (make install-alias) and ai8* agent
conversation shortcuts (docs/.../agents/alias.md), move skill
distribution to the external INFINITO_SKILLS_REPOSITORY (make
install-skills), and add make cheat, make alias and make onboard. Introduce
custom.env operator overrides (win over defaults and handlers) and pin
INFINITO_ALIAS_MD / INFINITO_ALIAS_REPOSITORY / INFINITO_SKILLS_REPOSITORY
in default.env. Drop the in-repo skills-lock.json and its update-skills CI
job. AGENTS.md instructs agents to suggest the matching cli/prompt alias
when an operator writes something an alias covers. Lints enforce alias
sorting, i8/ai8 prefixes, no consonant clusters, full make-target
coverage, and terminal/agent shortcut non-overlap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbach 9ee4cd0fb9 Merge branch 'main' of github.com:infinito-nexus/core into feature/swarm-nfs
# Conflicts:
#	docs/agents/action/iteration/compose.md
#	roles/svc-db-openldap/vars/main.yml
#	roles/svc-runner/defaults/main.yml
#	roles/svc-runner/tasks/02_cleanup_instance.yml
#	roles/web-app-discourse/meta/server.yml
#	roles/web-app-mailu/tasks/01_core.yml
#	roles/web-app-odoo/tasks/main.yml
2026-07-04 18:16:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ccd0908144 docs(agents/role-loop): add hard pre-redeploy verification gate
Before every redeploy that follows a failed deploy, require one explicit
line: either verified-in-container or a named post-deploy-state-exception.
Forbid treating the redeploy itself as the verification, and reject
state-reset friction, logical inference from a prior deploy, and
"looks correct" as reasons to skip. Playwright-spec failures must reset
any polluted state and be green via make compose-playwright first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 23:15:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 001d880a72 fix(test/swarm): pin roundtrip swarm to one variant per app
A multi-variant app expanded into N swarm matrix entries, and
swarm-zombie ran them all on one host under identical SWARM_NAME
cluster names, so every deploy after the first aborted with
'network ... already exists'. Pin the act matrix to variant 0
(default, overridable via variant=) so a multi-variant app validates
a single cluster, in parity with the compose step's variant=0.
workflow.sh now splits ACT_MATRIX on ';' into multiple --matrix
filters so apps and variant can be pinned together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 87c78275b8 feat(meta/complexity): multi-key sort, filter DSL, CI columns
Extend the 'meta roles applications complexity' report:

- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.

Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:33:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1fb96584b refactor(tests): rename swarm make targets to swarm-* and reorganize env steps
Rename the scripts/tests/environment deploy_* steps (07/08/09) to compose_*; add 12_swarm.sh (svc-db-mariadb via the swarm-* targets) and 13_roundtrip.sh (svc-db-postgres via make roundtrip), with MARIADB_APP/POSTGRES_APP in common.sh. Rename every make act-swarm-* target to make swarm-* (drop the act- prefix) across the Makefile (moved to the alphabetical slot), the deploy/act, deploy/swarm and e2e helper scripts, and the agent iteration docs. Strip mid-code narration comments from the touched scripts per the comment policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c59cb3c959 docs(iteration): require 95% in-container confidence before redeploy
Add the pre-redeploy confidence gate to the compose and swarm loops and
allow bringing up the other deploy mode in parallel for comparison.
Sync the roundtrip cheatsheet prompt to the same rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 11:11:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cf3531b9e3 refactor(svc-storage-nfs-server): drive nfs flavor from meta/variants.yml
The inventory applications override in default.yml is dropped during inventory provisioning (CI ran kernel despite RUNTIME=github + a ganesha override), so select the flavor in meta/variants.yml where it reliably applies: ganesha for RUNTIME in [dev, act], kernel for github CI and production. meta/services.yml keeps kernel as the base default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 07:41:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 539a078475 fix(svc-storage-nfs-server): select ganesha in DinD CI, kernel default
The github swarm legs picked the kernel NFS flavor inside the privileged DinD nfs-server, where kernel nfsd is not network-namespaced and wedges exportfs. Move flavor selection out of the role meta default and into the dev inventory: meta/services.yml now defaults to kernel (production); the development inventory overrides it to ganesha for the containerized runtimes (dev/act/github) and keeps kernel for real hosts (RUNTIME=host).

Add meta/variants.yml (kernel + ganesha) and a TODO to test both flavors via files/test.sh, plus server-side kernel-nfsd diagnostics in 12_collect_diagnostics.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 06:22:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9c029870fe feat(cli): add CI-logs download and analyze tooling
python -m cli.meta.ci.logs.download fetches a run's job logs and artefacts in parallel, with conclusion filters (-s/-f/-c/-k), a jobs.json/summary.tsv manifest, per-request jitter to dodge API rate limits, and idempotent re-runs. python -m cli.meta.ci.logs.analyze clusters the downloaded logs by their root failing task.

Rename the agent CI-triage page debug/pipeline.md to debug/ci.md, point its download/inspect step at the new CLI, and reference the tooling from the contributing CI-debugging page. Add unit tests for both commands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c76c6da6a8 feat(svc-storage-nfs-server): userspace nfs-ganesha flavor for the swarm-test + refactor agent iteration docs
svc-storage-nfs-server gains a backend flavor selected by services.nfs-server.flavor (default "{{ 'ganesha' if RUNTIME in ['dev','act'] else 'kernel' }}"). The ganesha flavor installs nfs-ganesha + FSAL_VFS and renders /etc/ganesha/ganesha.conf (NFSv4, Pseudo == Path so clients mount the real paths unchanged, Root_Squash on the export base and No_Root_Squash on the state subdir), reloaded via systemctl restart. It runs in the dev/act DinD swarm-test, where the host kernel nfsd is not network-namespaced and a privileged node driving exportfs would wedge it into a D-state. The kernel flavor keeps nfs-kernel-server + exportfs -ra for production and every non-dev/act runtime. 01_core dispatches on the flavor, 03_prepare holds the shared export preparation, flavor/{kernel,ganesha}/main.yml hold the per-backend install/render/service steps, and handlers gains the ganesha restart.

docs/agents/action/iteration: role.md renamed to compose.md (Compose Loop); new swarm.md for the swarm deploy loop split out of workflow.md; workflow.md trimmed to GitHub Actions workflow iteration; all cross-references updated; the duplicated README gotcha removed; the NFS gotcha rewritten to the ganesha design.

Also drops a retired-flag history comment in group_vars/all/01_modes.yml and a redundant comment in the init selection script, and allows WebFetch(repology.org) used during ganesha packaging research.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 07:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8547a800ed refactor(ci): unify deploy tests to compose+swarm, retire INFINITO_DEPLOY_TYPE
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.

Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.

Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00
Alejandro Roman c34fbf89cb clean last comments 2026-06-24 18:30:53 +02:00
Alejandro Roman 6161b604d7 fix svc runner with matomo 2026-06-24 18:30:52 +02:00
Alejandro Roman 5d4998452a fix cetos issues 2026-06-24 18:30:52 +02:00
Alejandro Roman 24b435d679 fix distro issue in CI 2026-06-24 18:30:52 +02:00
Alejandro Roman e1bcad75c6 fix github CI issues 2026-06-24 18:30:51 +02:00
Alejandro Roman 9cde8f873d reafactor CLI test for runner 2026-06-24 18:30:50 +02:00
Alejandro Roman c038d147a3 fix runner issue in DinD + addeed docs for CLI runners 2026-06-24 18:30:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3e6bc00bcc docs(agents): add the roundtrip iteration loop
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 20:46:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 761450363b feat(make): add act-runner-image target for Docker-28/29 act runs
Recent Docker tightened docker cp tar extraction, so act aborts at job setup when the stock catthehacker runner image's /var/run symlink is unpacked into. Add a make act-runner-image target that builds local/act-runner-fixed (stock image with /var/run removed); pass it via ACT_PLATFORM_IMAGE. Document the symptom + fix in iteration/workflow.md and cross-reference from playwright.md and README.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 12:16:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 821ff4c386 feat(act-swarm): add act-swarm-playwright helper to rerun specs on the live cluster
Mirror compose-playwright for the act-swarm DinD cluster: copy the working-tree's modified files into the node's frozen bootstrap copy (the nodes are not bind-mounted), then rerun via rerun-spec.sh. rerun-spec.sh gains an opt-in TEST_E2E_PLAYWRIGHT_NETWORK_HOST that switches it to --network host (mirroring the deploy-time swarm branch in 02_run_one.yml) so cross-app canonical domains are reachable on the node; the compose path is unchanged. Document the all-green-before-redeploy gate in playwright.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3de00b3054 docs(iteration): add 'Inspect before redeploy' to the workflow loop
Codifies the swarm/act iteration discipline: validate a candidate fix on the live zombie cluster before triggering another tens-of-minutes rebuild; no commits during an active iteration (commit only after green); use --no-verify if a mid-iteration commit is unavoidable since the pre-commit hook stashes working-tree files a live deploy is reading; make autoformat/make test are safe any time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 16:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7c84617758 docs(role-loop): make matrix-variant command selection unambiguous
Replace the prose that an agent could read as 'no variant passed implies mode=reinstall' with an explicit situation-to-command table plus a FORBIDDEN list: every mode=update is variant-pinned, and a no-variant mode=reinstall is reserved for the initial baseline and the final gate only. A failing variant is retried with mode=update variant=<idx>, never by restarting the whole matrix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 22:38:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e293bd921 fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".

Fix:

- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
  full_cycle, variant) from `passthrough.STATIC_KEYS` and from
  `default.env`. They are per-invocation; .env is the persistent SPOT
  and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
  `os.environ.get("KEY", "")` so the missing-key path stops at "no
  filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
  (lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
  disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
  full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
  INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
  name in every doc; routing-knob env vars now use the same name so
  the cmdline form, the recipe env-prefix, and the consumer reads
  all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
  knobs whose alias differs from the env name (`type` →
  INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
  auto-exports cmdline variables into the recipe's shell env, so the
  identical-name `$(if $(apps),apps="$(apps)")` lines were pure
  noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
  `extra_env` per `docker exec` call and forwards `disable` via
  `-e`, so the container env never needs the value frozen at
  compose-up time.
- Update the 3 actual leak call sites
  (07/08/09_deploy_*.sh) to the make-alias form
  (`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
  the deliberate make/env parity, not a typo.

Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.

Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 17:47:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3d50baf359 docs(agents/iteration/role): sharpen in-container verify rule + full-matrix flow
Three iteration-rule sharpenings, all surfaced by repeated foot-guns in
the web-app-zammad role-build loop:

- Promote "validate the fix in-container before redeploying" from SHOULD
  to MUST. Adds an explicit reproducible-there carve-out (container
  crashes at startup, misconfigured env var, image-side bug, in-container
  script failure) so the rule is unambiguous on when it applies; the
  only exception is failures that genuinely depend on multi-container or
  post-deploy Ansible state.
- Playwright-specific addendum: when the failing unit is a Playwright
  spec, "in-container validation" means running
  `make compose-playwright role=<role>` against the live stack — not
  just `curl` against the underlying API. The spec runs in a browser
  context with env loading, locator strategies, and cookie state that
  `curl` does not exercise.
- New "Full-matrix iteration flow" subsection under "Matrix variants"
  documenting the variant-pinned `mode=update` → per-remaining-variant
  `mode=reinstall full_cycle=true` → final unpinned full-matrix
  `mode=reinstall full_cycle=true` recipe. Closes the gap where the
  existing "Matrix variants" rules only described focused single-variant
  debug and full-matrix initiation, but not the cycle that keeps full
  cross-variant coverage as the iteration target.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 00:09:27 +02:00
kevinveenbirkenbachandClaude Opus 4.7 490f0919ab refactor(make): replace INFINITO_CMD with cmd parameter
INFINITO_CMD is a per-invocation argument, not a config value — drop it
from default.env, the passthrough STATIC_KEYS, and the dotenv-generator
test, and rename to lowercase `cmd` in Makefile targets, exec scripts,
internal callers, and user-facing docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 05:31:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c36929090d refactor(make): apply comment-block schema across Makefile targets
Splits mixed-content target descriptions into a single description line
plus `Note:` follow-ups, and dresses the targets that carry real
parameters (`compose-deploy`, `compose-exec`, `compose-inner-run`) with
explicit `Usage:` / `Example:` / `Param <name>:` lines. Adds a
`variant=<idx>` Make var alias to `compose-deploy` so multi-variant
pinning works without dropping to `INFINITO_VARIANT=` env prefixes.

Removes redundant inline `compose-deploy` / `compose-inner-run` param
enumerations from the docs and replaces them with
`make help target=<name>` pointers, so the help banner stays the SPOT.

Also fixes SC2016 in `scripts/make/help.sh` (literal backticks in the
help banner) with a scoped suppression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 91fc422f19 feat(make): add compose-playwright target + schema-aware help
Adds `make compose-playwright role=<role> [pw="..."] [keep=true]` as the
canonical inner-loop entry point for rerunning a role-local Playwright
spec against the live stack — replaces the raw
`scripts/tests/e2e/rerun-spec.sh` invocation across docs.

Defines a small comment-block schema for Makefile targets (description
line on top, followed by `Usage:` / `Example:` / `Note:` / `Param <name>:`
markers). Lint enforces it; `make help` shows the description per target
in the short list and `make help target=<name>` prints the full block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 605f29a2f8 refactor(make): rename deploy to compose-deploy
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.

Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 356e3366be feat(make): consolidate local deploy into single make deploy router
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.

Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.

Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.

Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 20:22:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a77a570409 refactor(env): consolidate INFINITO_* namespace and centralize path/image SPOTs
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:

* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
  env var → defaulted to infinito.localhost, which never resolved
  because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
  INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
  (env-builder, role default, run_one.yml docker exec, playwright.config.js,
  lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
  the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
  compose forwards it as build-arg, volume mount path, and container env;
  every Python CLI + shell helper + CI workflow now reads it strictly
  (os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
  /opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
  INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
  utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
  container) through compose.yml's environment block.

Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 02:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9a578952c3 feat(env): SPOT env-pipeline + INFINITO_* namespace
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.

Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
  BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
  PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
  WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
  promoted IMAGE_TAG from GHA-only to general passthrough

compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions

Tooling:
- scripts/install/all.sh owns the stamp-file install logic
  (Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
  8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env

Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
  filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator

Docs:
- variables.md is now a pointer; per-variable docs live next to
  the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:40:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f12fbe83e docs(agents/iteration): mandate fresh-purged on first contact with a new VARIANT
Codify the rule that switching the live stack onto a previously-untouched
VARIANT must go through deploy-fresh-purged-apps, never a reuse target.
Reuse re-pins the stack onto stale volumes, DB rows and network aliases
from the previously-pinned variant and produces split-brain app state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 09:44:24 +02:00