Add a `skip: [compose, swarm]` field on a role's meta/services.yml primary entity
so an app can opt out of the test-deploy matrix per deploy mode, replacing the
hardcoded exclude regex in apps.sh (the "TODO: Remove exclude condition"). The
filter threads through both discovery paths: the CI workflows pass
INFINITO_DEPLOY_MODE -> apps.sh -> `cli meta roles applications type
--skip-mode`, and roundtrip.sh honours the same get_role_skip SPOT per app/mode.
Apply skip: [compose, swarm] to svc-storage-nfs-server and svc-storage-nfs-client
(NFS storage is swarm-only and set up via the cluster inventory, not the matrix)
and web-opt-rdr-www (the previously regex-excluded app).
Covered by get_role_skip and skip_mode-filter unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every test-deploy-swarm matrix job died at 01_start_nfs.sh with
`INFINITO_SWARM_TEST_LABEL: unbound variable`: the label (consumed by
01_start_nfs.sh, 02_start_swarm_nodes.sh and clean_all.sh) was added to
default.env but never sourced by 00_topology.sh nor exported to the job env.
Broaden the 00_topology.sh source filter to `^INFINITO_SWARM_[A-Z0-9_]+=` (every
swarm key, future-proof) and export INFINITO_SWARM_TEST_LABEL from the
"Load swarm topology SPOT into job env" step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.
Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.
Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hoist the swarm-test node, network and NFS-export-base names out of the shell
harness (00_topology.sh) into default.env (INFINITO_SWARM_*), so the shell
harness and the Python harness (extend_inventory, write_extras) read one source
instead of each hardcoding the suffixes. 00_topology.sh, extend_inventory.py,
write_extras.py and 01/09/11 now read the SPOT; passthrough.py exports the keys
into the generated .env. 00_topology.sh reads default.env directly (not .env)
because the workflow sources it before make dotenv runs.
Rename topology.sh -> 00_topology.sh (sorts ahead of 01_/02_, the order it is
sourced in) and update all 10 references.
Add scripts/tests/deploy/swarm/clean_all.sh + make act-swarm-clean to reclaim
wedged act-swarm state between runs, label-scoped via INFINITO_SWARM_TEST_LABEL
so it never touches unrelated containers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The swarm workflow ran 3 DinD nodes plus builds on the default ~4 GB runner swap while compose/workstation/universal already enlarge it. Add the same free-disk-space (swap-storage) and enlarge_swap.sh steps, guarded with \!env.ACT, to prevent host OOM during builds. Orthogonal to swarm mem_reservation scheduling, which counts physical RAM not swap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every swarm-test cluster is named so parallel clusters get distinct container and network names. topology.sh and extend_inventory require SWARM_NAME; act-swarm-zombie defaults it to the app id (name= overrides), the CI workflow derives it from the matrix app, and act-swarm-down/shell/playwright require name=. The bare swarm-mgr-01 names are gone (node refs now carry the cluster prefix).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Thread a single `variant` selector (CSV) end-to-end so CI can split a
role's matrix deploy across runners.
- variant_select.py (SPOT, shared by init + deploy): one --variant taking
a CSV (2 pins one round, 0,1,2 is a runner bundle, empty = full matrix);
replaces the old separate --variant/--variants.
- planner.filter_plan_to_variants filters the plan to a round subset.
- CI discovery (utils/github/variant_bundles.py) splits roles with more
than INFINITO_VARIANT_BUNDLE_SIZE (default 3) variants into bundles of
consecutive indices, one matrix runner each (5 -> 0,1,2 + 3,4), threaded
via the variant env. test-deploy-{local,universal,workstation,server}.yml
switch to matrix.include with a comma-free variant_slug for artifact/job
names (GitHub expressions have no replace()).
- runtime_budget honours INFINITO_VARIANT_TIME_OVERFLOW (cut|fail) when a
runner cannot finish its slice within INFINITO_MAX_RUNTIME.
Unit tests + docs (variants.md, deploy.md) included.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The swarm CI runs the Ansible controller as the non-root runner. During the
deploy, svc-storage-nfs-server mounts the cluster-shared NFS state on the
controller, where the token store lives root:root 0600. Controller-side lookups
(service / service_registry via get_merged_applications) read it, and Ansible
lookups cannot `become`, so the non-root controller hits [Errno 13] Permission
denied. The state export is no_root_squash, so a root controller reads it
exactly like the swarm nodes do.
Run the whole orchestrator as root: provision/extend/deploy all read the store
once the NFS mount is active (round 2+, since purge does not unmount between
rounds), so wrapping only the deploy was insufficient. Re-pin PATH/HOME because
sudo's secure_path drops the venv and ~/.ansible/collections. No-op when the
controller is already root (local act, production).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The swarm e2e now iterates one provision+deploy+e2e round per app variant,
mirroring the compose matrix deploy, so the keycloak TOTP env-gate is no
longer needed: the variant's services.keycloak.realm.totp_enabled=false is
baked into host_vars per round and resolves at deploy time in swarm too.
New cli/administration/deploy/swarm/matrix.py drives the rounds by reusing
the compose planner (plan_dev_inventory_matrix, _resolve_variant_payloads,
_bake_overrides): per round it provisions with the baked --vars overlay plus
--app-variants, extends the swarm topology, deploys (e2e in-deploy), waits
for convergence, and purges prior-round stacks between rounds via the new
purge_stacks.sh. The workflow collapses its per-step provision/deploy/verify
into one orchestrator step; chaos (seed/drain/assert) still runs once.
- cli/administration/deploy/swarm/matrix.py: NEW round orchestrator
- scripts/tests/deploy/swarm/purge_stacks.sh: NEW per-round swarm purge
- scripts/tests/deploy/swarm/05_provision_inventory.sh: per-round inv dir plus
conditional --vars / --app-variants (standalone path unchanged)
- .github/workflows/test-deploy-swarm.yml: single orchestrator step, timeout 360
- default.env, utils/env/handlers/passthrough.py: register INFINITO_VARS_PAYLOAD
plus INFINITO_APP_VARIANTS passthrough keys
- roles/web-app-keycloak/vars/main.yml: drop the ENVIRONMENT env-gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under `act --bind` the working tree is bind-mounted into the runner, so running actions/checkout would clobber the live tree being iterated on. Guard the checkout step with `if: ${{ \!env.ACT }}` so real GitHub runs still check out while local act runs reuse the bind mount.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds `act-swarm-zombie app=<id>` which runs the swarm matrix-app test and leaves
the DinD cluster + lab network alive afterwards (INFINITO_KEEP_SWARM_NODES=true
forwarded via the new ACT_ENV passthrough in workflow.sh) so failures can be
post-mortem inspected with `make act-swarm-exec node=… cmd=…`,
`make act-swarm-shell node=…`, and finally released with `make act-swarm-down`.
Consolidates the swarm-test topology (manager / worker / nfs-server names, lab
network, NFS export base) into scripts/tests/deploy/swarm/topology.sh as the
single source of truth: 13_cleanup.sh and shell_node.sh source it directly,
the GitHub Actions workflow ingests it into $GITHUB_ENV via a setup step,
replacing the duplicate env: block entries.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Resolves CI swarm rc=69 ("msmtp: needs a password"): controller-side
Ansible lookups (lookup('email').password etc.) couldn't see the
tokens.yml that Mailu writes target-side, because the controller has
its own filesystem in swarm mode.
Mount NFS-shared /var/lib/infinito on every swarm host AND on the
controller during deploy (in-play delegate_to: localhost task at end
of svc-storage-nfs-server role). State subdir exported separately
with no_root_squash + wildcard CIDRs so root-owned writes succeed
and the controller can mount from outside the svc-swarm IP set.
Single-writer guarantee on FILE_TOKENS via delegate_to: STACK_HOST +
run_once on sys-token-store/{01_core,write}.yml. Backup roles
(svc-bkp-container-2-local, sys-ctl-cln-bkps, sys-ctl-cln-faild-bkps,
user-backup/03_permissions_folders, three direct callers of
04_seed-database-to-backup.yml) gated on IS_STACK_HOST to prevent
N-host write races on the shared NFS backup tree.
Fail-fast assertion in svc-storage-nfs-client + 05_controller_mount
when pre-existing local /var/lib/infinito would be hidden by mount-
over. Operator must migrate or wipe before re-deploying.
SPOTs: SWARM_LAB_NETWORK in workflow env block, storage.nfs.state_subdir
in group_vars/all/15_storage.yml, DIR_VAR_LIB unchanged. UPPER_CASE
constants in NFS roles. Cleanup script umounts controller view before
container teardown to prevent stale-mount leak between matrix jobs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Loading default.yml via `-e @"${INFINITO_INVENTORY_VARS_FILE}"` (commit
a3c75bde7) made the dev vars available on workers but at extra-vars
precedence (level 22), which overrode the `set_fact` merge in
tasks/stages/01_constructor.yml that reconciles defaults_oidc with the
inventory-supplied OIDC. Workers and mgr-01 ended up with OIDC =
{CLIENT: {SECRET: ...}} only, so the templates that resolved
OIDC.CLIENT.LOGOUT_URL (sys-front-inj-logout etc.) aborted with "dict
has no attribute 'LOGOUT_URL'".
Mirror the provisioned host_vars/${MGR}.yml to host_vars/${WRK1}.yml and
host_vars/${WRK2}.yml at the end of 05_provision_inventory.sh instead.
Same default.yml source via the existing INFINITO_INVENTORY_VARS_FILE
SPOT, same content on every swarm host, but at host_vars precedence
(level 9) so the role-level merge wins on access. Removes the
now-redundant `-e` line from the workflow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous commit hardcoded "inventories/development/default.yml" in
the swarm deploy step, which the inventory-vars-file SPOT drift guard
(tests/unit/cli/administration/deploy/development/test_inventory_vars_file_spot.py)
forbids outside the whitelist. Read the path from the env var that
scripts/meta/env/load.sh exports so both the workflow and the deploy
chain consume the same SPOT.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The swarm CI deploy invoked cli.administration.deploy.swarm with only
swarm.yml and the runtime extras as -e files. Default-inventory vars
(OIDC.CLIENT.SECRET, users, design, ...) were applied only to
host_vars/<MGR>.yml via the provision step's --vars-file argument, so
they reached mgr-01 but not the workers. Workers without OIDC.CLIENT
defaults rendered roles/web-app-keycloak/templates/import/clients/default.json.j2
into "object of type 'dict' has no attribute 'SECRET'" at the realm
import step (CI run 27044129926).
Load default.yml as inventory-wide extra-vars before swarm.yml, so the
two are merged uniformly across mgr-01 and workers. swarm.yml still
overrides any keys it sets (extra-vars merge left-to-right, last wins).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Match the workflow's actual purpose (building OCI images) more
literally than the Docker mascot, freeing 🐳 for compose-specific
contexts (see test-deploy-compose.yml).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.
Rename the workflow file and align all references:
- ci-orchestrator.yml job key + uses: path
- artifact names: inventory-server-* -> inventory-compose-*,
playwright-server-* -> playwright-compose-*
- scripts/github/inventory/zip_compose_for_upload.sh output path
- 3 docs files
INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add 08_verify_reachable.sh between wait_converge (07) and seed_content
(09, renumbered from 08): TCP-connect via the manager's local
routing-mesh entry confirms incoming traffic reaches a live task. For
multi-replica services, additionally probe each node that hosts a
Running task to verify the mesh works locally on every task-bearing
node.
Skips services without published ports (tag-only roles, internal-only
services). Logs replica count and port list for debugging.
Renumber 08-12 -> 09-13 to keep the sequence contiguous.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The swarm deploy step ran `python3 -m cli.administration.deploy.swarm`
directly in a fresh per-step shell, so the .env file written by `make
setup` was never exported into the deploy process. `lookup('env',
'INFINITO_DOMAIN')` then rendered to '' and canonical_domains_map
crashed during the OIDC merge on every web-app job.
Mirrors the server-CI pattern where scripts/tests/deploy/ci/all.sh
sources the loader before invoking ansible.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two distinct breakages that let the workflow fail before ever reaching
the ansible deploy:
* `python3 -m utils.tests.swarm.*` (and friends) ran with the
actions-setup interpreter that has no project deps, so the imports
of `plugins.filter.canonical_domains_map` blew up on `from ansible.errors`.
The swarm preamble scripts now source `scripts/meta/env/load.sh` so
PATH picks up the venv `make install` populated, and the workflow itself
uses `install-force` (under `act --bind` the build/install.stamp is the
host's and would short-circuit the in-container install) plus pushes
`\${VENV}/bin` onto GITHUB_PATH for downstream steps.
* `ansible-playbook -e @file` replaces top-level dicts rather than
deep-merging them. The runtime-only extras file was dropping
`storage.nfs.{export_base,version}`, `storage.backend`, and
`swarm.{registry.port,network}` that the static
`inventories/development/swarm.yml` ships. Re-state those keys
in write_extras.py so the merge ends up with everything the roles need.
The script's three lines were redundant: \`pip install -e .\` and the
ansible-galaxy collection step are already covered by \`make install\`
(now landed in the swarm matrix preamble), and \`PYTHON=python3 bash
scripts/setup.sh\` is exactly what \`make setup\` invokes. Replace the
step with a direct \`make setup\` call.
Renumber the remaining scripts to keep the sequence lückenlos:
06_provision_inventory.sh -> 05_provision_inventory.sh
07_debug_inventory.sh -> 06_debug_inventory.sh
08_wait_converge.sh -> 07_wait_converge.sh
09_seed_content.sh -> 08_seed_content.sh
10_drain_worker.sh -> 09_drain_worker.sh
11_assert_state.sh -> 10_assert_state.sh
12_collect_diagnostics.sh -> 11_collect_diagnostics.sh
13_cleanup.sh -> 12_cleanup.sh
Workflow YAML updated to the new names. No script content changed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
