The stamp_dir already gets 0777 so any uid can land the stamp on a
shared host, but `touch ${STAMP}` requires write access to the file
itself. When the first run was sandboxed (uid 65534) and the second is
the real user, the second touch failed and pre-commit's `make test`
re-triggered a full `make install`. Matching the dir's permissions on
the file closes that gap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Match the workflow's actual purpose (building OCI images) more
literally than the Docker mascot, freeing 🐳 for compose-specific
contexts (see test-deploy-compose.yml).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous DOMAIN_PRIMARY fallback was reading a non-existent env var
(DOMAIN_PRIMARY is an Ansible variable, not an env var) and defaulted to
infinito.localhost. In swarm-CI that produced /etc/hosts entries for the
wrong domain, so dashboard's "Wait for dashboard branding assets" task
got NXDOMAIN for file.infinito.example and exhausted its 60 retries.
INFINITO_DOMAIN is the SPOT (default.env:107, exported via
scripts/meta/env/load.sh which 02_start_swarm_nodes.sh sources). Read it
directly and let KeyError surface a misconfigured caller — matches
scripts/tests/dns.sh's "${INFINITO_DOMAIN:?...}" convention.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The role's block-level `delegate_to: STACK_HOST` made five `stat:` tasks
run on the manager, which has no repo checkout — `playbook_dir`-based
paths always returned `exists=false` and the subsequent `fail:` tasks
aborted runs like web-app-baserow / web-app-akaunting with
"Missing Playwright env template". Override delegate_to to localhost on
each repo-path stat so the existence check runs where the files
actually live; CA-cert stats keep their STACK_HOST scope because those
files belong to the host.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The unconditional run_once_<provider> assertion fires on every swarm
deploy where sys-service-loader's loop iteration order does not strictly
honour bucket order (a known race when include_tasks runs across mixed
host scopes). MODE_ASSERT keeps the check available in debug runs to
surface the underlying loader race, while IS_STACK_HOST scopes it to
the host where the provider would actually have been preloaded. In
production the immediately following Initialize task lets svc-db-<x>
self-heal via its own run_once gate, so the assertion is no longer
load-bearing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The earlier idempotent guard latched onto a hidden duplicate of biber's handle (submitButton.or(handle).first() resolved to a hidden vcard-short-addr div), so the follow test still timed out on the async pass.
Reset the relationship before following instead: a single GET to /contact/unfollow?auto=1&url=<profile> unfollows biber when already followed and is a harmless redirect otherwise. The follow confirmation form is then always fresh, so every run exercises the real follow action (submit, 302 to /contact/<id>, biber's handle).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The async deploy pass re-runs the suite against the same persistent instance, where biber is already a contact. /contact/follow then renders an "already added this contact" page without #dfrn-request-submit-button, so the test timed out 60s waiting for the submit button (only the sync pass passed).
Wait for the submit button or biber's identity handle, submit the confirmation only on the fresh-follow path, and assert biber's handle as the shared post-follow proof across both passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both flavors integrate with Infinito's central postgres (no more MDAD
bundled DB inside the DiD container). The connection vars live in a
single set of MATRIX_POSTGRES_* constants in vars/main.yml and are
referenced by both flavors' templates so the source-of-truth is
explicit and identical.
- vars/main.yml: MATRIX_MDAD_POSTGRES_* renamed to MATRIX_POSTGRES_*
(drop MDAD prefix — these are flavor-agnostic) + add
MATRIX_POSTGRES_DATABASE.
- templates/flavor/ansible/vars.yml.j2 (was mdad-vars.yml.j2):
postgres_enabled: false; wire postgres_connection_*/synapse
database_* to MATRIX_POSTGRES_* + matrix_synapse_container_network:
host so the inner Synapse inherits matrix-mdad's outer networks and
resolves `database` / `openldap` by hostname. valkey_enabled: false
(Synapse single-process; central redis path not used here).
- templates/flavor/ansible/hosts.j2 (was mdad-hosts.j2): renamed, no
content change. mdad- prefix dropped since the file is already
under flavor/ansible/.
- templates/flavor/ansible/services.yml.j2: matrix-mdad joins the
central `postgres` network (conditional on DB shared) and
`openldap` network (conditional on MATRIX_LDAP_ENABLED).
- templates/flavor/compose/synapse/homeserver.yaml.j2 and the six
templates/flavor/compose/mautrix/*.config.yml.j2 templates: switch
inline `lookup('database', application_id, 'host'/'username'/...)`
calls to MATRIX_POSTGRES_* (same constants the ansible flavor uses).
- tasks/flavor/ansible/: renumber 01-05 cleanly
- 03_database.yml (was 02a_database.yml) — sys-svc-rdbms include
- 04_run_playbook.yml (was 03_run_playbook.yml)
- 05_proxy_wiring.yml (was 04_proxy_wiring.yml)
- tasks/flavor/ansible/main.yml + 02_render_inventory.yml + README.md:
update references to the renamed files.
- tasks/flavor/compose/02_create-and-seed-database.yml: switch host/port
lookups to MATRIX_POSTGRES_* too.
Tests: 186/186 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Failing CI run 27010233874 (job 79718652049) and the local DiD test
both timed out at `probe Synapse client API on host loopback` with
`Connection reset by peer` for the full 140s retry window. Workflow
rootcause inspection of the matrix-mdad container surface identified
two compounding issues:
1. `matrix_mdad_systemd:/etc/systemd/system` named volume shadowed the
image-baked `multi-user.target.wants/matrix-mdad-bootstrap.service`
enable symlink. First-boot's empty named volume masks the image
content so systemd never sees the unit as enabled — bootstrap never
runs, MDAD never installs, Synapse never starts. Smoking gun.
2. 60×2s = 140s probe window was an order of magnitude too short for a
cold MDAD bootstrap (galaxy install + image pulls + ansible-playbook
setup-all on a fresh container). Even with the unit enabled the
probe would race the bootstrap.
Changes:
- services.yml.j2: drop `matrix_mdad_systemd:/etc/systemd/system`
mount; MDAD-written units live in the container's writable layer
alongside the image's enable symlinks. Add `matrix_mdad_state:
/var/lib/matrix-mdad` so the bootstrap marker survives container
restarts. Normalize federation port publish to `DOCKER_BIND_HOST`.
- compose.yml.j2: rename declared volume `matrix_mdad_systemd` ->
`matrix_mdad_state`. Inside the `matrix_flavor == 'ansible'` branch
only — compose flavor untouched.
- bootstrap.service: drop `network-online.target` (never fires in a
container without systemd-networkd-wait-online). Add
`ExecStartPre=/usr/bin/docker info` for fail-fast if the inner
dockerd never came up.
- bootstrap.sh: log markers + per-step timing so the systemd journal
inside matrix-mdad shows a clear timeline.
- 04_proxy_wiring.yml: 60×2s -> 180×5s = 900s; on probe failure under
MODE_DEBUG, run a new diagnostics.sh script (container logs +
systemctl --failed + journalctl + docker.service status) so the next
iteration has actionable output instead of a silent retry stack.
make test: 186/186 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.
Rename the workflow file and align all references:
- ci-orchestrator.yml job key + uses: path
- artifact names: inventory-server-* -> inventory-compose-*,
playwright-server-* -> playwright-compose-*
- scripts/github/inventory/zip_compose_for_upload.sh output path
- 3 docs files
INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add 08_verify_reachable.sh between wait_converge (07) and seed_content
(09, renumbered from 08): TCP-connect via the manager's local
routing-mesh entry confirms incoming traffic reaches a live task. For
multi-replica services, additionally probe each node that hosts a
Running task to verify the mesh works locally on every task-bearing
node.
Skips services without published ports (tag-only roles, internal-only
services). Logs replica count and port list for debugging.
Renumber 08-12 -> 09-13 to keep the sequence contiguous.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add the compose_replicas lookup plugin as the single source of truth for
swarm replica calculation. Default emits `replicas: len(groups[application_id])`
(topology-driven); override term forces an explicit value. In compose mode
the lookup returns the empty string because docker-compose ignores
deploy.replicas.
Refactor deploy.yml.j2 to use the lookup. The previous service_replicas
default in defaults/main.yml is no longer needed and is dropped.
Add a lint test that walks every roles/*/templates/compose*.yml.j2 and
verifies each service entry under the top-level services: declares
replica handling via one of three valid paths:
1. Include sys-svc-container/templates/base.yml.j2 (full inheritance).
2. Include sys-svc-container/templates/deploy.yml.j2 directly (escape
hatch for services that own their restart/logging/env_file and
therefore can't use base.yml.j2 without duplicate-key conflicts).
3. Call lookup('compose_replicas', ...) explicitly for handcrafted
deploy: blocks.
Fix the 7 web-app compose templates flagged by the new lint (bluesky,
dashboard, magento, matrix, moodle, nextcloud, opentalk) by including
deploy.yml.j2 directly and setting service_name as a SPOT before each
service entry, so the service name in the yaml key and the resource_filter
lookups in deploy.yml.j2 both flow from one variable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Default service_replicas to (groups[application_id] | length) instead of
hard-coded 1. Single-host inventories still get replicas=1; 3-host swarm
inventories now get replicas=3 for every stateless service that renders
the deploy: block. Stateful services (DBs etc.) are unaffected because
deploy.yml.j2 gates the block on DEPLOYMENT_MODE == 'swarm' and stateful
services run as plain compose.
Simplify deploy.yml.j2 (the | default fallback in the template is now
redundant) and drop the explicit replicas:3 + SHOPWARE_WORKER_REPLICAS
plumbing from web-app-shopware - the new default covers it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The IS_STACK_HOST gate in main.yml never blocked workers because
test-e2e-playwright does not set its own application_id, so DEPLOYMENT_MODE
fell through to 'compose' and IS_STACK_HOST evaluated true on every host.
Workers then ran 02_run_one.yml and crashed on the CA cert check because
the cert only exists on the manager.
Replace with block: + delegate_to: STACK_HOST + run_once: true so the
orchestration runs once on the primary manager regardless of which hosts
are in the play.
Also rename SWARM_PRIMARY_MANAGER -> STACK_HOST repo-wide to match the
existing IS_STACK_HOST symmetry and drop the swarm-specific terminology
(the variable falls back to inventory_hostname in compose mode).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The swarm deploy step ran `python3 -m cli.administration.deploy.swarm`
directly in a fresh per-step shell, so the .env file written by `make
setup` was never exported into the deploy process. `lookup('env',
'INFINITO_DOMAIN')` then rendered to '' and canonical_domains_map
crashed during the OIDC merge on every web-app job.
Mirrors the server-CI pattern where scripts/tests/deploy/ci/all.sh
sources the loader before invoking ansible.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
svc-cache-registry -> svc-registry-cache
svc-docker-registry -> svc-registry-docker
svc-docker-swarm -> svc-swarm
svc-docker-swarm-manager -> svc-swarm-manager
categories.yml: drop svc.docker, add svc.registry. Deliberately no svc.swarm
category so get_entity_name('svc-swarm') stays 'swarm' instead of collapsing
to '' (which would break discover_role_services).
Entity keys updated to match the new svc.registry prefix:
services.cache-registry.* -> services.cache.*
services.registry.* -> services.docker.*
01_constructor: replace svc-docker bootstrap-loop entry with explicit
include_role of svc-swarm + bootstrap loop over [svc-registry, svc-storage].
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
_render_one_template.yml gated render on IS_STACK_HOST and relied on
the manager being in the same play to actually execute it. nextcloud's
flow hits this in a worker-only play (uid/group fixups + config
render together), so manager was not in the play, render skipped on
both workers, and the very next task ([swarm] Slurp rendered ... from
manager) failed with File not found because the file was never
written. Affected job in CI run 27006276488: web-app-nextcloud.
Switch render to delegate_to SWARM_PRIMARY_MANAGER + run_once so it
fires once on the manager regardless of which hosts are in the play.
The pattern mirrors the slurp + copy-to-peers tasks below; in compose
mode SWARM_PRIMARY_MANAGER resolves to inventory_hostname so the
single-host case is unchanged.
01_install.yml waits on `compose ps -q snipe-it` from
/opt/compose/snipe-it/, which only exists on the host that owns
the stack. Under swarm-mode delegation the role plays on every host
so workers chdir-ENOENT-failed the wait task after 30 retries.
02_ldap.yml and 03_admin.yml have the same shape (compose exec from
the same chdir). Affected job in CI run 27006276488: web-app-snipe-it.
Move the three trailing `import_tasks` from main.yml into a new
01a_manager_ops.yml and include it from main.yml only when
IS_STACK_HOST is true. The 3 children switch from import to include
since the parent is now dynamic; this matches the espocrm /
bigbluebutton / 27-role-sweep pattern. sys-stk-full stays ungated
so worker prereqs still bootstrap.
The disc-space cleanup script unconditionally ran
systemctl start cln-bkps.<version>.infinito.nexus.service || exit 2
and the same for cln-docker. On swarm worker hosts cln-bkps is not
installed (backups are a manager-side concern), so once the cleanup
threshold tripped, cln-disc-space exited 2 with
"Failed to start cln-bkps.<...>.service: Unit ... not found." and
the sys-service-loader Fail-after-diagnostics handler aborted the
deploy. Affected job in CI run 27006276488: web-app-kix
(swarm-wrk-01, swarm-wrk-02 both at 93% disc use during the run).
Gate both systemctl start calls with `systemctl cat <unit>` so the
chain silently skips a sub-cleanup whose unit is absent on the host
rather than treating absence as a fatal. The compose-mode case is
unchanged: both units exist on the single host so both calls fire
as before.
Set BBB Facts referenced compose_repository_path which sys-stk-full only
materialises on the host that owns the stack. Under swarm-mode
delegation the role plays on every host so workers hit
"compose_repository_path is undefined" before any container exec ran.
Affected job in CI run 27006276488: web-app-bigbluebutton.
The same applies to every task that follows on the BBB side (compose
override render, env-file symlink, compose.yml include, the
flush/postgres/admin trio) - all touch host fs paths or run container
exec against the BBB stack. Move them into a new 01a_manager_ops.yml
and have 01_core include it only when IS_STACK_HOST is true. The
sys-stk-full include, the optional seed include and the
websocket_upgrade.conf gate stay where they are; the run-once flag
stays ungated at the end so workers still flip the marker.
inventories/development/swarm.yml carries only swarm-mode-specific
overrides and omits cross-cutting cred seeds (OIDC.CLIENT.SECRET,
biber, administrator password). Until now the swarm CI provision step
did not pass --vars-file, so default.yml was never merged into the
generated host_vars; the first deploy that preloaded web-app-mailu as
a shared email service then hit object of type 'dict' has no attribute
'SECRET' when sys-svc-compose rendered roles/web-app-mailu/templates/
env.j2. Affected jobs in CI run 27006276488: web-app-chess,
web-app-espocrm, web-app-funkwhale, web-app-prometheus.
Match the server-CI pattern (scripts/tests/deploy/local/reset/
inventory.sh and apps/initialize/all.sh both pass --vars-file
${INFINITO_INVENTORY_VARS_FILE}). 05_provision_inventory.sh now does
the same: the env loader sourced via _context.sh already exports
INFINITO_INVENTORY_VARS_FILE in the script shell, so the path
resolves through the SPOT.
write_extras.py re-reads default.yml users dict and patches the
runtime SSH key onto admin authorized_keys before emitting the
extras file. Without this, the deploy-time -e @swarm-nfs-extras.yml
would REPLACE the host_vars users entry (ansible later-wins on
top-level dicts) and wipe biber + the admin password the
--vars-file just baked in.
02_run_one.yml asserts that the self-signed CA cert exists at
/etc/infinito.nexus/ca/root-ca.crt and bails out hard when it does
not, but the cert is only laid down on the host that owns the stack.
Under swarm-mode delegation the role plays on every host so workers
hit the CA-presence guard and the matrix job fails before any
Playwright spec runs. Affected jobs in CI run 27006276488:
web-app-bridgy-fed, web-app-fider, web-app-fusiondirectory,
web-app-openwebui, web-app-postmarks, web-app-yourls.
Move every task from main.yml (dirs, setup include, run loop,
failure aggregation) except the leading debug into a new
tasks/01a_orchestrate.yml, then include it from main.yml only when
IS_STACK_HOST is true. Workers print the role list and stop; the
stack-host runs Playwright as before. compose mode keeps
IS_STACK_HOST true so single-host runs are unchanged.
Deploy OpenResty NGINX Base Configuration (template render against
lookup('nginx','files.configuration')) and the surrounding cleanup,
reset, directory, load_app and assert chain all touch host-side
nginx config that lives only on the host running OpenResty. Under
swarm-mode delegation the role plays on every host, so workers tried
to render nginx.conf against an undefined applications variable
(group_vars/all/17_resource.yml:15 RESOURCE_ACTIVE_DOCKER_CONTAINER_COUNT)
and aborted with applications is undefined. Affected jobs in CI
run 27006276488: web-app-mastodon, web-app-peertube, web-opt-rdr-domains.
Move the cleanup, reset, directory, openresty-load, template, flush
and assert chain into a new tasks/01a_manager_setup.yml and have
01_core.yml include it only when IS_STACK_HOST is true. The run-once
flag stays ungated at the end so workers still flip the role marker.
- bootstrap.sh / bootstrap.service: systemd doesn't inherit Dockerfile
ENV PATH, so the oneshot couldn't find ansible-galaxy. Export PATH in
the script and set Environment=PATH in the unit explicitly.
- 04_proxy_wiring.yml: rewrite the inner-docker admin-register exec as
shell scalar with `# nocheck: raw-docker` (the inner docker CLI talks
to matrix-mdad's DiD daemon; not host docker).
- vars/main.yml: drop now-unused MATRIX_MDAD_POSTGRES_* + MATRIX_REDIS_*
vars (DiD uses bundled postgres + valkey).
make test: 186/186 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Across roles/web-app-*, the typical body shape is:
- include_role: sys-stk-full (or sys-svc-compose) cluster prereqs
- <host-fs writes, container exec, compose exec> stack-host only
Under swarm-mode delegation the second group only makes sense on the
host that holds the compose stack (manager), but the role plays on
every host in groups[<role>] so workers abort on missing paths.
Two workflow sweeps (gate-manager-ops + gate-manager-ops-01core, 58
agents total, discover/plan/adversarial-verify per role) plus three
manual fixes surfaced 27 roles to refactor. Each role keeps
sys-stk-full ungated so worker-side prereqs (sys-svc-container, NFS
client) still bootstrap; the trailing manager-only block either moves
into a new tasks/02_manager_ops.yml included with
when: IS_STACK_HOST | bool (SUB_FILE pattern) or gets the gate
inlined on the one or two trailing tasks (INLINE pattern). In
compose mode IS_STACK_HOST stays true so single-host deploys are
unchanged.
Manual catches the workflow agents missed:
- web-app-fediwall: utils/once/flag.yml lives AT THE END of 01_core,
not at the start; preserved verbatim outside the gate so workers
still flip the run_once_<role> fact.
- web-app-xwiki: a pre-existing block(...) wraps sys-stk-full plus
_flush_config.yml; gated only the _flush_config.yml include so
sys-stk-full keeps running on workers.
- web-app-dashboard: 02_javascript.yml (creates /opt/compose/dashboard/
JS dirs) ran BEFORE sys-stk-full and was ungated; added the gate.
Roles touched (INLINE): akaunting, jenkins, magento, mobilizon
Roles touched (SUB_FILE): baserow, dashboard, decidim, erpnext,
fediwall, friendica, gitea, jitsi, keycloak, listmonk, littlejs,
mailu, matomo, moodle, odoo, openproject, opentalk, pixelfed,
shopware, suitecrm, taiga, xwiki, zammad
Add a Comments section to CLAUDE.md that overrides the agent's
default training nudge toward descriptive comments. Only three
sanctioned categories survive: exception trip-wires, parameter
documentation, and linter / type-checker directives. Everything else
deletes before the edit lands.
The old name suggested "is the swarm manager up?" but the variable
actually answers "is this host the one that holds the compose stack?".
It is true on the single host in compose mode and on every swarm
manager in swarm mode. The SWARM_ prefix is also misleading in compose
mode where no swarm exists.
Replace every reference across group_vars and the gating call sites
(sys-svc-compose, web-app-espocrm, web-app-bigbluebutton). Definition
itself moves to group_vars/all/18_swarm.yml line 6; the body is
unchanged so the truth value at runtime stays identical for both modes.
Everything past `sys-stk-full` in this role wrote to /opt/compose/espocrm/
or talked to the running container via `container exec` / `compose
exec`. Both only exist on the host that actually runs the espocrm
compose stack. In compose mode that is the single host; in swarm mode
only the swarm manager. Without gating, the file deploys aborted on
workers because /opt/compose/espocrm/volumes did not exist there, and
the follow-up container execs would have aborted just past that.
Move the post-stk-full task list into `02_manager_ops.yml` and include
it from main.yml with `when: SWARM_MANAGER_ACTIVE | bool`. The fact is
true on every host in compose mode (DEPLOYMENT_MODE \!= 'swarm') so
single-host deploys are unchanged. In swarm mode only the manager
runs the include, which keeps file deploys, the patch_config include,
the upgrade command and the seed runner all on the host that holds
the stack. Template rendering for compose.yml.j2 / env.j2 is already
gated by SWARM_MANAGER_ACTIVE in sys-svc-compose/tasks/04_files.yml.
The `configure websocket_upgrade.conf` task writes into the openresty
maps directory. The directory only exists on the host that runs
openresty. In compose mode that is every host (single-host), in swarm
mode only the swarm manager. Without a gate the task fires on workers
where /etc/nginx/conf.d/maps does not exist and the deploy aborts.
Gate the copy with `when: SWARM_MANAGER_ACTIVE | bool` so it runs on
every host in compose mode (the fact stays true when DEPLOYMENT_MODE
\!= 'swarm') and only on the manager in swarm mode. The
`restart openresty` handler is queued by the same task, so workers do
not try to restart a service they do not run.
`lookup('command_path', 'ldapsm')` runs on the ansible controller via
shutil.which against a hardcoded /usr/local/sbin:/usr/local/bin:...
search path. The preceding `sys-pip-install` task installs ldapsm on
the TARGET, not the controller. In compose mode controller == target,
so the lookup works; in swarm mode the controller (the act runner) has
no ldapsm and the lookup raises before the command ever ships.
Drop the lookup and let ansible.builtin.command resolve bare `ldapsm`
through the target's PATH. sys-pip-install with global_install=true and
--break-system-packages drops the console script in /usr/local/bin
which is in the default PATH on every supported distro.
`svc-docker-registry` declares `run_after: svc-docker-swarm` (matched
by a `services.swarm: enabled+shared` block per the coherence rule
enforced in tests/integration/roles/meta/services/run_after). The
combined CombinedResolver follows run_after edges unconditionally, so
under universal / server CI svc-docker-swarm lands in the inventory
include set even though DEPLOYMENT_MODE resolves to 'compose' and the
`svc-docker-swarm-manager` group is empty — the first assertion in
01_core blows up before the registry ever gets to come up as a regular
compose service.
Gate the cluster bootstrap behind DEPLOYMENT_MODE == 'swarm'. The seven
bootstrap tasks move into `01a_bootstrap_cluster.yml` so 01_core stays
within the block-with-when ≤3-tasks parse-cost rule and the
run-once-flag-first lint stays happy. In compose mode the include is
not parsed at all, in swarm mode the bootstrap runs unchanged.
matrix-mdad is now a long-running compose service with systemd PID 1
and its own bundled docker daemon. MDAD writes its systemd unit files
and runs matrix-synapse/coturn/element/bridges/jitsi/element-call
entirely INSIDE that container via the inner docker daemon. Nothing
leaks to the host beyond standard /opt/compose/matrix/ role state.
Container layout:
- /sbin/init (systemd) as PID 1
- bundled docker.io daemon (inner DinD)
- bootstrap.service oneshot: runs `ansible-playbook setup.yml` once on
first boot; marker at /var/lib/matrix-mdad/bootstrap.done
- Named volumes: matrix_mdad_docker (inner /var/lib/docker),
matrix_mdad_systemd (/etc/systemd/system), matrix_mdad_matrix (/matrix)
- /sys/fs/cgroup:rw + cgroup: host for systemd-in-container
- Outer compose publishes synapse/element/federation ports to host loopback
mdad-vars.yml.j2:
- postgres_enabled: true (MDAD bundled postgres inside DiD)
- valkey_enabled: true (bundled redis)
- exim_relay_enabled: true when no outbound Mailu
- host_bind_port: 0.0.0.0 inside container so outer compose port-publish
can forward to host loopback
Dropped (no longer needed):
- 02a_database.yml — MDAD provisions postgres inside the DiD container
- 00_cleanup_mdad_residue.yml — no host residue to clean
- ensure rshared mount task — no host bind-propagation needed
- container wait matrix-mdad-runner — service is long-running not one-shot
- matrix_synapse_container_additional_networks_custom — DiD has its own
internal docker network
- matrix_synapse_container_extra_arguments CA mount — bundled CA inside
- coturn_container_additional_volumes — anonymous volumes live in inner
docker daemon, outer hlth-volumes can't see them
04_proxy_wiring.yml:
- admin user creation now uses two-hop exec: outer `container exec
matrix-mdad` -> inner `docker exec matrix-synapse register_new_matrix_user`
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two distinct breakages that let the workflow fail before ever reaching
the ansible deploy:
* `python3 -m utils.tests.swarm.*` (and friends) ran with the
actions-setup interpreter that has no project deps, so the imports
of `plugins.filter.canonical_domains_map` blew up on `from ansible.errors`.
The swarm preamble scripts now source `scripts/meta/env/load.sh` so
PATH picks up the venv `make install` populated, and the workflow itself
uses `install-force` (under `act --bind` the build/install.stamp is the
host's and would short-circuit the in-container install) plus pushes
`\${VENV}/bin` onto GITHUB_PATH for downstream steps.
* `ansible-playbook -e @file` replaces top-level dicts rather than
deep-merging them. The runtime-only extras file was dropping
`storage.nfs.{export_base,version}`, `storage.backend`, and
`swarm.{registry.port,network}` that the static
`inventories/development/swarm.yml` ships. Re-state those keys
in write_extras.py so the merge ends up with everything the roles need.
The script's three lines were redundant: \`pip install -e .\` and the
ansible-galaxy collection step are already covered by \`make install\`
(now landed in the swarm matrix preamble), and \`PYTHON=python3 bash
scripts/setup.sh\` is exactly what \`make setup\` invokes. Replace the
step with a direct \`make setup\` call.
Renumber the remaining scripts to keep the sequence lückenlos:
06_provision_inventory.sh -> 05_provision_inventory.sh
07_debug_inventory.sh -> 06_debug_inventory.sh
08_wait_converge.sh -> 07_wait_converge.sh
09_seed_content.sh -> 08_seed_content.sh
10_drain_worker.sh -> 09_drain_worker.sh
11_assert_state.sh -> 10_assert_state.sh
12_collect_diagnostics.sh -> 11_collect_diagnostics.sh
13_cleanup.sh -> 12_cleanup.sh
Workflow YAML updated to the new names. No script content changed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add the canonical "Set up Python" + "Install project (host venv)" steps
to the swarm matrix job's preamble. The discover job already runs
`make install`; the matrix job did not, so when
03_start_swarm_nodes.sh shells out to
`python3 -m utils.tests.swarm.write_hosts_entries`, the import of
plugins.filter.canonical_domains_map failed with
ModuleNotFoundError: No module named 'ansible'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* tests/unit/cli/administration/deploy/swarm/test_closure.py — drop
the `parents[6]` + `nocheck: project-root-import` combo. The
file-path loader still bypasses the dedicated CLI's heavy package
__init__, but the path now comes from the canonical
`from utils import PROJECT_ROOT` so the project-root-import lint
passes without a per-line opt-out.
* utils/tests/swarm/write_extras.py — restructure the
`Path.read_text()` call onto a single line so the
`# nocheck: cache-read` annotation lands on the call site itself.
Autoformat had previously line-broken the chained `.strip()`
expression and pushed the marker onto a different line, making
the suppression invisible to the lint scanner.
* tests/integration/infrastructure/compose/test_compose_volumes_call_required.py —
the `test_no_template_uses_deprecated_pipe_form` walker used a raw
`roles_dir.rglob("templates/**/*.j2")`, which trips
`test_no_raw_project_walk` in the lint suite. Switch to the cached
`iter_project_files()` helper with a roles-prefix + `/templates/`
filter; behaviour is unchanged but the walk goes through the
shared SPOT.
* roles/svc-cache-registry/meta/services.yml — annotate the role
with `# nocheck: required-by-coverage`. svc-cache-registry is
intentionally opt-in via `default_placement: manager` plus the
swarm CLI closure, NOT via a downstream `required_by` chain, so
the lint's non-invokable-without-required_by check would otherwise
reject it.
`make autoformat` passes. `make test` is now green across lint /
test-external / test-integration / test-lint / test-unit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI failures from run 26987751848:
* `cache-read` (3 hits): annotate the `Path.read_text()` calls in
utils/tests/swarm/{write_extras,extend_inventory}.py — every one
reads a file the same script just wrote, so the cache helper would
serve stale content. Per `# nocheck: cache-read` carve-out.
* `project-root-import`: annotate the `parents[6]` in the swarm
closure test loader. The file-path loader bypasses the package
__init__ to avoid pulling in the heavy dedicated CLI imports.
* No-unittest-tests: tests in test_closure.py inherited only via
`_BaseClosureCase` (which itself extends `unittest.TestCase`), but
the AST lint walks bases literally and does not follow indirect
inheritance. Switch each `Test*` class to multi-inherit
`(_BaseClosureCase, unittest.TestCase)` so the discovery check
recognises them as runnable.
* `hardcoded-dns-resolver`: the per-block head suppression in
inventories/development/swarm.yml was insufficient; the lint scans
per-line and needs the marker on the actual `dns: "1.1.1.1"` line.
Verified locally: 16 of the previously-failing tests now pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`-e @<file>` replaces sibling dicts wholesale; the extras file only
spelled networks.internet.dns, so the loopback ip4/ip6 from
group_vars/all/08_networks.yml were dropped at deploy time and
templates dereferencing networks.internet.ip4 (e.g. web-app-bigbluebutton
env.j2 EXTERNAL_IPv4) failed with "object of type 'dict' has no
attribute 'ip4'".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New ``cli.administration.deploy.swarm`` package wraps the strict
``cli.administration.deploy.dedicated`` runner with a layered closure
so swarm deploys never require operators (or test scripts) to
enumerate every infra role:
effective --id = seed (operator --id, else inventory_role_groups)
∪ dep-walk(seed via applications_if_group_and_all_deps)
∪ default-placement safety net
The dep-walker is the same SPOT
(``utils.roles.applications.in_group_deps.applications_if_group_and_all_deps``)
the runtime constructor consumes via ``lookup('applications_current_play')``,
so the CLI-side closure and the in-play role set stay in sync.
The dedicated CLI keeps strict explicit-only semantics: ``main`` was
split into ``parse_args`` + ``run_from_args`` so the swarm wrapper
can share parser, mode wiring, validator and runner verbatim and
only mutate ``args.id``.
Meta-side: ``svc-db-postgres`` gains ``default_placement: manager``
(semantic symmetry with mariadb). ``default_placement`` is now a
PLACEMENT hint only ("if deployed, run on the manager"); deploy
decisions come from the dep-walk. Apps that declare ``mariadb:``
pull svc-db-mariadb; apps that declare ``postgres:`` pull
svc-db-postgres; apps that declare neither get no central DB.
Shared utils extracted so the four consumers stay aligned:
* ``utils.inventory.groups.inventory_has_group``: YAML + INI static
inventory group probe (extracted from ValidDeployId).
* ``utils.roles.meta_lookup.{get_role_default_placement,
iter_roles_with_default_placement}``: single source of truth for
``default_placement`` discovery (lookup plugin is now a thin wrapper).
Test pipeline (``scripts/tests/deploy/swarm/*``):
* Heredoc-Python shells converted to proper modules under
``utils.tests.swarm.*`` and invoked via ``python -m``:
- extend_inventory.py (was 08_extend_inventory.sh)
- write_extras.py (was 09_write_extras.sh dynamic half)
- derive_includes.py (replaces 06's hardcoded includes array)
- write_hosts_entries.py (replaces 02's hardcoded /etc/hosts heredoc;
derives every canonical + www-redirect domain from APP_ID via
canonical_domains_map + generate_all_domains, so the test
works for any app in the matrix without per-app maintenance)
* 01_create_network.sh deleted (single docker command inlined in workflow).
* 11_deploy.sh deleted (swarm CLI now called directly in workflow).
* 09_write_extras.sh: static half moved to inventories/development/swarm.yml.
* Remaining shell scripts renumbered 01-13 without gaps.
* 08_wait_converge.sh: case block on DB_DEP replaced with single
conditional (container name == DB_DEP).
* 06_provision_inventory.sh: hardcoded includes array replaced with
`mapfile -t includes < <(python -m utils.tests.swarm.derive_includes)`.
* Workflow YAML step references updated to new script names + module
invocations.
New tests (39 covering the closure surface):
* tests/unit/cli/administration/deploy/swarm/test_closure.py (14)
* tests/unit/utils/inventory/test_groups.py (11)
* tests/unit/utils/roles/test_meta_lookup.py (+3 for default_placement)
* tests/unit/utils/roles/validation/test_deploy_id.py (unchanged
8 tests still pass after refactor onto the shared util).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single SPOT lookup `plugins/lookup/compose_volumes.py` absorbs the
applications-lookup pipe step and auto-wires `DEPLOYMENT_MODE` plus
the `storage` group var from the templating context. Callers only
pass what genuinely varies per service (`application_id` and an
optional `extra_volumes` dict).
Behavior change (bug fix): the filter was always invoked with the
default `deployment_mode='compose'` because no caller passed it
explicitly, which silently skipped the swarm-NFS driver-opts
injection path. The lookup reads `DEPLOYMENT_MODE` from globals,
so swarm-NFS volumes now render the `driver: local` / `driver_opts:
type: nfs` block correctly. Callers can still override via
`deployment_mode=` / `storage=` kwargs when a template needs
non-default rendering.
Other coordinated changes:
- 81 template call sites migrated from pipe form to lookup form
(single-line and multi-line shapes).
- `FilterModule` registration in plugins/filter/compose_volumes.py
removed so the deprecated pipe form fails fast at render time.
The underlying `compose_volumes()` function stays exported and is
imported by the new lookup plus the existing 21-test filter unit
test module.
- 10 new lookup unit tests pin the contract (term shape, kwargs
pass-through, auto-wiring, override precedence, error cases).
- Integration test `test_compose_volumes_call_required.py` updated:
asserts the new lookup form is present in every `compose.yml.j2`
and adds a second test forbidding the deprecated pipe form
anywhere under `roles/**/templates/`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.
Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
`build:` with a harmless warning when `image:` is pre-pushed (via
the existing handlers/swarm_registry_push.py flow in
roles/sys-svc-compose). The actual swarm-rejected key is
`pull_policy:` ("Additional property pull_policy is not allowed"),
and 10 templates were silently broken because they declared inline
`pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
\!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
the gating logic is identical (emit-key-or-nothing).
Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
removed.
Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
`container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
deleted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Added while researching docker swarm vs compose mode-incompatibility
during the compose_restart / mode-incompatible-keys audit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror compose_container_name pattern: plugins/lookup/compose_restart.py
emits `restart: <policy>` only in compose mode. In swarm the key is
silently ignored (replaced by deploy.restart_policy) and produces
"Ignoring unsupported options" warnings at docker stack deploy time.
Sweep 19 raw restart: sites across 10 templates to the no-arg form
`{{ lookup('compose_restart') }}` (the lookup falls back to
DOCKER_RESTART_POLICY from ansible-vars).
base.yml.j2 keeps the explicit-arg form
`{{ lookup('compose_restart', docker_restart_policy | default(DOCKER_RESTART_POLICY)) }}`
so the 5 templates that override the policy via Jinja-scope
`{% set docker_restart_policy = ... %}` (erpnext, zammad, shopware,
matomo, openproject) retain their override path. Jinja-scope bindings
are visible only in the rendering template, never in the Ansible
variable context that the lookup sees, so the caller must resolve them.
Generalize the container-name lint into a data-driven multi-rule
scanner at tests/lint/ansible/jinja/test_mode_incompatible_keys.py
covering both container-name and restart-key. Adding a new rule is one
ModeRule row sharing the file-walk / suppression / line-grouping
scaffolding.
Also gate roles/svc-db-redis/templates/service.yml.j2's
resource.yml.j2 include with DEPLOYMENT_MODE \!= 'swarm'. Mirrors
base.yml.j2's existing pattern; prevents cpus / mem_limit /
mem_reservation / pids_limit from leaking into swarm output where
deploy.resources.* in deploy.yml.j2 already handles them.
11 unit tests pin the compose_restart lookup contract. Both rules
registered in docs/contributing/actions/testing/suppression.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
