04_wait.yml resolves the sidekiq container via container_address, which requires services.sidekiq.name; it was unset (unlike mastodon/streaming), so the wait aborted with "services.sidekiq.name not set". Add name: mastodon-sidekiq (and mastodon-migrate) matching the already-built container names.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm config objects mount root:root; a 0440 mode is unreadable by a non-root container app user and crashloops the service (jenkins JCasC AccessDeniedException, exit 5). docker compose never hits this since there the file is a host bind-mount inheriting host perms.
New lint test_config_object_world_readable flags type:config modes lacking o+r (type:secret exempt, per-line # nocheck escape). Widen the 22 offending 0440 configs across 18 roles to 0444: adds read only, cannot break a reader, and fixes latent non-root cases (e.g. synapse, pixelfed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker stack deploy interpolates $ in the compose file and rejects a bare $@ ("invalid interpolation format"); docker compose tolerates it. $$@ normalises to $@ in both, so the entrypoint is compose- and swarm-compatible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.
- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
lint enforces whole-file on any edited file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A network pull with no time bound can hang indefinitely: run10 sat 6h
in 'docker compose build --pull' for bbb with no new output, and the
streaming monitor stayed silent because a hang emits no log lines.
Add timeout: 3600 to both network-pull handlers (compose pull, Build
compose) so a stuck pull self-aborts at 60min and the deploy fails
loudly instead of hanging forever.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/var/lib/infinito on the swarm manager is an NFS mount inside the container's own mount namespace. docker cp reads the daemon-side rootfs and sees the empty mountpoint, so it copied zero files and the uploaded Playwright artifact was always empty. Stream the reports out through docker exec (in-namespace), the same access pattern the swarm diagnostics already use.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the swarm CID/redis fix that make quality flagged:
- Extract the site-bootstrap sequence into 02b_create_site.yml (renamed from
02b_restart_frappe.yml; now new-site + restart + re-resolve + wait) and include
it with when: rc \!= 0, so the oversized when-block disappears and the sub-file
is not parsed at all when the site already exists.
- The backend entrypoint now execs the image's real /usr/local/bin/entrypoint.sh
after the config-ready gate, so the sites/assets symlink that entrypoint creates
is preserved instead of relying on a sibling replica to recreate it.
- Drop the now-dead ERPNEXT_BACKEND_CONTAINER_ADDRESS var, shorten two over-long
OIDC restart task names, and remove stale narration comments.
Also suppress the INFINITO_MAIN_REMOTE/BRANCH env lints in the git pre-push helper:
they are maintainer tooling knobs, not deployment keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The manager-op tasks ran 'container exec {{ ERPNEXT_BACKEND_CONTAINER_ADDRESS }}'
where the address resolves a LOCAL container id. Under swarm the backend task can
run on another node (and 'service update --force' reschedules it), so the id came
back empty, collapsing the command and failing 'apply API-bot' with a bogus 'No
such file'. Resolve the backend's hosting node + local id via resolve_host_cid,
delegate_to that node, and re-resolve after each forced restart. The four apply_*.py
tasks can no longer use a host '< file' redirect once delegated, so feed the script
through 'stdin:' instead; the OIDC on-change restart splits into compose vs swarm.
docker stack deploy strips depends_on conditions, so queue/websocket/scheduler
booted before the one-shot configurator wrote redis_* into common_site_config.json
and wedged on frappe's 127.0.0.1 defaults. Gate each worker (and the backend
entrypoint) to block until redis_socketio appears, closing the race in swarm and
staying a no-op in compose where the config already exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/git/assert_main_merged.sh, which fetches the upstream main
tip and fails unless it is fully merged into HEAD. Wired in two places:
- pre-push: a new make-test-main-merged pre-commit hook (pre-push stage)
runs 'make test-main-merged' before every push.
- pull request: a new assert-main-merged job checks out the real PR head
(not the merge ref) and asserts the base branch is merged into it, so a
PR whose branch lags main goes red.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.
Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
quality runs autoformat then the full test suite in one shot as a
pre-commit gate.
autoformat-restage runs autoformat and re-stages files it rewrote that
were already staged, but only when the working tree held no unstaged
tracked changes beforehand -- so a partially-staged hunk is never swept
into the index. Logic lives in scripts/git/autoformat_restage.sh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The embedded typesense sidecar crashed under 'docker stack deploy' with 'API key
is not specified': its command set no --api-key and relied on an env_file value,
but stack deploy ignores env_file and funkwhale's env.j2 never set
TYPESENSE_API_KEY anyway. Declare a funkwhale-owned api_key credential and pass it
as --api-key on the command line (honored by both compose and stack deploy). In
compose the sidecar stays profiles-gated off, so this only changes the swarm path.
Apply the same fix to the standalone svc-db-typesense engine, which has the
identical latent gap (its key already lives in vars, so --api-key just mirrors the
env value and CLI precedence makes it a no-op in compose).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the swarm-DinD test the ansible controller (ansible_connection: docker) cannot read the STACK_HOST-rendered xwiki.properties, so compose_volumes' content hash falls back to the source path and the config name is constant. xwiki deploys twice (bootstrap with the OIDC block off, final with it on after the extension installs), so the config content changes but the name does not, and the final docker stack deploy fails to update the immutable config with 'only updates to Labels are allowed'.
Honor an explicit name: on config/secret entries in compose_volumes (volumes already did), and give xwiki's properties config a name that rotates with the OIDC phase, so the final deploy creates a new config instead of updating in place. Backward-compatible: no other role sets name: on a config. make test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Checkmk's oauth2-proxy header-auth GUI exposes no logout control the shared
Playwright personas can reach: the classic GUI is iframe-based, its top-frame
toolbar is bare <div>/<a> (no <header>/<nav> anchor), and the native logout
hides behind the 'User' menu. So the biber and administrator personas failed at
'app -> universal logout' with 'no in-app logout control reachable'.
Add a 'javascript' service (gated on services.sso.enabled) so the platform
inlines templates/javascript.js.j2 into every Checkmk response. The script marks
a logout control with data-injected-logout (which the persona helper checks
first): it appends a themed icon to a header/nav group when one exists, else
falls back to a fixed-position top-right button on the top-frame <body>. The
control points at OIDC.CLIENT.LOGOUT_URL so a click clears both the Checkmk and
the Keycloak/oauth2-proxy session. Inline injection is admitted by Checkmk's own
CSP ('unsafe-inline'). Verified in-container before redeploy, then full-matrix
reinstall: V1 6 passed (incl. biber+admin logout), V2/V3 2 passed/4 skipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compose_volumes emitted a top-level configs:/secrets: entry for every declared
config/secret, but container_volumes already gates each per-service reference on
its mount's when:. So a config whose only mount was gated off (e.g. LDAP
disabled) became an orphan: a top-level object no service references, whose file:
source points at a file the role never renders (same flag). Under docker stack
deploy that source is read at deploy time and the deploy fails; under compose it
was a harmless unused entry.
Gate the top-level emission on the same mount_when_passes container_volumes uses,
so a top-level config/secret exists iff a service actually references it. To share
the predicate, promote the private _mount_when_passes from container_volumes into
utils/roles/applications/mounts and import it in both filters; the bare except
moves with it (inline noqa), so container_volumes no longer needs its BLE001
per-file-ignore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The embedded minio/seaweedfs object store emitted only compose-native resource
keys (cpus/mem_limit) and no replica pin, so under 'docker stack deploy' it got
the default replica count and raced its own state across nodes. Gate the
compose-native resource block to compose and add a swarm deploy: block pinned to
replicas=1 (deploy.yml.j2 self-gates to swarm, so compose emits nothing new).
Give the embedded seaweedfs an explicit name to avoid colliding with a
standalone web-app-seaweedfs sharing the cluster.
The kcadm provisioning tasks ran 'container exec' against a local container id
that only exists on the node hosting Keycloak; in swarm that node is not
necessarily the one running the play. Resolve the Keycloak host node and its
local container id, delegate_to that node, and split the controller restart into
'container restart' (compose) vs 'container service update --force' (swarm).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
moodle mail failed two ways in a swarm cluster. (1) moodle's build_context
pulled the FULL sys-svc-mail-msmtp role early, which rendered /root/.msmtprc on
the manager before Mailu had created the no-reply token and tripped the role's
run-once flag, so the manager kept an empty-password msmtprc and the health
check failed 'authentication method PLAIN needs a password'. Pull only
tasks_from: ca_bundle so the manager re-renders on the normal post-token pass.
(2) msmtprc forced host=127.0.0.1 for any in-container host, but the NFS server
sits outside the swarm mesh and has no local Mailu, so it hit 'connect to
127.0.0.1:465 refused'. Narrow the 127.0.0.1 branch to compose or a real
svc-swarm-node AND to when Mailu is actually deployed (email.external); else use
email.host, and map the NFS node's test DNS to the manager IP. Also import the
shared self-signed Root CA for the relay's TLS trust.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The role flushed handlers to start web/streaming/sidekiq and only then exec'd
'rails db:migrate' into the running web container, so on a fresh DB the app
booted against an unmigrated schema (swarm converge gate timed out) and the
compose migration was OOM-killed sharing the 2g web cgroup with the live puma
server. Add a dedicated one-shot migrate service (own 2g container, db:prepare,
replicas 1) that web/streaming/sidekiq depend on (service_completed_successfully
in compose; swarm converges via crash-restart + the run-once gate). Refactor the
four near-identical service blocks into a mastodon_service macro.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The stock image CMD exits on an empty DB ("Run --install."), so the server
crash-looped on first deploy while the role exec'd 'yes | ./listmonk --install'
into that same crashing container, which was SIGKILLed (rc 137) and never
converged. Adopt listmonk's own recommended container command (idempotent
install, upgrade, then serve; the super admin is seeded from LISTMONK_ADMIN_USER
/ _PASSWORD), and replace the exec dance with a DB-reachability probe plus a
wait for the users table to be seeded. Pin latest to the current stable v6.2.0
so the CLI/schema contract cannot drift. Same flow in compose and swarm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gateway nginx hardcoded compose container_names (taiga-front/back/...) that
do not exist as DNS names in swarm, so it crash-looped on [emerg] host not found
in upstream. Point the upstreams at the service keys (which resolve in both
compose and swarm), render the role gateway config unconditionally (gating only
the OIDC location block), pin the protected service name to its SPOT var, and
add the back service to the async depends_on so it waits for the shared image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The version pin sat on the app-level penpot entry, which container_image never
reads, so the frontend/backend/exporter deployed :latest. Since Penpot 2.15 the
backend SSRF-blocks outbound HTTP to hosts resolving to private IPs, so the OIDC
token exchange to the privately-resolving Keycloak failed with unable-to-auth
and login never completed. Pin the three image-bearing services to 2.16.2 (the
release CI actually exercised) and set PENPOT_SSRF_ALLOWED_HOSTS to the Keycloak
host (https? strips either scheme). env.j2 stays shared by compose and swarm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mailu_admin uniquely runs on public DNSSEC resolvers, so internal service names
are pinned to overlay VIPs via extra_hosts. That pinning assumed a central redis
stack; with a local redis sidecar MAILU_SWARM_REDIS_HOST is the bare name
'redis', and pinning it to an absent central VIP shadowed stack DNS, so admin
blackholed its first redis dial and was SIGKILLed unhealthy in a loop (blocking
chess and mini-qr). Gate the redis VIP inspect, the fact, and the extra_hosts
line on the redis engine not being local. The shared-redis path is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pdsadmin binary is mounted from the volumes dir, but 02_pds.yml ran only
after the sys-stk-front-proxy include whose flush already deploys the stack, so
the deploy aborted on the missing source. Move the PDS staging ahead of the
front-proxy block. Also revert pdsadmin from a docker config back to a bind
mount: it is a ~9MB Go binary and swarm configs are capped at ~1MB, so type:
config fails stack deploy. The pds service is manager-pinned, so the
manager-staged bind source is always present on the node that runs it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The KIX backend runs a FRESH_INSTALL schema bootstrap that outlives the default
150x2s converge budget on loaded CI runners (db reaches 1/1 while the backend
stays 0/1 Starting), so the swarm converge gate timed out. Make the gate's
retries overridable (default unchanged at 150) and raise it to 600 for kix.
Swarm-only gate; compose is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The build copied only package.json and ran npm install, discarding the
upstream package-lock.json (present at the repo root on master); npm then
fresh-resolved @types/node to a major that the app's pinned TypeScript cannot
parse, failing 'npm run build' with TS1110. Copy the lockfile and use npm ci
so the pinned dependency tree is honoured. Same build path in compose and swarm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared curl healthcheck probes with curl, which the upstream
libretranslate image (python slim) does not ship, so every replica was killed
unhealthy (exit 137) even after gunicorn was listening and the stack never
converged. Use the image's bundled scripts/healthcheck.py, which passes during
the two-phase argos-model cold boot and probes /health afterwards. Renders
identically for compose and swarm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the forced OpenResty rollout handler fires twice in one play, the second
'service update --force' can submit a stale object version and the daemon
rejects it with 'update out of sequence', failing the deploy. Retry until the
CLI reports success; any other stderr fails on the first attempt. Swarm-only
and stack-host gated, so compose is byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The re-injection loop stat-checked each web-svc consumer's compose instance dir,
which proxy-only vhosts (web-svc-html) never have, so they were skipped and kept
a placeholder matomo token, failing the Playwright tracker assertion. Stat the
per-consumer body_filter Lua that sys-front-inj-all renders (the identical
lookup used to write it), so every deployed+injected consumer is re-rendered
with the real token. compose-dir-exists implies lua-exists for all six
candidates, so no consumer that was re-injected before is now skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With trim_blocks on, the two-space indented block tags leaked their leading
whitespace into the rendered output, prepending spaces to the next line and
producing a mis-indented key inside the env: mapping that broke Psych YAML
parsing on a fresh wordpress+discourse deploy. Move the block tags to column 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The env file was written in YAML colon syntax; docker compose's lenient parser
tolerated it but docker stack deploy's kvfile reader rejects the colon lines and
aborts the swarm deploy. Convert to canonical KEY=value (dotenv_quote on the two
passwords) and add a repo-wide lint (env-file-dotenv-syntax) so no env.j2
regresses to YAML mapping syntax.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
