The LiveKit JWT service checks each OpenID token over federation
against the user's homeserver. Server discovery found no delegation:
https://<server_name>/.well-known/matrix/server hit the redirect vhost
of web-opt-rdr-domains, which answered 301 to the dashboard and then
404, so the lookup fell back to <server_name>:8448. In compose OpenResty
runs in the host network and answers there; in swarm it publishes 80
and 443 only, the dial was refused and Element Call's token exchange
returned 401 M_UNAUTHORIZED (main run 40017166046, Nextcloud#12).
The role also wrote the delegation to `<well-known>/matrixserver`,
because the file path joined the directory and `server` without a
separator.
The file now lands at `<well-known>/matrix/server`, and the redirect
vhost serves `/.well-known/matrix/` from the shared well-known
directory, so discovery delegates to the Synapse domain on 443 in both
modes. Both flavors share tasks/utils/well_known.yml, which also
includes web-opt-rdr-domains for the server name when no play-wide
redirect claims it, so the delegation has a vhost there too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An earlier commit on this branch turned the three off because the ansible flavor ran
them without a published port, proxy entry, domain or certificate. They
now stay on and are wired end to end.
Element Call: MDAD's LiveKit server and JWT service get their own
entities in meta/services.yml with resource limits, local ports and a
public media port (TCP and UDP). The runner container publishes them
through container_ports, and the Synapse vhost proxies /livekit-server/
(websocket) and /livekit-jwt-service/ to them. The JWT service checks
each OpenID token against the user's homeserver over federation, and
federation was not reachable in this flavor: Synapse served it only on
a plain-HTTP listener that the runner published directly on 8448.
Synapse now serves the client and federation resources on one listener,
OpenResty serves the Synapse vhost on 443 and as default_server on the
federation port, the direct 8448 publish is gone, and the role writes
/.well-known/matrix/server as the compose flavor does. With a
self-signed CA the JWT container mounts the CA bundle as Synapse does.
Jitsi: a new services.jitsi entity follows the shared-service contract.
It is enabled in the ansible flavor off tor nodes. When web-app-jitsi is
in the inventory, Element points at that shared Jitsi and MDAD's own
Jitsi stays off. Otherwise MDAD runs its own Jitsi on jitsi.<domain>
with a front-proxy vhost, its web port published to the runner and its
videobridge on a public UDP port. The entity lists the jitsi domain, so
the web health probe skips it whenever Matrix does not serve it. The
jitsi_*_container_network overrides were dropped: MDAD reads none of
them, Jitsi always ran on its own network. Variant 0 covers the shared
case, variant 1 the internal one. On an onion-primary deploy both call
stacks stay off, so Element never advertises a clearnet call surface.
MDAD's Prosody keeps an account's first password, so after a credential
rotation jicofo failed SASL with the new one. The role now derives the
jicofo and videobridge passwords from the macaroon secret and recreates
both accounts on every deploy.
Heisenbridge stays on by default. It runs in the nested host network,
where MDAD's container-name URLs do not resolve, so its homeserver URL
and the appservice URL in its registration point at 127.0.0.1.
A bootstrap that a previous deploy left failed (for example after a
Docker Hub rate limit) is now reset and restarted, so a mode=update
deploy converges instead of failing on the stale unit state.
Playwright proves each path: the RTC spec fetches a LiveKit token from
the JWT service and opens the SFU signalling websocket, the Jitsi spec
checks Element's preferred domain and joins a conference (signing in
through Keycloak first when the shared Jitsi sits behind SSO), and the
Heisenbridge spec opens a DM with the bot and waits for its answer to
"help".
Closes#1165
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
b3ce43a596 set no_log on every task of two included files, which
test_include_uniform_apply_keyword rejects: a keyword every included task
repeats belongs in the include's apply block. The Cloudflare development
mode tasks and the ChatGPT bridge token tasks now receive no_log through
apply at their include sites. The matrix include had set no_log on the
include task itself, which never reaches the included tasks.
Two loops over MATRIX_BRIDGES were unmasked: the bridge folder creation
and the readiness gate. A loop result prints the whole item, and every item
carries the bridge's database_password, so both printed it at -v whenever
MASK_CREDENTIALS_IN_LOGS is on. The credential lint cannot see this because
the loop variable's name carries no secret marker.
Verified: test_include_uniform_apply_keyword, the credential masking lint,
the task lints and the matrix unit tests pass, and ansible-lint is clean on
the five files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
At -vvv Ansible prints each module argument and result, so a password in
a command line, a request body, a header or a fact lands in the deploy
log. Nothing enforced masking outside the MCP task files, and no check
looked at parents at all.
A new lint, test_credential_tasks_are_masked, finds tasks whose printed
parts (module arguments, environment, loop) reference an identifier ending
in password, secret, token, api_key or private_key, or
secrets.credentials. A task passes when it or an enclosing block sets
no_log, when every file that pulls it in does so through include_tasks
with apply.no_log or import_tasks with no_log (transitively), when the
secret only reaches a parameter the module masks itself, or when the
module masks its values. no_log on an include_tasks task itself does not
count, because it does not reach the included tasks. The rule can be
suppressed per task with `# nocheck: credential-masking`, now listed in
the suppression catalog, and the lint tests its parent and suppression
logic on synthetic cases.
It found 39 unmasked tasks in 19 roles, all real: join tokens, client
secrets, API tokens in Cloudflare headers, database passwords in
command lines, the ChatGPT bot login body and more. Each now sets
no_log from MASK_CREDENTIALS_IN_LOGS. The sys-dns-hetzner-rdns alias
hetzner_no_log, which only restated that switch, is gone.
Not covered yet: template and copy tasks whose rendered file carries a
secret print it as a diff under --diff.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
304144ce67 probed each bridge's healthcheck at /_matrix/mau/ready. mautrix
sets Ready only after Synapse has pinged the appservice (connector.go
starts the listener, then ensureConnection, then AS.Ready = true), and a
swarm task with a healthcheck stays "starting" and receives no VIP traffic
until it is healthy. The ping could never arrive: Synapse logged 150
ConnectionRefused per bridge, each bridge exited FTL 13, and the stack
never converged (run 34615516009, Matrix#2, job 103327034630).
The healthcheck now probes /_matrix/mau/live, which answers once the
listener is up, before the ping. A deploy task after the Synapse wait asks
every bridge for /_matrix/mau/ready from inside Synapse, over the same
name Synapse pings, so a bridge that never becomes ready still fails the
deploy without gating swarm routing.
The ChatGPT bot is rendered only once its access token exists. Its account
and token are created by stack_host_provisioning after the first stack
deploy, so an earlier bot task failed its login with M_FORBIDDEN until the
converge wait called it stuck. The re-render after provisioning adds it
with a valid token.
Verified in a compose run of variant 2: the readiness task got 200 from
meta, signal, slack and whatsapp on the first attempt. Tests pin the live
probe and the token guard and fail against the previous template.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every mautrix bridge generated its own registration with -g on start,
with fresh random tokens. Synapse had to read that file from a shared
directory, be restarted afterwards, and on swarm the services had to be
removed first so the registrations could be wiped (93735fd533). The
tokens changed on every redeploy and the deploy depended on a wait for
files a container writes.
Each bridge now has as_token and hs_token declared in meta/secrets.yml
and a fixed appservice_port in its addon. The bridge config reads the
tokens through lookup('config'), and registration.yaml.j2 renders the
registration from that config in the upstream -g format, so both sides
are known before anything starts and stay identical across redeploys.
The files reach the containers as secrets: matrix_bridge_mounts offers a
stack view (top-level definitions, content-hash named by compose_volumes),
a synapse view (one registration per bridge under /registrations, mode
0444) and a bridge view (the config at /config/config.yaml). The bridges
run as the synapse uid with an explicit --config and --no-update, keep
their scratch /data on a 16m tmpfs so hlth-volumes finds no anonymous
volume, and depend on a healthy synapse. Their healthcheck is
/_matrix/mau/ready, which the bridge only reports after whoami with the
as_token and an appservice ping carrying the hs_token, so a healthy
bridge proves the token pair.
The compose tasks lose the compose down, the swarm service rm block, the
registration wait, the permission fixes and both synapse restarts; this
reverts the workaround of 93735fd533.
Verified on the local compose stack: four bridges ready, synapse pings
answered, registration and config sha256 unchanged across a redeploy,
test-bridge-roster and 12 matrix Playwright specs green. A token
rotation (new pair in both files, compose up --force-recreate as the
handler runs it) came back healthy; a mismatched pair left the bridge
unhealthy. The swarm-shaped stack file passes docker stack config with
the 0444 secret modes, the 16 MiB tmpfs and content-hash secret names.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every deploy of the compose flavour wipes /opt/compose/matrix/mautrix/ and
re-renders each bridge config with placeholder tokens, so the bridges generate
new registrations on their next start and synapse has to load them. Compose
gets that ordering from the `compose down` right before the wipe. Swarm had no
counterpart: the bridge services kept running on the deleted bind-mount
directories and never regenerated anything, while synapse was restarted by the
flush and found no registration.
Job 102705652096 of run 34421944328 (swarm, variant 2) hits it on the
credential-rotation redeploy, which changes homeserver.yaml, a swarm config,
and therefore restarts synapse every time:
FileNotFoundError: [Errno 2] No such file or directory: '/data/mautrix-meta/registration.yaml'
stuck: matrix_synapse
From 05:44:56 dockerd reported "current working directory is outside of
container mount namespace root" for the running slack and signal bridges, and
mautrix-meta, started at 03:50:50, only initialised again at 05:53:11, when its
healthcheck finally replaced it; stack_ready.sh had given up at its 180s grace
long before.
Before the wipe, swarm mode now removes the synapse and bridge services a
previous deploy left running, once from the stack host, so the flush's
`docker stack deploy` creates them together, exactly as on a first deploy
(which converged in about 32s in the same job). Removing only the bridges would
not be enough: a synapse that is not recreated keeps the old tokens, and the
new bridges would be refused before the synapse restart at the end of the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moving the statement into files/sql left it on stdin as a quoted scalar, and
the password-quoting guard strips stdin sub-blocks only when they are block
scalars. A quoted one falls through and is scanned as if it were shell, where
any expression naming a password must carry the quote filter - and the file is
called realign_user_password.sql.
realign_synapse_password.yml:24: In shell tasks, password expressions must
include '| quote': {{ lookup('file', 'sql/realign_user_password.sql') }}
Quoting it would paste literal quotes into the SQL, and renaming the file
would only hide the token from the pattern. A folded scalar is the shape the
guard already understands, and it holds whatever the file is called.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ChatGPT bridge got its homeserver URL from container_address, which
answers "what do I docker exec into". Under swarm that is a shell subshell,
and it was being written into a compose value, so docker stack config refused
the whole stack before anything deployed:
invalid interpolation format for
services.matrix-chatgpt-bot.environment.MATRIX_HOMESERVER_URL
container_service answers the other question, "what is this service
addressable as", and answers it per mode: the declared container name under
compose, <stack>_<key> under swarm. Compose therefore keeps exactly the name
it already used; only the branch that was broken changes.
Escaping the $ would have satisfied the parser and shipped a literal $(...)
into the bridge's environment, and the declared container name is emitted
through compose_only, so under swarm it has no DNS record at all. Neither is
a fix.
Name the resolved value once. The same lookup was already spelled out a
second time for the swarm restart, so both now read MATRIX_SYNAPSE_SERVICE.
Rename MATRIX_MDAD_SERVICE to MATRIX_MDAD_SERVICE_KEY while the vocabulary is
in hand: every one of its uses passes it as a service key, one of them as the
key of a rendered compose service, and _SERVICE now means the resolved form.
The role's four synapse identifiers finally say what they are - the declared
name, the addressable service, the exec target, and the key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The realign statement was passed through psql -c with :'hash' and :'name'
placeholders. psql expands those only while reading a script; under -c the
string goes to the server untouched, so postgres answered
ERROR: syntax error at or near ":"
LINE 1: UPDATE users SET password_hash = :'hash' WHERE name = :'name...
on every attempt, in both matrix rows of the run. The statement now arrives on
stdin through -f -, where the lexer does substitute, so the values stay
parameters instead of being pasted into SQL.
Move the statement to files/sql and read it with lookup('file', ...), the way
listmonk already provisions its MCP identity. That also makes the script form
structural: the statement now lives where only psql's script lexer reaches it,
so folding it back into -c is no longer a one-character edit.
The exec target was mode-blind as well. lookup('database', ..., 'container')
resolves the same value in every mode, while 'address' is the one that becomes
the resolver subshell under swarm; the shell evaluates it, so the quote filter
goes with it.
This alone does not turn the role green: the swarm rows die earlier, when
docker stack config rejects a compose value carrying the same kind of resolver
subshell. That is a separate defect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
register_new_matrix_user only creates. On PASS 2 of a matrix-deploy round the administrator and the ChatGPT bridge bot already exist carrying their PASS-1 password, the call returns 'User ID already taken', and both tasks tolerate that. chatgpt_bridge_token.yml then logs in with the rotated password against an account that still holds the old one and burns ten retries, which is where job 99670614406 of run 33445831564 ended.
A new realign_synapse_password.yml hashes the declared password with Synapse's own hash_password and writes it into the users table, addressing the engine through lookup('database', application_id, 'container') the way realign_dedicated_db.yml already does. It runs for both accounts and only when the registration found them, so a first deploy never reaches it.
The hash is handed to psql as a variable and used as :'hash', so it never reaches the SQL parser as text, and it is passed through Ansible's quote filter, so the bcrypt dollar segments survive the shell. Verified by running the rendered command against a stub that prints its argv: hash=$2b$12$abc/def$ghi arrives as a single unsplit argument.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge container was in a restart loop and never joined a room, so no
Matrix surface could reach the gateway. Four distinct causes, in the order
they surfaced.
MATRIX_ACCESS_TOKEN came from a credential declared with algorithm: plain,
a random string Synapse would never accept, so it resolved empty. With no
token, matrix-chatgpt-bot runs its token-fetch path, prints "Set
MATRIX_ACCESS_TOKEN to above token" and exits after 0.6s. Log the bot in
against Synapse and hand the container the token it actually issued.
compose.yml is written before the containers start, but the bot account only
exists after the Synapse health check further down the play, so the file has
to be rendered a second time once the token exists.
E2EE is disabled. The bridge uploads device keys that Synapse 1.159.0
rejects with `M_INVALID_PARAM: device_keys must not be null`, after which it
backs off forever and never completes a sync. This is a deliberate tradeoff:
bot DMs are no longer end-to-end encrypted, and the alternative is upstream
(a newer bot or an older Synapse). It hangs on MATRIX_CHATGPT_ENCRYPTION and
is one variable to flip back. The spec exercises an unencrypted room anyway,
because an E2EE timeline is opaque to an API-only client.
rc_login and rc_joins get a burst of 30. Synapse's defaults (login burst 3)
are too tight for a bridge that re-authenticates on every deploy, and the
restart loop drove them into M_LIMIT_EXCEEDED and later 429 (rc_joins.local).
This widens the burst allowance while leaving the sustained rate at 0.17/s.
The stack-host work moves into utils/stack_host_provisioning.yml so the
IS_STACK_HOST condition appears once instead of on two consecutive siblings.
Verified live: token present, container up rather than restarting, and
test-litellm-chatgpt.js passing. Not covered: a bot that meets an unjoinable
invite still crashes; 26 stale test rooms had to be cleared by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.
That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.
Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.
Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.
Timing cost is roughly one to two minutes on a deploy measured in hours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The flavor:compose `synapse.conf` template builds its proxy target from proxy_service_key + proxy_internal_port, which were never supplied, so the synapse homeserver domain proxied to an empty upstream (variant 2 only). Supply synapse + MATRIX_SYNAPSE_CONTAINER_PORT, matching the sibling front-proxy include.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The compose-flavor front-proxy wiring gave the element and synapse vhosts only their local port, so in swarm mode resolve_upstream fell back to services.matrix.ports.internal.http (the ansible-flavor MDAD service has no ports) and aborted. It now passes each vhost its own service key and internal port, mirroring the ansible flavor, so the proxy resolves matrix_element and matrix_synapse on the overlay; MATRIX_ELEMENT_CONTAINER_PORT is added as its source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On a fresh swarm cluster the bridges wrote registration.yaml as uid 1337, synapse (991) crash-looped on PermissionError, and the stack converge gate sits before the ansible chmod tasks, so the deploy could never turn green; swarm also ignores the service_healthy depends_on that serializes this in compose mode. The mautrix entrypoint chowns /data to $UID and drops privileges via su-exec, so pinning the bridge UID/GID to the synapse uid makes every registration owner-readable the moment it exists. The synapse uid moves into vars as the single source for the compose env and the volume chown.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bridges write their registration.yaml into directories they create with a restrictive mode, and the role only chmodded the file; synapse, running as its unprivileged user with the directory bind-mounted read-only, crash-looped on PermissionError and the stack never converged. The directories now get mode 0755 before the file chmod and the synapse restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three layers of one chain. The CI verifier polled the service matrix_matrix, which no variant creates (the ansible flavor deploys a node-local MDAD runner, the compose flavor names its services matrix_synapse and friends); workload: node-local routes it to the established skip like bigbluebutton. Under swarm the stack deploy is a notified handler, so the registration wait started before any bridge existed and killed the play with the handler still pending; a flush before the wait deploys the stack first, and is a no-op in compose mode. Synapse resolves the OIDC issuer through two docker layers where both 127.0.0.1 and host-gateway point at the wrong loopback; the issuer now maps to the runner bridge's node gateway (local subnet .1), emitted only when OIDC is enabled so a no-auth setup keeps its previous arguments, proven by the full Playwright suite against the live cluster.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The seed include re-passed the caller's loop vars as database_name/username/password: '{{ database_name }}' etc., shadowing each var with a template that resolves to itself; ansible-core 2.19 detects the recursion and dies on first access inside the seed assert. include_tasks inherits the enclosing scope, so the bridge loop's vars flow through without the re-pass; the seed's assert still sees them defined and non-empty.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.
Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm mode the container_address lookup returns a $(/usr/bin/resolve-container-id ...) fragment that must expand at exec time; piping it through Jinja quote hands docker the literal string and every exec fails with 'No such container' (broke all three snipe-it CI variants). Remove the quote at the seven snipe-it sites and the live matrix/openproject sites, convert openproject's argv exec to shell so the fragment can expand, split the matrix bridge restart per deployment mode with a manager gate, and mark the compose-only or eval-contract sites with nocheck. A new lint (container-address-quote) bans the pattern repo-wide.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.
- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
lint enforces whole-file on any edited file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DEPLOYMENT_MODE is binary (swarm / compose), so a negated `!= 'swarm'` gate
always means `== 'compose'`. Replace every negated gate across roles/ and
group_vars with the positive form for readability, and add a lint
(tests/lint/ansible/variables/test_no_deployment_mode_negation.py) that forbids
`DEPLOYMENT_MODE !=` in .yml/.yaml/.j2 with a `# nocheck:
deployment-mode-negation` escape.
The compose-verb / chdir / jinja-gate lint guidance messages now suggest the
positive form too; the gate matcher (utils/annotations/task_gate.py) and its
test inputs keep recognising the legacy `!= 'swarm'` so a nocheck-marked
exception is still detected as compose-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-creating an empty registration.yaml made the bridges skip registration generation ('appservice.as_token not configured') and left synapse broken. Reverts to the two-phase state where synapse + element + NC deploy healthy; the bridge registration/directory chicken-and-egg needs proper pre-generation (bridge -g one-off) as a follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After dropping the synapse->bridges depends_on (two-phase start), synapse starts before the bridges write registration.yaml, so docker bind-mounted the non-existent path and created it as a DIRECTORY, crash-looping the bridges ('Failed to save registration: open /data/registration.yaml: is a directory'). Pre-create empty registration.yaml files (force: false, mode 0666) before the compose up so docker mounts them as files, and wait for the bridge to write real content (search_regex: as_token, timeout 180) before restarting synapse.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-ups after the bridgev2 config migration, each verified in the live DiD stack: (1) URL-encode the bridge DB password in the postgres uri (the credential contains '/', which the stricter bridgev2 Go URL parser rejected as an invalid port) so signal/whatsapp/slack connect to postgres and generate their registration; (2) drop the telegram bridge from MATRIX_BRIDGES when its Telegram api_id/api_hash credentials are absent (telegram requires them at startup and crash-loops without — unlike the other bridges — blocking synapse); (3) give the bridge healthcheck a 5m start_period so the registration generation has grace before it is marked unhealthy.
(4) Two-phase synapse start: drop the synapse->bridges service_healthy depends_on (a single 'compose up -d' did not respect the start_period and aborted with 'dependency ... unhealthy' before the bridges generated registration) and instead restart synapse after the Ansible 'wait for registration files' step so it loads the freshly written appservice registrations. The bridges generate registration independently of synapse, so synapse no longer needs to block on their health.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
svc-db-postgres 02_init.yml asserts application_id == 'svc-db-postgres' AND database_consumer_id \!= 'svc-db-postgres'. When 01_core is skipped (run-once), the include_role application_id var does not reliably reach 02_init through a consumer's nested looped include (web-app-matrix's per-bridge DB seeding), so the assert fails. Pin application_id on the 02_init include so it holds for any caller. On the matrix side, freeze the consumer id eagerly before the provider include so database_consumer_id stays the matrix consumer instead of lazily resolving to svc-db-postgres once application_id is pinned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Six mautrix network bridges declared as mechanism: bridge, enabled: false, with DB creds referenced from meta/schema.yml. MATRIX_BRIDGES derived from addons; vars/bridges.yml, filter_plugins/bridge_filters.py and its unit test removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both flavors integrate with Infinito's central postgres (no more MDAD
bundled DB inside the DiD container). The connection vars live in a
single set of MATRIX_POSTGRES_* constants in vars/main.yml and are
referenced by both flavors' templates so the source-of-truth is
explicit and identical.
- vars/main.yml: MATRIX_MDAD_POSTGRES_* renamed to MATRIX_POSTGRES_*
(drop MDAD prefix — these are flavor-agnostic) + add
MATRIX_POSTGRES_DATABASE.
- templates/flavor/ansible/vars.yml.j2 (was mdad-vars.yml.j2):
postgres_enabled: false; wire postgres_connection_*/synapse
database_* to MATRIX_POSTGRES_* + matrix_synapse_container_network:
host so the inner Synapse inherits matrix-mdad's outer networks and
resolves `database` / `openldap` by hostname. valkey_enabled: false
(Synapse single-process; central redis path not used here).
- templates/flavor/ansible/hosts.j2 (was mdad-hosts.j2): renamed, no
content change. mdad- prefix dropped since the file is already
under flavor/ansible/.
- templates/flavor/ansible/services.yml.j2: matrix-mdad joins the
central `postgres` network (conditional on DB shared) and
`openldap` network (conditional on MATRIX_LDAP_ENABLED).
- templates/flavor/compose/synapse/homeserver.yaml.j2 and the six
templates/flavor/compose/mautrix/*.config.yml.j2 templates: switch
inline `lookup('database', application_id, 'host'/'username'/...)`
calls to MATRIX_POSTGRES_* (same constants the ansible flavor uses).
- tasks/flavor/ansible/: renumber 01-05 cleanly
- 03_database.yml (was 02a_database.yml) — sys-svc-rdbms include
- 04_run_playbook.yml (was 03_run_playbook.yml)
- 05_proxy_wiring.yml (was 04_proxy_wiring.yml)
- tasks/flavor/ansible/main.yml + 02_render_inventory.yml + README.md:
update references to the renamed files.
- tasks/flavor/compose/02_create-and-seed-database.yml: switch host/port
lookups to MATRIX_POSTGRES_* too.
Tests: 186/186 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
matrix-mdad is now a long-running compose service with systemd PID 1
and its own bundled docker daemon. MDAD writes its systemd unit files
and runs matrix-synapse/coturn/element/bridges/jitsi/element-call
entirely INSIDE that container via the inner docker daemon. Nothing
leaks to the host beyond standard /opt/compose/matrix/ role state.
Container layout:
- /sbin/init (systemd) as PID 1
- bundled docker.io daemon (inner DinD)
- bootstrap.service oneshot: runs `ansible-playbook setup.yml` once on
first boot; marker at /var/lib/matrix-mdad/bootstrap.done
- Named volumes: matrix_mdad_docker (inner /var/lib/docker),
matrix_mdad_systemd (/etc/systemd/system), matrix_mdad_matrix (/matrix)
- /sys/fs/cgroup:rw + cgroup: host for systemd-in-container
- Outer compose publishes synapse/element/federation ports to host loopback
mdad-vars.yml.j2:
- postgres_enabled: true (MDAD bundled postgres inside DiD)
- valkey_enabled: true (bundled redis)
- exim_relay_enabled: true when no outbound Mailu
- host_bind_port: 0.0.0.0 inside container so outer compose port-publish
can forward to host loopback
Dropped (no longer needed):
- 02a_database.yml — MDAD provisions postgres inside the DiD container
- 00_cleanup_mdad_residue.yml — no host residue to clean
- ensure rshared mount task — no host bind-propagation needed
- container wait matrix-mdad-runner — service is long-running not one-shot
- matrix_synapse_container_additional_networks_custom — DiD has its own
internal docker network
- matrix_synapse_container_extra_arguments CA mount — bundled CA inside
- coturn_container_additional_volumes — anonymous volumes live in inner
docker daemon, outer hlth-volumes can't see them
04_proxy_wiring.yml:
- admin user creation now uses two-hop exec: outer `container exec
matrix-mdad` -> inner `docker exec matrix-synapse register_new_matrix_user`
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
