Commit Graph
8 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 945f816193 fix(matrix): give the ChatGPT bridge a session Synapse will accept
The bridge container was in a restart loop and never joined a room, so no
Matrix surface could reach the gateway. Four distinct causes, in the order
they surfaced.

MATRIX_ACCESS_TOKEN came from a credential declared with algorithm: plain,
a random string Synapse would never accept, so it resolved empty. With no
token, matrix-chatgpt-bot runs its token-fetch path, prints "Set
MATRIX_ACCESS_TOKEN to above token" and exits after 0.6s. Log the bot in
against Synapse and hand the container the token it actually issued.

compose.yml is written before the containers start, but the bot account only
exists after the Synapse health check further down the play, so the file has
to be rendered a second time once the token exists.

E2EE is disabled. The bridge uploads device keys that Synapse 1.159.0
rejects with `M_INVALID_PARAM: device_keys must not be null`, after which it
backs off forever and never completes a sync. This is a deliberate tradeoff:
bot DMs are no longer end-to-end encrypted, and the alternative is upstream
(a newer bot or an older Synapse). It hangs on MATRIX_CHATGPT_ENCRYPTION and
is one variable to flip back. The spec exercises an unencrypted room anyway,
because an E2EE timeline is opaque to an API-only client.

rc_login and rc_joins get a burst of 30. Synapse's defaults (login burst 3)
are too tight for a bridge that re-authenticates on every deploy, and the
restart loop drove them into M_LIMIT_EXCEEDED and later 429 (rc_joins.local).
This widens the burst allowance while leaving the sustained rate at 0.17/s.

The stack-host work moves into utils/stack_host_provisioning.yml so the
IS_STACK_HOST condition appears once instead of on two consecutive siblings.

Verified live: token present, container up rather than restarting, and
test-litellm-chatgpt.js passing. Not covered: a bot that meets an unjoinable
invite still crashes; 26 stale test rooms had to be cleared by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:45:09 +02:00
kevinveenbirkenbachandClaude Opus 5 f713ba8350 refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.

The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.

--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.

Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.

tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.

Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 16:51:56 +02:00
kevinveenbirkenbachandClaude Fable 5 f0ddb4c61d fix(oidc): reach the onion issuer server-side for peertube, penpot, matrix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:39:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 674b58fdfe fix(matrix): load bridge registrations in synapse + url-encode legacy bridge db passwords
Two bugs kept every mautrix bridge from working: (1) homeserver.yaml.j2 gated app_service_config_files on '{% if MATRIX_BRIDGES | bool %}', but MATRIX_BRIDGES is a list and Ansible's bool filter on a list is always False, so synapse loaded zero appservices and rejected every bridge's as_token (401). Use '| length > 0'. (2) the legacy facebook/instagram bridges interpolated the DB password into the postgres:// URL unencoded; a password with a reserved char crashed mautrix-python with 'Invalid URL: port can't be converted to integer'. URL-encode it like the bridgev2 bridges. Verified by a clean redeploy that recreated the bridge containers and regenerated registrations from the templates: all 5 bridges + synapse healthy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 07:08:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 646109690d fix(matrix): mount the bridge dir into synapse so registration.yaml stays a file
Mounting the not-yet-existing per-bridge registration.yaml into synapse made docker create it as a directory, which crash-looped the bridge ('registration.yaml: is a directory'). Mount the existing bridge directory (mautrix/<bridge> -> /data/mautrix-<bridge>) read-only instead and point app_service_config_files at <dir>/registration.yaml, so the bridge generates the file normally and synapse loads it after the restart.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 04:34:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ffac55c1d5 fix(proxy): stop cross-role webserver_* var leak (config fields + task-scoping + front-proxy reset + lint)
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).

- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
  resolve client_max_body_size from the role's own server config
  (lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
  include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
  on the render tasks, so a leaked value from any other source is neutralized;
  a role's own include params (#20) still override the reset (precedence-tested,
  direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
  (suppressable via '# nocheck: leaky-proxy-var').

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:18 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2f4ef3cfad refactor(web-app-matrix): central postgres + shared MATRIX_POSTGRES_*
Both flavors integrate with Infinito's central postgres (no more MDAD
bundled DB inside the DiD container). The connection vars live in a
single set of MATRIX_POSTGRES_* constants in vars/main.yml and are
referenced by both flavors' templates so the source-of-truth is
explicit and identical.

- vars/main.yml: MATRIX_MDAD_POSTGRES_* renamed to MATRIX_POSTGRES_*
  (drop MDAD prefix — these are flavor-agnostic) + add
  MATRIX_POSTGRES_DATABASE.
- templates/flavor/ansible/vars.yml.j2 (was mdad-vars.yml.j2):
  postgres_enabled: false; wire postgres_connection_*/synapse
  database_* to MATRIX_POSTGRES_* + matrix_synapse_container_network:
  host so the inner Synapse inherits matrix-mdad's outer networks and
  resolves `database` / `openldap` by hostname. valkey_enabled: false
  (Synapse single-process; central redis path not used here).
- templates/flavor/ansible/hosts.j2 (was mdad-hosts.j2): renamed, no
  content change. mdad- prefix dropped since the file is already
  under flavor/ansible/.
- templates/flavor/ansible/services.yml.j2: matrix-mdad joins the
  central `postgres` network (conditional on DB shared) and
  `openldap` network (conditional on MATRIX_LDAP_ENABLED).
- templates/flavor/compose/synapse/homeserver.yaml.j2 and the six
  templates/flavor/compose/mautrix/*.config.yml.j2 templates: switch
  inline `lookup('database', application_id, 'host'/'username'/...)`
  calls to MATRIX_POSTGRES_* (same constants the ansible flavor uses).
- tasks/flavor/ansible/: renumber 01-05 cleanly
  - 03_database.yml (was 02a_database.yml) — sys-svc-rdbms include
  - 04_run_playbook.yml (was 03_run_playbook.yml)
  - 05_proxy_wiring.yml (was 04_proxy_wiring.yml)
- tasks/flavor/ansible/main.yml + 02_render_inventory.yml + README.md:
  update references to the renamed files.
- tasks/flavor/compose/02_create-and-seed-database.yml: switch host/port
  lookups to MATRIX_POSTGRES_* too.

Tests: 186/186 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 19:00:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 70c1a03f13 feat(web-app-matrix): add ansible flavor via unified compose.yml.j2
Implements requirement 025 (docs/requirements/025-web-app-matrix-ansible-flavor.md).

Architecture:
- services.matrix.flavor: ansible | compose
- Single templates/compose.yml.j2 branches on matrix_flavor
- ansible flavor: defines matrix-mdad-runner one-shot compose service that
  DiD-executes upstream matrix-docker-ansible-deploy (pinned ref + migration
  validated version). MDAD writes /etc/systemd/system/matrix-*.service units
  on the host; post-up Ansible task container-waits the runner, then probes
  loopback ports + wires sys-stk-front-proxy.
- compose flavor: legacy in-repo stack (synapse/element/coturn/bridges) kept
  for regression coverage, marked deprecated.

Variant-switch ansible -> compose:
- 00_cleanup_mdad_residue.yml tears down MDAD-installed systemd units +
  leftover matrix-* containers before compose-flavor brings up its own
  matrix-synapse (else: name conflict, compose down doesn't touch
  systemd-managed state).

Reorganization:
- roles/web-app-matrix/{tasks,templates,files}/flavor/{ansible,compose}/
- Renamed legacy compose-flavor task files into flavor/compose/
- New runner Dockerfile (multi-stage, ARG-parameterized FROM via build-args)

Verified: 3 variants x 2 passes (sync+async) full_cycle green, all 6
PLAY RECAPs failed=0. V1 (ansible all-on), V2 (ansible no-auth), V3
(compose deprecated all-on). Playwright matrix tests pass per variant.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 10:21:41 +02:00