Commit Graph
10 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 df19dd103c fix(matrix): match the invite tile on the button, not on a row that is absent
9661800cb0 scoped the tile to [role="row"][aria-level="2"], taken from
upstream's test helpers for Element 1.12.28. meta/services.yml pinned v1.12.27
then and has never held 1.12.28 on any ref, so the selector described markup
the deployment does not render.

Read out of biber's DOM in the trace of the failing job: role="row" occurs zero
times, aria-expanded zero times, aria-label="Toggle " zero times, and every
aria-level in the document is "1" on a heading. The only aria-level="2" in the
whole trace is inside the test's own injected script. The invite is a bare
button in a virtuoso list:

  DIV[data-testid="virtuoso-item-list"] > DIV[data-index="0"]
    > BUTTON[aria-label="Open room Administrator Infinito.Nexus invitation."]

So the tile was never clicked, biber stayed on the welcome screen, and the poll
spent its 120 seconds on a page that could not change. The screenshots of all
three attempts show the same state: the invite row visible in the list, a
device-verification prompt beside it, no room open.

Matching the button directly drops the dependency on the wrapper. The label
prefix and the partner match are unchanged, and "Open room settings" is the
only other button carrying the prefix, which the partner match excludes.

The section expansion stays. It is a no-op against 1.12.27, where nothing
reports aria-expanded, and it is what upstream's helpers describe for the
release after it.

Not verified: the accept step behind the tile. biber never reached the invite
view in any attempt, so which of "Start chatting", "Accept" or "Join" 1.12.27
renders there is unobserved, and the anchored regex covering all three is
carried over untested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:49:39 +02:00
kevinveenbirkenbachandClaude Opus 5 9661800cb0 fix(matrix): drive the DM invite through Element's new room list
Element 1.12.28 rewrote the room list. Pending invites now sit in their own section that renders collapsed, so no room row exists in the DOM until it is expanded. The predicate iterated an empty list and the poll spent its whole budget with no reachable path to success.

Expand every collapsed section first, then match the row by the shape upstream's own test helpers document at revision fe37d199: role=row, aria-level=2, and a button whose aria-label starts with 'Open room'. Of the four previous selectors, three matched nothing in this build and the fourth reached only the Spaces navigation. Match aria-label as well as text, because Element renders the DM partner sometimes as a display name and sometimes as a raw MXID.

role=gridcell is deliberately not among them: the cell precedes its own button in document order, so break would land on the DIV while Element's handler sits on the button.

The accept regex is unchanged. Element 1.12.28 labels the DM accept 'Start chatting' and still ships 'Accept' and 'Join'; all three are covered and Decline stays excluded. Every click stays a raw DOM click because a persistent device-verification toast overlaps the room list and would intercept an actionability-checked click.

Verified on a local compose deploy of variant 0 with sso and tor against 1.12.28: test-dm-admin-biber.js passed in 4.2m, 7 passed, no failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:18:11 +02:00
kevinveenbirkenbachandClaude Opus 5 dcb9ddb9cf fix(playwright): stop stretching waits on server-side clocks by the onion multiplier
The matrix DM spec sleeps 120s before the administrator signs in and 30s
between the two sign-ins, and the rate-limit dialog handler cools down
for 45s, all so Synapse's rc_login bucket (per_second 0.17 in the
compose flavor) can refill. That refill runs on Synapse's clock, yet the
sleeps went through resolveTimeout and became 600s, 150s and 225s over
Tor. They now use fixed constants with the clearnet values, which pass
there as they are. The 5s pause before the marker is sent stays scaled:
Element shares its megolm session through the homeserver, so that one
does depend on the network.

Mailu's two poll intervals (500ms for a visible element, 3s for an inbox
refresh) were scaled as well, so an arrived mail was noticed up to 15s
later over Tor. They are fixed now; the deadlines still scale. Named
constants are what the raw-timeout lints reserve for intervals that must
not scale.

Across runs 34421944328 and 34419507038 the matrix DM test took 960s
over Tor against 210s on clearnet; the two refill sleeps alone account
for 600s of that difference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:51:29 +02:00
kevinveenbirkenbachandClaude Opus 5 a0cc3dbe40 feat(lint): make default.env the enforced source of the primary domain
default.env declares INFINITO_DOMAIN and everything is meant to derive from it.
Nothing checked that, so copies accumulated. A copy is a second source nobody
updates, and it fails in the least useful way: the code keeps running and
quietly talks about the wrong host. integration_mastodon is the worked example
-- it asserted the provisioned partner equalled a clearnet literal, so the spec
passed on a clearnet node and failed on an onion one while the product was
right in both.

tests/lint/repository/spot/test_no_hardcoded_primary_domain.py now fails on the
literal anywhere in tracked source. It reads the value it hunts for FROM
default.env, so the rule follows the SPOT instead of repeating it and would
otherwise be its own first violation.

Scope, and why each exclusion is not a loophole: default.env itself is the
source; .md is documentation; tests/ and scripts/tests/ hold fixtures that must
assert against a named host; inventories/ is where writing the real values is
the entire point; gitignored output carries the value because it correctly
derived it. Everything else is in.

Two real hardcodes are repaired rather than suppressed.
utils/domains/default_primary.py mirrors utils/roles/lifecycle.py -- one
function reading one key through parse_static_env -- and replaces the module
constant in utils/cleanup/nginx_vhosts.py and the argparse default in
cli/meta/domains/__main__.py. Both resolve to the same value as before.

Seven documentation sites become <primary-domain> placeholders instead of
carrying a nocheck: a Python docstring cannot host one without polluting the
docstring, and a placeholder does not rot when the default changes. The single
nocheck is scripts/system/tls/trust/wsl2.sh:61, where the marker is matched
verbatim against what an earlier run wrote into the Windows hosts file --
deriving it would orphan that block instead of replacing it.

Collateral, not intent: touching roles/sys-ctl-hlth-csp/files/script.py brought
it under the comment lint, which found five pre-existing narration comments.
Three deleted, `fn[:-5]  # strip ".conf"` replaced by fn.removesuffix(".conf"),
and the root-user comment rewritten in the sanctioned Exception form. Its 33
tests stay green.

tests/lint/repository/test_no_hardcoded_container_name.py (05f425ea5) is the
same family for INFINITO_CONTAINER and could move into spot/ later; this change
leaves it where it is.

make test-lint: 326 passed. The rule found 18 sites before the cleanup and none
after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:28 +02:00
kevinveenbirkenbach eb921895ff Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-08-06 10:43:55 +02:00
kevinveenbirkenbachandClaude Opus 5 145f86bf11 fix(matrix): let the DM test inherit the shared retry budget
web-app-matrix 0,1,2 failed with no assertion at all - JUnit reports
errors=1, failures=0. Chromium crashed: dmesg holds exactly two traps in the
whole 04:34-05:55 window, second-exact to both attempts and at the identical
binary offset, so one fatal CHECK() twice. int3 is Chromium aborting itself,
not a kill from outside.

The same job proves the budget is what decided the outcome:
tests/test-seaweedfs.js hit the identical error class twice, ~5s in each
time, and passed on attempt 3 because it inherits retries: 2 from
playwright.config.js. The DM test opted down to 1 and died.

Ruled out with evidence rather than by elimination: not the application
(zero 502s in the whole openresty log, Synapse 200 throughout), not the auth
chain (both OIDC logins succeeded on attempt 1), not the test (no assertion
reached), not the timeout (effective budget 1200s, attempts took 170s and
164s), not OOM (dmesg-oom.txt empty, 9G available).

The comment goes with it. It attributed the crashes to processes being
'OOM-killed', which is exactly what the artifacts refute, and it would have
sent the next investigator after memory pressure that was never there.

Worst case is 8.5 min instead of 5.7 against a 59-minute suite budget.

Not addressed here: which CHECK fires at that offset is unknowable from
these artifacts because Playwright discards the browser's stderr.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 10:22:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a581f6622 feat(matrix): full onion support — OIDC backchannel, MXIDs, Tor-stable specs
Synapse on an onion node could never complete SSO: it has no Tor, so the
http://auth.<onion> issuer was unresolvable server-side (discovery/jwks/
token), and Synapse hard-rejects non-https issuers on top.

- runner/hostpin.{sh,service}: pin the onion auth host to the runner's real
  default gateway (node OpenResty) from /proc/net/route, Before=docker.service
  so nested containers see it from their first request; host-gateway is
  unusable in nested DinD (resolves to the inner docker0)
- Synapse bind-mounts the runner's /etc/hosts; token iss stays the onion
  issuer the browser uses
- bootstrap.service: PassEnvironment (systemd units do not inherit container
  env; MATRIX_MDAD_PLAYBOOK_TAGS only worked via its fallback default)
- vars.yml.j2: skip_verification on onion (Synapse's escape hatch for
  issuers without https; Tor provides transport security per RFC 7686)
- 02_render_inventory: drop the MDAD bootstrap marker + bounce the runner on
  inventory changes — mode=update reconfigures the nested stack at all now
- MATRIX_SERVER_NAME follows matrix_domain onto the onion so MXIDs built from
  it (DM spec, bridge configs) reference users that exist
- specs: shared Firefox onion fixture (16 files), gotoElementApp reload-retry
  against Element's Tor-triggered init race, seaweedfs spec gates on the
  filer frontend, env uses canonical_url consumer=

Verified in-container: SSO 302 to onion Keycloak, admin+biber OIDC login,
E2EE DM admin<->biber, CSP — all green over Tor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 10:56:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e720ce4394 fix(web-app-matrix): harden user provisioning + split spec into per-scenario files
Role-code (meta/services.yml + tasks/01_docker.yml):
- Drop the services.matrix.setup gate. The flag defaulted to false and
  silently skipped admin + chatgptbot creation, leaving Synapse with 0
  users. Both tasks now run unconditionally; idempotency comes from the
  failed_when filter on "User ID already taken" instead of the previous
  ignore_errors: true that masked every other error class too.
- exec -it → exec -T: Ansible has no TTY, so -it failed silent and the
  ignore_errors swallowed it; -T is the no-tty/no-stdin variant the
  Ansible command module expects.
- Quote every template var passed to register_new_matrix_user with
  `| quote`. The administrator password (`'$tr0ng€xampl3PW\!`) carries
  characters that break the command module's shlex split otherwise.
- no_log: MASK_CREDENTIALS_IN_LOGS so plaintext passwords stop leaking
  into Ansible output.

Playwright spec (files/playwright/):
- Split playwright.spec.js (643 lines, single file) into _shared.js +
  six test-*.js modules following the taiga/nextcloud/pixelfed pattern.
  Each scenario is now individually grep-able and inspectable from the
  reporter output instead of all sharing the playwright.spec.js:NNN
  prefix.
- Add test-login-admin-native.js: native password login for the
  administrator persona, gated to OIDC-off variants
  (`test.skip(oidcServiceEnabled, ...)`). Reuses the existing
  signInViaElementPassword helper.
- test-dm-admin-biber.js: add `skipUnlessServiceEnabled("oidc")` —
  biber only exists in Synapse via Keycloak first-login federation,
  so the DM scenario is structurally impossible without OIDC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:13:08 +02:00