Commit Graph
69 Commits
Author SHA1 Message Date
Prageeth Panicker bf37d5e7b3 fix(web-app-wazuh): address PR review feedback
Tor: use the shared tor block (bond 0.5, enabled when svc-net-tor is in the node's groups, tor pinned on in both variants) so a node serves one network, onion on a Tor node and clearnet otherwise.

SPOT and sizing: WAZUH_INDEXER_URL replaces four hand-built copies of the indexer URL, and the indexer heap is half of mem_limit instead of a hardcoded 1g (unchanged for the current 2g limit).

Fixes: bcrypt_hash rejects secrets over 72 bytes; the Keycloak group-removal helper uses the same resolver as the add helper and checks every response (the realm is no longer URL-encoded, matching the add helper), and a failed cleanup fails an otherwise passing test; the swarm certs generator removes a stale container before each attempt; vm.max_map_count is set before the early handler flush; securityadmin failed_when now needs rc 0 and the success marker.

The regenerated role README also picks up generator drift (mailu to stalwart).
2026-10-06 05:15:19 +00:00
Prageeth Panicker 87c8199012 feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
Add single-node Wazuh stack (manager, indexer, dashboard) as a new
Ansible role, integrated with the platform's Keycloak realm via native
OIDC and secondary LDAP authorization, with three-tier RBAC.

- Pins wazuh/wazuh-{manager,indexer,dashboard}:4.14.6 and
  wazuh-certs-generator:0.0.4 (one-shot cert bootstrap job)
- No external database: wazuh.indexer (OpenSearch) is self-contained
- Auth: native OIDC against Keycloak, working end-to-end after two
  infra fixes (JVM cacerts trust for the indexer's outbound calls to
  Keycloak, front-proxy vhost wiring)
- LDAP wired as a secondary, authentication-independent authorization
  backend (group-based backend_roles enrichment only)
- RBAC: three roles (administrator, security-analyst, readonly-auditor)
  via the platform's standard meta/rbac.yml + rbac_group_path pipeline
- Documents a genuine upstream Wazuh bug: wz-home as the default
  landing route throws an uncaught client exception for any
  non-all_access role; default route moved to OpenSearch Dashboards'
  own /app/home

Addressed two full rounds of review feedback (Kevin, Alejandro):
WEB_PROTOCOL usage, service-account naming, comment cleanup, shared
Keycloak-admin-helper extraction, a real TOCTOU race in RBAC test
setup, a Mermaid auth-flow diagram, meta/services.yml placement for
the certs-generator, CA-trust-path SPOT usage, CSP tightening
(including a genuine worker-src: blob: requirement found via live
testing), and a loud (non-silent) dev-bucket fallback warning.

Rebased onto main after a 3-month gap; brought the role in line with
~30 platform-convention changes introduced in that window (swarm
compose-template rules, Tor/onion contract, Playwright timeout/goto
wrappers, file layout, task numbering, comment policy, and more).

Live-deploy verification (9 iterations) found and fixed 6 further
real bugs invisible to lint/unit tests alone:
- credentials.* / meta/schema.yml -> secrets.credentials.* /
  meta/secrets.yml (another platform-drift casualty)
- notify: "compose up" -> "compose-up" (day-one handler-name typo)
- local.subnet collision with web-app-mailu's /24
- missing primary: false alongside tor.exclusive: false, which let
  the onion domain shadow the clearnet vhost entirely
- a second front-proxy include for the onion domain, later reverted
  (see below)
- shared sys-stk-front-proxy/sys-front-tls TLS-lookup keying bug
  (app-id-keyed vs. domain-keyed), which silently applied TLS to
  Wazuh's onion vhost

The last two are related: enabling genuine dual-stack (clearnet +
onion) reachability for tor.exclusive: false surfaced a real bug in
two shared platform roles. That fix was verified safe against all 103
apps currently in the repo, but is deliberately NOT included here -
shared-role changes deserve their own PR and review, not to ride in
on a single app's feature branch. Wazuh's second front-proxy include
was reverted accordingly; the resulting onion-reachability gap is
documented in the README as a known, platform-scoped limitation.

make test: 5508 passed, 3 failed (all three trace to untracked local
scratch files - Dockerfile.bak, scripts/lab-*.sh - not present in any
CI checkout).
2026-10-06 05:15:00 +00:00
Alejandro Roman c2ed7ce209 fix(stalwart): moved the provision to only keycloak use 2026-09-30 17:10:48 +04:00
Alejandro Roman 6144c85978 fix(stalwart): fixed secrets 2026-09-30 16:00:58 +04:00
Alejandro Roman 90a0f06396 fix(e2e): confirm the mapache password reset inside its dialog 2026-09-29 23:32:36 +04:00
Alejandro Roman 73b2593e47 fix(swarm-stalwart): mapache once and make fixes regarding comments 2026-09-29 18:31:02 +04:00
Alejandro Roman a390fea9e9 fix(swarm-stalwart): fix CI issues, cmments mapache 2026-09-28 18:23:16 +04:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 303c47fe2a fix(test-e2e-playwright): retry transient onion transport errors for every API verb, not only GET
0a52433dcb gave the standalone request fixture a Tor-resilient GET,
because the bundled socks client caps the CONNECT at 30s and a cold
onion circuit fails a request regardless of its timeout. POSTs and calls
made through context.request still went out once, and a circuit Tor
rejected ("Socks5 proxy rejected connection") was not recognised as
transient at all.

apiFetchOnion now takes the verb from opts.method and calls
request.fetch, so it serves both APIRequestContexts; apiGetOnion becomes
its GET shorthand. The rejection joins the transient patterns. Retries
stay onion-only and transport-only: clearnet keeps a single attempt and
HTTP or other errors re-throw at once.

The Mattermost LiteLLM spec routes its login, users/me, users/username
and channels/direct calls through it. The /posts call that sends the
prompt stays a single attempt, because a replayed post would message the
bot twice. The Hermes administrator spec reads /v1/models through
apiGetOnion.

env.test.js pins a POST surviving a rejected circuit, the single
clearnet attempt, no retry of a non-transport error over Tor, and the
GET shorthand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:05:02 +02:00
kevinveenbirkenbachandClaude Opus 5 b582846ab3 fix(test-e2e-playwright): make the Keycloak redirect and login-control checks able to fail
performKeycloakLogin polled the page URL with toContain(canonicalDomain).
On the Keycloak auth page the app domain already sits in redirect_uri,
so the poll was satisfied before the login had completed. The denial
path and the biber/admin iframe checks used the same substring test.
All of them now compare hostnameOf(url) with the canonical domain, which
is a bare hostname (env.js reads it with /\.onion$/).

LOGIN_CONTROL_NAME never matched a login link that carries an
icon-font glyph: Playwright's accessible name includes the private-use
character, and the pattern only allowed whitespace before the text. The
post-login waitFor({state: "hidden"}) therefore passed at once, even
before oidc.js had swapped the link. The pattern now admits \p{Co}
before the text, so that wait really waits for the swap.

dom.test.js pins both: glyph-prefixed login controls match while Logout
and Accessories do not, an IdP URL carrying the app domain in
redirect_uri is rejected, a look-alike suffix is rejected, and
about:blank yields an empty host.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 11:15:10 +02:00
kevinveenbirkenbachandClaude Opus 5 92ccfa2570 fix(test-e2e-playwright): stop the logout helpers waiting out windows whose answer is already known
confirmKeycloakLogoutIfPrompted waited the full 2s times the onion
multiplier for Keycloak's confirmation button even when Keycloak had
already redirected away because the end-session request carried an
id_token_hint. It now races the button against the page leaving the
logout endpoint (waitUntil: commit) and still clicks the button when
Keycloak renders the prompt. The dashboard's own confirmLogoutIfNeeded
stops the same way, and its poll interval no longer scales, so a slow
target is noticed sooner rather than later.

inAppLogout's first candidate poll ignored a visible settings link, so a
surface without a recognisable logout control or menu trigger, such as
the Mastodon home timeline, burned 30s times the multiplier before the
existing settings fallback ran. A visible settings link now ends that
poll as well; logout controls and menu triggers are still checked first
in every round. The settings selector moves to settingsLinksOn so both
callers share it.

Traces of run 34421944328: the dashboard's confirm probe expired after
2.03s on clearnet and 10.07s over Tor, and Mastodon's biber logout
polled 165s over Tor before reaching the settings page. Replayed in
Chromium, a Mastodon-shaped surface logs out in 3.24s instead of 33.45s
and a redirecting Keycloak logout returns in 0.20s instead of 2.00s,
while a prompting one is still confirmed. In the traced persona logouts
of six artifacts every other poll found a candidate in its first round,
so their path does not change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:49:10 +02:00
kevinveenbirkenbachandClaude Opus 5 0b79afdc0c fix(test-e2e-playwright): stop the OIDC login helper waiting out probes that cannot match
Every dashboard persona login spent 25s times the onion multiplier in
clickOidcLoginLink before it clicked anything. The markup probe matched
a[href*='openid-connect'], and the dashboard header renders the Account
dropdown's hidden Logout item, whose Keycloak end-session URL contains
the same string, before its Login link. .first() pinned the hidden item
until the 5s probe expired; the exact-name probe then missed for 20s
because Playwright counts the Font Awesome ::before glyph in the link's
accessible name. The probe now skips logout hrefs and resolves to the
Login anchor the loose probe used to click.

The adapter readiness wait cost every other role 15s times the
multiplier, because only the dashboard sets __oidcLoginReady. The helper
now waits for the load event and then only while the flag is false. The
dashboard loader, an inline head script, declares it false before the
Login link renders; oidc.js still sets it true once the click handler
is wired.

Traces of run 34421944328 show the two probes expiring at 5.07s + 20.00s
on clearnet and 25.01s + 100.00s over Tor. Across runs 34421944328 and
34419507038 the Tor dashboard persona test took 150-156s in all 102
compose runs against 32-34s in the 92 clearnet ones.
Replayed on the dashboard HTML those traces captured, the helper reaches
the Keycloak authorize endpoint in 0.08s instead of 40.09s, and still
waits for the adapter (1.52s) when the flag flips to true after 1.5s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:46:32 +02:00
kevinveenbirkenbachandClaude Opus 5 7f59f48ef1 perf(test-e2e-playwright): wait for the logout control instead of sleeping for it
Three waits in the logout flow were blind sleeps scaled by the onion multiplier.
A timeout is an upper bound and shrinks to nothing when the answer arrives early,
but page.waitForTimeout always burns its full duration, so multiplying it by five
spends the time whether the target is slow or not. The settle before the logout
sweep cost three seconds on clearnet and fifteen on an onion, every persona, every
role; the dropdown settle cost that again per menu trigger tried; and the poll
interval of waitForAnyLogoutCandidate was multiplied too, which made detection
five times coarser exactly where the target is slowest.

So the sleeps become waitForLogoutControl, a bounded poll that keeps the scaled
deadline and drops the scaling from the interval. It returns the moment a
logout-shaped element is visible, which is the condition the click that follows
needs anyway. Worst case is unchanged: with nothing to find it runs the deadline
out, exactly what the sleep did unconditionally. The first of the two
waitForAnyLogoutCandidate calls around the settle went with it; its result was
discarded, and the second still carries the thirty second deadline that covers a
navbar arriving late.

Four unit tests pin the contract, since the argument for the change is entirely
about when it stops waiting: it returns without sleeping when the control is
already there, returns as soon as one appears, costs at most the deadline when
none ever does, and polls on a fixed 250ms interval no multiplier touches.

Measured context, from run 34348821577: the same dashboard logout test takes
161.0 to 164.0s over Tor and 34.1 to 34.8s on clearnet across five jobs spanning
two distros, both deploy modes and three filesystems. A 1.9% spread across five
runner VMs is not network latency, it is a constant being multiplied, and the
ratio 4.73 is the multiplier itself. Solving the pair puts 32.0s of that test
inside scaled waiting against 2.3s of work. This commit removes the part of that
which is a sleep; the larger part is timeout caps on waits that expire and are
swallowed by a catch, which is a separate defect and stays for now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 00:31:02 +02:00
Alejandro Roman 4287dee96b fix(swarm-stalwart): fix new issue on CI and allow videos on logic playwright 2026-09-09 22:45:08 +04:00
Alejandro Roman e809765e23 fix(stalwart): fix all comments, fix a tor issue fouind and fix all SPOT issues 2026-09-02 15:02:46 +04:00
kevinveenbirkenbachandClaude Opus 5 43ec22ee20 fix(e2e): read the CSP off the response, not a second fetch
readCspString re-fetched the current URL through the Playwright API
request context and, when that failed, waited actionTimeout (30s) for a
<meta http-equiv="Content-Security-Policy"> that no template in this
repo emits. On a healthy runner that path works and costs one extra
request; when name resolution degrades it collapses, because the API
request context resolves DNS separately from the browser: the re-fetch
burns its 10s cap and the meta wait then drains whatever is left of the
caller's test budget.

That is what killed web-svc-logout's guest persona in run 32597248629,
job Pretix#0, whose swarm manager had a single resolver on 127.0.0.1
with dnsmasq no longer answering. Every guest test on that host inflated
to ~40s and the logout one, loading two external assets on top, died on
its 60s timeout having asserted nothing. The same role in Bluesky#0 and
YOURLS#0 of the same run, same role set, finished in 0.85s and 0.82s -
so this is a resilience fix, not a repair of a broken assertion.

gotoOnion now installs a page.on("response") recorder, so the header is
taken from the document the browser already received. That also covers
the surface reached through an SSO round trip, where no navigation
Response is in scope at the assertion - only guest.js holds one, biber,
admin and the suitecrm saml spec discard theirs. The re-fetch stays as
the fallback for a spec that navigates without gotoOnion; the meta
fallback is deleted.

The suite pins the header precedence (enforced over report-only) and
every guard in assertCspInjections: a disabled injector, a gate that
throws, an enabled injector without a base URL, a missing gate, a second
enabled injector, and the case-insensitive host match. Inverting the
precedence or dropping toLowerCase each trips exactly one test.

Verified against a headless Chromium and a local server counting
requests: the header survives a 302, no extra request is issued, a
missing injector host still fails the assertion, and a header-less page
falls back to exactly one re-fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 04:31:55 +02:00
kevinveenbirkenbachandClaude Opus 5 e2debf6945 fix(test-e2e-playwright): name the real failure when the session died first
inAppLogout's only failure message claimed "no in-app logout control
reachable on the current authenticated surface" even when the surface was
not authenticated at all. suitecrm variant #2 burned a full triage on
that: the app had destroyed its own session mid-journey (HTTP 500,
SCRMSESSID=deleted), rendered a blank shell, and the message pointed
everyone at a missing button that provably existed.

Before failing, the helper now checks - read-only, behind every success
path - whether a login surface is visible (the password-field marker
admin.js already uses) or the body rendered an empty shell, and says
"session was lost before the logout attempt" instead. A genuinely
missing control keeps the original message byte-identical.

Proven with a fake-page differential harness against the patched and
unpatched module: green paths never query the guard selectors, all three
red shapes produce the intended message, and the unpatched baseline
yields the old message in every case (10/10 checks).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:18 +02:00
kevinveenbirkenbachandClaude Opus 5 b48f9dd1f1 fix(test-e2e-playwright): reach a logout anchor that carries no href
SuiteCRM 8 renders its logout as <a class="dropdown-item global-links-sublink">Logout</a> inside <scrm-logout-ui>, and that anchor has no href, no role and no ngbdropdownitem - unlike every sibling in the same user menu (Edit Profile, Employees, Admin, About), which all carry one. An anchor without href gets no implicit link role, so getByRole('link'|'menuitem'|'button') cannot see it and neither can any href-based selector. Every existing candidate therefore missed a control that was on the page.

Read out of the failing run's Playwright trace: the dropdown did open - the Logout node is present in the frame snapshots - and the helper still clicked its 26 menu triggers without ever matching it. The earlier sibling fix widened the dropdown render wait, which this branch already carries via resolveTimeout(1_500); the wait was not the problem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:04 +02:00
kevinveenbirkenbachandClaude Opus 5 a30d67a63c fix(web-app-shopware): click through Keycloak's logout confirmation
Shopware's in-app logout leaves the browser on Keycloak's end-session page, which asks 'Do you want to log out?' because the request carries no id_token_hint. The spec waited 60s for a password field that page never renders. The shared persona helper already handles this for inAppLogout; it is now exported so the role-local spec can use it too.

Evidence: the error-context of job 95119330653 shows heading "Logging out" and button "Logout" where the login form was expected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:29 +02:00
kevinveenbirkenbachandClaude Opus 5 b97cf88881 fix(e2e): let the admin persona wait out the OIDC popup handoff
Playwright declares the timeout option on locator.isVisible() and isHidden() as deprecated and ignored - both return immediately. 7538b8a0b already fixed two call sites in this file and deliberately left four, arguing that is the password field STILL there and is a Login control STILL showing are correctly answered at once. That holds for the native login path it measured. It does not hold for the OIDC popup path.

keycloak.js polls the POPUP url and is satisfied once that lands on the callback, while the opener has not re-rendered yet - and admin.js inspects the opener. The trace of EspoCRM variant 0 shows the check starting 5.5 ms after the credential submit and taking 10.7 ms, the error-context snapshot shows the Sign in button visible but disabled, mid-submit, and the network log has oauth-callback.php and api/v1/App/user both returning 200 about 150 ms after the assertion had already failed. Biber passed the same journey in one attempt because biber.js:147 waits; administrator burned three attempts because it did not. That commit assumed EspoCRM 0 failed for an unrelated back-channel reason; it fails at admin.js:191, inside this branch.

The wait uses base 12000, not the 60000 biber carried. resolveTimeout multiplies by 5 on an onion target, so this is 60 s there. Across 13 onion traces the largest wait of any kind that ever SUCCEEDED was 790 ms and the worst navigation 1893 ms; 60 s is 76 times the former. At 60000 the same line would burn 300 s per attempt, and it burns only on the path that already fails, which is the path that has to stay diagnosable. Biber moves to the same base so the two personas cannot drift again.

The three remaining ignored arguments are deleted rather than kept. 7538b8a0b called them inert but harmless; inert is right, harmless is the part that cost it two call sites - the argument reads as a wait that never existed. A repository lint now rejects the construct for isVisible and isHidden only; isChecked, isDisabled, isEditable and isEnabled take a real timeout and are deliberately not matched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 13:43:28 +02:00
kevinveenbirkenbachandClaude Opus 5 e96c752549 Merge fix/playwright/env-template-gate into feature/svc-net-tor
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.

Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.

Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:

- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
  Same value, so nothing breaks today; editing the documented one would have
  silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
  the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
  rename never saw it. Its credential would not have resolved and the role
  would have deployed with an unusable administrator account.

The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.

make quality green on the merge result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:20:19 +02:00
kevinveenbirkenbachandClaude Opus 5 d34666c3a8 fix(e2e): follow a window.open IdP handoff in the shared OIDC login helper
clickOidcLoginLink watched only the opener. EspoCRM hands off to
Keycloak via window.open, so the opener correctly never navigates and
the helper abandoned a fully rendered Keycloak form in the child window;
the persona then died 45s later at the generic 'did NOT reach an
authenticated surface' gate. The trace shows BrowserContext.page with an
openerPageId, and the click itself succeeded - 'element is visible,
enabled and stable' with no error on the after-event. The [disabled]
Sign in button in the snapshot is EspoCRM's own signIn() disableForm(),
set after the click, not before it.

The helper now races the opener against the popup with Promise.any over
branches that only fulfil once a page reaches openid-connect/auth, and
returns that page; biber and admin pass it to performKeycloakLogin.

Promise.any rather than race: EspoCRM closes the popup after it
authenticates the opener by XHR, and a race would let that losing branch
resolve null and truncate the opener's budget. Measured on a stub
harness: popup closed at 500ms with the opener reaching the IdP at
1200ms returns the opener, where a race returns null. The 15s popup
window matches the idiom already used in the nextcloud addon specs; the
2s window this replaces would have returned null for any adapter that
runs PKCE or discovery before window.open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 22:11:13 +02:00
kevinveenbirkenbachandClaude Opus 5 a94a7ec73e fix(e2e): anchor the login-control name in the post-login gates
The gates that assert no sign-in affordance survived the OIDC round-trip
matched /log\s*in|sign\s*in|sso/i unanchored, so the pattern hit ordinary
navigation entries of an already authenticated page as substrings:
Accessories, Login Enabled, Blog inbox, Asset Login History. Any of them made
the gate report 'OIDC login did NOT establish a session' while the login had in
fact succeeded. Snipe-IT 2 of run 31340965955 failed exactly that way.

LOGIN_CONTROL_NAME anchors both ends and still admits the 'with ...' suffix, so
'Sign in with Keycloak' keeps matching and the pattern is not narrowed into
matching nothing.

The pre-login fallbacks in admin.js:58 and biber.js:79 stay deliberately broad:
they hunt for a sign-in control on the UNAUTHENTICATED landing page after the
anchored attempt misses, where no authenticated menu exists to collide with.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 13:56:57 +02:00
kevinveenbirkenbachandClaude Opus 5 72cbdeb9ab fix(playwright): retry the OIDC trigger after a login click that navigated
clickOidcLoginLink assumed the element matching the persona's anchored login pattern is the provider trigger. On mobilizon it is not: the navbar carries a plain Login router link, which the anchored pattern matches, while the actual provider button sits on the page that link opens and is labelled with the instance name, SSO Login (INFINITO.EXAMPLE). The helper clicked the navbar link, landed on /login, waited fifteen seconds for openid-connect/auth, returned false, and the persona then failed its authenticated-surface assertion with the login page as its final URL.

Retrying with the loose pattern would not have helped. On the landing page the provider button does not exist at all, and on /login the navbar link stays visible and sits ahead of the button in DOM order, so both patterns resolve to it a second time. What is missing is a second resolution on the page the click opened.

Run the existing resolution again, but only when the click navigated somewhere new without reaching the auth endpoint. That second pass additionally accepts a provider control by accessible name, which is the only thing that separates the button from the navbar link. Pass one keeps its locator chain, its timeouts and its order, so a role whose strict candidate already is the provider trigger returns on the first pass and never pays for the retry; guest.js does not call this helper at all, and the remaining callers are biber.js, admin.js and the mastodon seaweedfs spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 17:18:37 +02:00
kevinveenbirkenbachandClaude Opus 5 49d9ee5ec1 fix(test-e2e-playwright): stop the menu sweep clicking a profiles directory
ACCOUNT_MENU_NAME_RE matched profile unanchored, so Mastodon's public 'Profiles directory' link counted as an account menu. The sweep clicked it, navigated the authenticated SPA onto a Rails page with no settings link, and the settings fallback added in 141ff0645 then ran with nothing to find.

Across all 53 traces of run 31195451847 the only names that lose their match are 'profiles directory' and 'browse profiles' - the target - plus the class name profiler-toolbar, which is never an accessible name.

actionTimeout was unset, so Playwright's default of 0 let a click hang until the whole 300s test budget expired; Fediwall 2 reports an error with an empty body instead of a clean failure because of it. The slowest successful action in those traces takes 1.6s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 12:25:04 +02:00
kevinveenbirkenbachandClaude Opus 5 141ff06452 fix(test-e2e-playwright): look for the logout behind a settings link too
Mastodon keeps no logout on the surface the persona lands on, so the helper gave up after two steps. Its trace carries three /settings/preferences links and no sign_out anywhere, and the universal-logout service that would inject a fallback stays passive because Mastodon authenticates natively rather than through oauth2-proxy.

The step runs only after both existing ones fail, follows the first visible same-origin settings link, and repeats the two searches there. Relative hrefs only, so it cannot leave the app; no /account match, which would hit Mastodon's own /accounts/ profile URLs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 10:11:17 +02:00
kevinveenbirkenbachandClaude Opus 5 cad464ecf1 fix(test-e2e-playwright): match the identity provider by markup, not by label
clickOidcLoginLink only ever matched accessible names, so a role rendering its own login form beside the SSO entry won the match and the persona submitted the native form instead of reaching Keycloak. GitLab's sign-in button was clicked while form[action=/users/auth/openid_connect] and data-testid=oidc-login-button sat unused next to it; Mastodon's entry is named 'Login or Register', which no login regex matches.

The provider locator runs first with a 5s window and falls through to the existing strict/loose path, so roles without OIDC markup are unaffected. It requires the full protocol name with a separator rather than a bare 'openid' substring: the only href of that shape anywhere in roles/ is Mastodon's own login link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 10:01:01 +02:00
kevinveenbirkenbachandClaude Opus 5 2009d2c315 fix(test-e2e-playwright): stop the persona harness mistaking login controls for account menus
A Sign-in button matched ACCOUNT_MENU_NAME_RE, so inAppLogout clicked it as a menu trigger, and a username field matched the auth marker's data-testid rule, so an unauthenticated login page counted as an authenticated surface. The second one is why a broken auth chain surfaced as 'no in-app logout control reachable'.

Traces from CI run 31195451847 show the sign-in button clicked at CALL@123 and CALL@137, the marker returning true at ret@85, and the match resolving to input#user_login at @145 with no POST to /users/sign_in anywhere in the network log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 08:42:03 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a52433dcb fix(e2e): retry onion API requests past the 30s socks connect cap
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 11:29:14 +02:00
kevinveenbirkenbachandClaude Fable 5 c4e41217c3 test(e2e): recover onion circuits via a Tor-guided wait script + retry goto timeouts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:39:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1b9d6cc862 fix(e2e): retry onion navigation on Firefox/WebKit transient network errors
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 21:08:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f3496b4421 fix(e2e): confirm Keycloak logout page in inAppLogout; prometheus uses the persona logout
The injected universal-logout control navigates to Keycloak's
end-session endpoint without id_token_hint, so modern Keycloak renders
a logout-confirmation page. Nothing clicked it: the SSO session and the
oauth2-proxy cookie survived, and the prometheus spec's strict
post-logout assertion (redirect back to Keycloak auth) timed out on
onion nodes across five CI jobs (svc-net-tor, discourse, svc-bkp,
fedora/manjaro env).

inAppLogout now confirms the Keycloak prompt (#kc-logout /
logout-confirm form) after clicking the logout control, completing the
real user journey for every persona flow. The prometheus spec drops its
bespoke fire-and-forget /logout goto in favour of the shared
inAppLogout plus a networkidle settle; the strict step-6 redirect
assertion is unchanged.

Verified live over onion: prometheus suite 5 passed / 2 gated skips.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 20:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5abfcfc6d6 feat(e2e): shared Tor-aware Playwright harness
Promote the Firefox onion fixture (secure-context .onion prefs, SOCKS
remote-DNS) from web-app-dashboard to test-e2e-playwright as the single
source of truth; the dashboard copy now re-exports it. Stage the shared
fixture on all three paths (deploy runner, lint-playwright, rerun-spec).

gotoOnion: onion navigations default to waitUntil=domcontentloaded —
heavy chunked SPAs (Element) serialise 30+ bundle requests over Tor, so
the load event regularly exceeds the navigation cap; callers' explicit
onion-scaled selector waits cover app boot instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 10:56:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f8c016b294 fix(test-e2e-playwright): give onion targets a realistic navigation budget
gotoOnion hard-coded a 60s page.goto timeout, so a cold full-page load
over freshly built Tor circuits aborted before the onion answered and
the deploy gate flaked. The deploy now passes an onion-aware
PLAYWRIGHT_NAVIGATION_TIMEOUT (180s onion / 60s clearnet) plus a larger
onion per-test budget (600s), and gotoOnion honors the env instead of
the hard-coded ceiling. Fast loads are unaffected; only slow onion
navigations may use the larger budget.

Verified: full deploy matrix green with zero page.goto timeouts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 01:56:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 edb96e9ae3 feat(dashboard): make keycloak-js login work over Tor onion
Chromium never treats plaintext .onion origins as a secure context, so
keycloak-js (PKCE via crypto.subtle) could not run. A custom Playwright
browser fixture launches Firefox with the Tor-Browser prefs
(dom.securecontext.allowlist_onions, origin/mixed-content handling) for
onion targets and keeps Chromium for clearnet; all dashboard specs share
it via the register pattern.

keycloak-js is now vendored from the deployment's own CDN as an ES
module (jsdelivr is unreachable over Tor) with CORS opened on the CDN
proxy. Personas and the oidc-login spec wait for adapter readiness and
navbar settle before driving login/logout, which the async code->token
exchange needs over Tor latency.

Verified: full deploy matrix green (onion 14/14 twice, clearnet pass).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 01:55:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 05c082fc14 test(e2e/personas): run biber/admin journeys over onion with adaptive timeouts
keycloak.js/logout.js wrap every wait in resolveTimeout so Tor's higher latency does not abort the OIDC adapter/logout waits. biber.js/admin.js drop the onion skip (personas now execute over onion instead of being skipped) and add a sharpen guard: a completed OIDC round-trip that still shows a Login control fails explicitly as a login failure, not a misleading 'no logout control'.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 17:52:14 +02:00
kevinveenbirkenbachandClaude Opus 4.8 82606ef2af test(playwright): onion transport + secure-context test primitives
env.js gains gotoOnion (retries transient Tor-circuit errors on page.goto, clearnet unchanged) and isOnionCanonical (detect a .onion canonical). playwright.config.js marks .onion origins as a secure context via --unsafely-treat-insecure-origin-as-secure, emulating real Tor Browser (RFC 7686) so Clear-Site-Data logout and the Storage Access API behave as in production over onion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:47:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 266a37d965 feat(svc-net-tor): onion mail/SSH ports + TLS-off compat (CA CN, HSTS, msmtp)
- svc-net-tor: expose SSH (22) + Mailu SMTP/submission/IMAP (25/465/587/143/993)
  as HiddenServicePorts on the node onion (TOR_ONION_SSH_ENABLED, 19_tor.yml).
- email lookup: relay plaintext (tls off, port 25) for .onion mail domains —
  no public TLS cert for a .onion; onion transport is already authenticated.
- CA root CN: drop the " Root CA" suffix when a 62-char v3 onion + suffix would
  overflow the X.509 commonName 64-char cap (02_tls.yml).
- Playwright onion compat: HSTS is HTTPS-only (RFC 6797), absent on the http
  onion vhost. New shared helper expectHstsWhenTls() gates the assertion on the
  reached scheme; wired into web-svc-cdn spec.
- rerun-spec.sh: route the inner-loop browser through the Tor SOCKS proxy when
  the staged env targets a .onion, mirroring the deploy-time runner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 16:39:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36ebf34b64 Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:13:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4698c65ce0 fix(test-e2e-playwright): bound networkidle waits with a 15s timeout
Unbounded `page.waitForLoadState("networkidle")` in the shared admin/seaweedfs
personas and the dashboard simpleicons-card test waited the long default before
the existing `.catch` swallowed it, so apps that never go network-idle
(websocket/long-poll: mailu admin, matomo) made the Playwright run exceed its
1200s task timeout. The result was already discarded, so capping each wait at
15s only bounds it; it matches the timeout already used elsewhere
(web-app-mailu playwright.spec.js, pihole, bluesky).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8fb1af2345 fix(test-e2e-playwright): widen in-app logout dropdown render wait to 1.5s
The universal-logout helper clicked the account-menu trigger and waited only 750ms for the dropdown to render its items. OpenProject's Angular SPA renders them slower under deploy load, so the logout control was missed (flaky biber universal-logout, recovered only on retry). Bump the post-click wait to 1500ms. Verified in-container: both universal-logout specs pass, no regression.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 97a43feb5f fix(e2e): tolerate networkidle timeout in CSP injection check
Realtime apps (websockets/polling) keep the connection busy so page.goto never reaches networkidle; the asset loads the assertion checks are captured during navigation regardless, so swallow the navigation timeout instead of failing the spec.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 41f12cb42c fix(test-e2e-playwright): seaweedfs check requires a NEW object key by default; opt-in mtime for single-slot consumers
Background workers (PeerTube transcode, Pixelfed media jobs) rewrite existing objects during the poll window, so accepting any changed mtime made those consumers false-green. Default is now a new key only (the original count-growth semantics); single-slot consumers that overwrite one fixed key in place (fider tenant logo) pass expectInPlaceOverwrite to also accept a changed mtime.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e8d7c2f0d3 test(test-e2e-playwright): make the SeaweedFS storage-check poll deadline configurable (pollDeadlineMs, default 60s) so slow async-S3 movers can extend it
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1fa5161dd2 feat(web-app-nextcloud): provision on SeaweedFS S3 object store + consumer e2e test
Render the S3 objectstore config (templates/config/objectstore.config.php.j2) into NC's config dir before install, gated on objstore.enabled — NC cannot switch an already-installed instance to objectstore primary storage, so it must be present at install time.

Add the SeaweedFS consumer storage Playwright test (files/playwright/test-seaweedfs.js, playwright.env.j2 SEAWEEDFS_* vars, spec wiring; drop the playwright-service-flag nocheck for the seaweedfs service) plus the shared runSeaweedfsStorageCheck helper (admin logs into the Filer once, counts bucket objects via the Filer JSON listing, asserts growth) and the seaweedfs-consumer integration lint.

web-app-seaweedfs: bind weed to 0.0.0.0 (-ip.bind) so the multi-homed gateway is reachable by consumers that resolve the service name to its IP on the shared network.

nextcloud_install: classify 'was not found on the appstore' as unavailable (skippable, non-fatal) so a plugin without a release for the pinned NC version does not abort the whole deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4cdfde8eef fix(test-e2e-playwright): seaweedfs check requires a NEW object key by default; opt-in mtime for single-slot consumers
Background workers (PeerTube transcode, Pixelfed media jobs) rewrite existing objects during the poll window, so accepting any changed mtime made those consumers false-green. Default is now a new key only (the original count-growth semantics); single-slot consumers that overwrite one fixed key in place (fider tenant logo) pass expectInPlaceOverwrite to also accept a changed mtime.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:58:00 +02:00