Commit Graph
39 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 04d4d31335 fix(test-e2e-playwright): give every Playwright timeout one source
The config held literals for values vars/main.yml already owned, and the two
drifted without a sound. actionTimeout carried `|| 30_000` for its whole life
while nothing ever set PLAYWRIGHT_ACTION_TIMEOUT, so every run used the
unscaled 30 s. expect had no entry at all, leaving Playwright's built-in 5 s to
govern every assertion over Tor while its neighbours were scaled by the onion
multiplier. That is what killed the second Nextcloud login attempt: an
expect(usernameField).toBeVisible() that gave up after 5 s inside a 150 s
budget.

vars/main.yml now owns the values in TEST_E2E_PLAYWRIGHT_CONFIG_TIMEOUTS, the
harness renders that map into the staged .env both the deploy and
scripts/tests/e2e/rerun-spec.sh already pass, and the config demands each name
through requiredTimeout() instead of inventing a fallback. The two `-e` flags
that fed the old pair are gone with it.

scripts/lint/playwright.sh stages the same config and runs `playwright test
--list`, which loads it, but built its stub env only from each role's
playwright.env.j2 and no such template carries a timeout. Without this it would
have aborted all 87 specs on a missing variable. It now takes the names from
the same map; the value is a placeholder because --list consumes none of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 12:04:27 +02:00
kevinveenbirkenbachandClaude Opus 5 214e7e29d1 fix(test-e2e-playwright): scale onion timeouts by the tor flavor instead of a flat five
resolveTimeout multiplies every Playwright window by 5 for a .onion
target. That default arrived in 35dee215c8 (2026-07-08), before
5399b48482 (2026-07-24) gave CI the chutney flavor, a private Tor net on
localhost, and PLAYWRIGHT_ONION_TIMEOUT_MULTIPLIER was never set. Waits
that expire by design therefore cost five times their base under chutney
too, which is where most of the Tor surcharge came from.

The Playwright run now passes the multiplier for onion targets only,
looked up from svc-net-tor's services.tor.flavor: 2 for chutney, 5 for
the public network, whose circuits really are slow. An unknown flavor
fails the lookup instead of falling back, and a unit test holds every
flavor directory of svc-net-tor to an entry.

Chutney traces of run 34421944328: 700 successful timed waits across 186
tests used at most 0.42 of their unscaled window, and the longest
client-side expect took 5.1s. Across runs 34421944328 and 34419507038 no
chutney test with its own ceiling (401 of them) used more than 53% of
the new x2 ceiling, even at today's x5-stretched duration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:53:08 +02:00
kevinveenbirkenbachandClaude Opus 5 6db3efdd87 perf(test-e2e-playwright): drop the fixtures copy the tree copy already made
Staging copied every role fixture twice. 3b60b36fb2 added the flat
fixtures fileglob when no recursive stager existed; 8e92f15f6a then
introduced the tree copy with a trailing-slash source, which subsumed it,
and the older pair stayed behind.

Both wrote to the same destination from the same source, so the second
copy reported ok rather than changed on every run. The mkdir preceding it
created an empty directory for 23 of 25 roles: only web-app-dashboard and
web-app-peertube ship a fixtures tree, and their specs reach into it
relatively from inside the copied tree.

Costs 21.17s per CI job before the retry-pass multiplier, which stages
web-app-nextcloud three times.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 15:16:02 +02:00
kevinveenbirkenbach 87f762bd75 fix(e2e): derive the shared Playwright helpers instead of listing them
A helper that no stager copies fails at import, and it fails for every
role that imports it at once, because the whole spec file stops loading.
That is what mcp-endpoint.js did to all ten MCP specs: they ran zero
tests rather than one failing one.

The set lived in three places and no two agreed - the deploy task named
four helpers, rerun-spec.sh named three, and the specs required five.
addon-gating.js was already missing from the rerun path before any of
this. Both stagers now glob the runner's own files/*.js, minus the
config, so the directory listing is the set.

The lint derives what must be staged from the requires the specs
actually make, and holds both stagers to globbing, since a set that can
be typed twice will differ twice.
2026-08-29 10:45:20 +02:00
kevinveenbirkenbachandClaude Opus 5 3b4bc4ee65 perf(playwright): stage the persona flows once per run, mount them read-only
The previous commit packed the personas into one archive but still transferred them per role. orchestrate.yml runs 01_setup.yml once and then loops 02_run_one.yml over every app plus a retry pass, which is where the 22 transfers came from. Moving the staging into the setup phase leaves exactly one.

Each container now mounts the shared tree read-only at /e2e/tests/personas, the same pattern node_modules already uses. Both import directions were checked: specs require ./personas from /e2e/tests and land in the mount, while personas/utils/env.js requires ../../timeouts and reaches two levels up into the writable per-app staging dir, where the shared helpers are still placed. The nested mount therefore does not cut the relative imports.

Measured saving is 4.1 of the 4.2 minutes the copy cost in odoo 0, since only one transfer remains.

Not verified: the nested bind mount was not executed. Docker orders mounts by destination depth so /e2e lands before /e2e/tests/personas, but if that ordering did not hold the spec would fail loudly on require, not silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:24:14 +02:00
kevinveenbirkenbachandClaude Opus 5 431ec56e14 perf(playwright): stage the persona flows in one transfer
Copying a directory with ansible.builtin.copy walks it file by file, one round trip each. Measured in the green onion job odoo 0 from the log timestamps: 22 invocations at 10 to 13 seconds, 4.2 minutes total, for 15 files totalling 108 kB - about 0.7 s per file, all of it connection overhead rather than payload.

The tree is now packed once on the controller and transferred as a single archive. Uncompressed on purpose: at this size gzip saves 28 kB and would add a second binary that has to exist on every distro image. tar itself was verified present in the CI fedora image and in the arch and centos node images.

Equivalence checked rather than assumed. The old copy used a trailing slash and therefore copied the contents; `tar -C personas .` does the same. Building the archive produces 17 entries - 15 files plus ./ and ./utils/ - so the subdirectory layout survives.

This continues 6e14d12b5, which collapsed the per-file tasks into one task. That left the per-file transfer inside that task, which is what this removes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:16:51 +02:00
kevinveenbirkenbach e04c448cdd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, none resolved by picking a side wholesale:

- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
  instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
  rc now counts as failure; the previous default(0) let an aborted Playwright
  run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
  which this branch's lint enforces, and adopted the incoming retry-inside-poll
  for the OIDC button plus its networkidle wait.
2026-07-30 15:38:15 +02:00
kevinveenbirkenbachandClaude Opus 5 7062d9c02d fix(test-e2e): report the abort instead of dying on it
An aborted Playwright run - watchdog SIGTERM, task timeout - returns a result dict with neither stdout nor rc. The reporting task dereferenced both, so it became the fatal error itself: the log ended on "object of type 'dict' has no attribute 'stdout'" while the real cause, a disk-exhaustion abort with rc 75, sat eight thousand lines earlier. Guard all three reads, fall back to msg for stderr because that is what Ansible fills on an abort, and treat an absent rc as a failure so the run is recorded and named by the aggregate check rather than vanishing behind a template error.

The distro sweep's summary also mislabelled the distros it never started: they kept the initial "not reached within the budget" note even when the sweep aborted on a failure with 12214 seconds of budget left, pointing at a budget problem that did not exist. Say what happened instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
kevinveenbirkenbachandClaude Opus 5 f5bafd377f fix(e2e): skip frontend-less roles in the readiness gate and stack config
seaweedfs disables its frontend on tor nodes by contract, which drops
the filer/master vhosts and leaves nginx.conf unrendered. The readiness
probe still hit the canonical filer domain and died on
TLSV1_UNRECOGNIZED_NAME, and the config mount kept nginx.conf in the
stack file, which docker stack deploy validates and rejects. Both now
carry the same frontend condition the role already renders into its
test env.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:51:17 +02:00
kevinveenbirkenbachandClaude Opus 5 6e14d12b59 perf(test-e2e-playwright): stage shared assets once per run
The Playwright image was pulled once per role, and the shared helpers
and persona modules were staged file by file. The pull now happens once
for the whole run, and the personas tree is copied in a single task.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:38 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5 407b8a8a1f fix(e2e): use the tls lookup instead of the unregistered urlsplit filter; autoformat
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 21:05:59 +02:00
kevinveenbirkenbachandClaude Fable 5 c4e41217c3 test(e2e): recover onion circuits via a Tor-guided wait script + retry goto timeouts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:39:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 88af0b883b fix(e2e): scale the Playwright global timeout for onion; hoist onion gate/proxy/timeout to vars constants
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 21:08:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7511852bd1 fix(e2e): never skip onion suites; unpin entry guards in test envs
The Playwright onion gate probed the app onion over Tor and silently
SKIPPED the whole suite when it did not answer — a broken onion could
go green. Remove the gate: the probe now waits hard (40x15s, covering
normal 2-5 min descriptor publication) and an unreachable onion fails
the deploy loudly. Reachability is part of what is under test.

The readiness probe treated transport-level failures (status: -1) as
'< 500', satisfying its until condition on attempt 1 and never using
its retries (nextcloud [0,1] died to TLSV1_UNRECOGNIZED_NAME this way).
Require a real HTTP status >= 100 so transport errors retry.

Tor pins one long-lived entry guard; its own log showed ours degraded
('failing more circuits than usual ... 38 collapsed, 21 timed out'),
which throttles every flow of the node at once — the most plausible
cause of the 89-minute descriptor-publication outage and the
intermittent ERR_SOCKS_CONNECTION_FAILED in E2E. New config key
services.tor.use_entry_guards (default true keeps the Tor default and
production anonymity posture); the matrix variant pins it false so test
deployments render UseEntryGuards 0 and a single bad relay cannot stall
the node. Anonymity is not under test there, only reachability.

Verified: full svc-net-tor closure deploy over onion with the gateless
semantics, failed=0, every onion suite executed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 20:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5abfcfc6d6 feat(e2e): shared Tor-aware Playwright harness
Promote the Firefox onion fixture (secure-context .onion prefs, SOCKS
remote-DNS) from web-app-dashboard to test-e2e-playwright as the single
source of truth; the dashboard copy now re-exports it. Stage the shared
fixture on all three paths (deploy runner, lint-playwright, rerun-spec).

gotoOnion: onion navigations default to waitUntil=domcontentloaded —
heavy chunked SPAs (Element) serialise 30+ bundle requests over Tor, so
the load event regularly exceeds the navigation cap; callers' explicit
onion-scaled selector waits cover app boot instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 10:56:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f8c016b294 fix(test-e2e-playwright): give onion targets a realistic navigation budget
gotoOnion hard-coded a 60s page.goto timeout, so a cold full-page load
over freshly built Tor circuits aborted before the onion answered and
the deploy gate flaked. The deploy now passes an onion-aware
PLAYWRIGHT_NAVIGATION_TIMEOUT (180s onion / 60s clearnet) plus a larger
onion per-test budget (600s), and gotoOnion honors the env instead of
the hard-coded ceiling. Fast loads are unaffected; only slow onion
navigations may use the larger budget.

Verified: full deploy matrix green with zero page.goto timeouts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 01:56:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c2fa2da88e style: apply make autoformat (whitespace only)
Indentation/whitespace normalization emitted by make autoformat across role meta/services.yml, meta/variants.yml and a task file. No semantic changes (verified: git diff --ignore-all-space is empty for every file). Kept separate from the Tor egress-router feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 08:22:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 897abcc28c feat(test-harness): route Tor tests via the SOCKS SPOT + secure the CLI stage
test-e2e-cli renders test.env at 0600 in a 0700 stage dir and always removes the stage tree (it holds base64 DB credentials). test-e2e-playwright + the e2e scripts resolve the node SOCKS from the single spot: playwright via lookup('tor_socks', 'host.docker.internal'), onion.sh takes it as a required arg, rerun-spec.sh composes it from INFINITO_TOR_SOCKS_PORT — no 9050 literal or backup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3f7face0aa feat(test-e2e-playwright): route .onion apps through Tor + defaults->vars refactor
Per-app PLAYWRIGHT_PROXY: an app whose canonical URL is a .onion (services.tor primary/exclusive) has its browser routed through the node Tor SOCKS proxy (socks5://host.docker.internal:9050); clearnet apps stay direct. A SOCKS readiness gate probes the hidden service and skips (not fails) the app's run when Tor cannot reach it. TEST_E2E_PLAYWRIGHT_PROXY is removed since the per-app logic supersedes the global knob. Non-overridable tuning constants moved defaults->vars (UPPERCASE); only genuinely overridable knobs (only_roles/skip_roles/keep/VARIANT_INDEX) remain in defaults.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e6538e4f61 feat(svc-net-tor): Tor onion-service provider (full onion node)
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.

- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
  key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
  INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
  .onion-identity/hs (idempotent). The onion is a deploy-time input;
  no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
  installs the minted keys, torrc with one node HiddenService + a SOCKS
  proxy restricted to loopback/DiD ranges. files/test.sh probes every
  served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
  cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
  skip the clearnet DNS sanity check for onion; raise nginx
  server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
  navigation/action timeouts so specs reach .onion over Tor and fail
  fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.

Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 12:38:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 96aa7784b6 feat(sys-svc-compose): scale all task timeouts via TIMEOUT_FACTOR lookup
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.

- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
  string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
  task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
  lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
  network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
  lint enforces whole-file on any edited file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e11bf6f31b refactor(lint): drop caveat/workaround comment markers
Remove caveat and workaround from the comment-lint marker vocabulary and delete
the comments that used them across the tree. Move test_comments_valid's own
git-safe.directory note from a marked comment into the function docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:54:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 23d1447566 refactor(ansible): add path_absolute lookup, replace playbook_dir joins
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:

  [playbook_dir, '<rel>'] | path_join   -> lookup('path_absolute', '<rel>')
  playbook_dir ~ '/<rel>'               -> lookup('path_absolute', '<rel>')
  {{ playbook_dir }}/<rel>              -> lookup('path_absolute', '<rel>')

Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.

Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.

Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36ebf34b64 Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:13:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6d417aa9a4 fix(test-e2e-playwright): keep reports on e2e timeout via a Playwright globalTimeout
The Ansible task timeout hard-kills the Playwright sidecar at TEST_E2E_PLAYWRIGHT_RUN_TIMEOUT, discarding every junit/html/trace and aborting the play before the archive step. Set Playwright's own globalTimeout a buffer (60s) below that cap so it self-terminates with reports, the task returns rc\!=0 instead of being killed, and the diagnostics get archived for the next run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:09 +02:00
kevinveenbirkenbach 21e484bf30 Merge origin/main into feature/026-unified-addon-syntax
Resolve conflicts: keep 026's bond/field/addon additions (services.yml, variants.yml,
yourls leaky-proxy fix, nextcloud per-file addons), take main's 027 integration-matrix
(generated artifact). Verified: all 44 main inline-plugins are covered by meta/addons/.
2026-06-21 06:32:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d2f5efa7fe feat(test-e2e-playwright): parallel workers via plugin + tunable timeouts; split role vars by precedence
Add plugins/lookup/playwright_workers.py computing a conservative parallel-worker count (CPU/RAM-derived, app-under-test concurrency-capped, reduced on CI) and wire test_e2e_playwright_workers + test_e2e_playwright_fully_parallel + test_e2e_playwright_test_timeout into playwright.config.js (read from env) and 02_run_one.yml (passed via -e). fullyParallel stays false so parallelism is file-level only and intra-file ordering assumptions stay safe. Raise run_timeout to 3600 for addon-heavy roles (nextcloud ~50 specs).

Split the role's variables by Ansible precedence and intent: derived/fixed constants live in vars/main.yml (UPPERCASE, high precedence, not operator-tuned), operator-tunable knobs live in defaults/main.yml (lowercase, overridable). VARIANT_INDEX stays UPPERCASE with a single definition. Update references across tasks/templates/inventory/docs accordingly. Includes a unit test for playwright_workers and an autoformat pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:03:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8e92f15f6a test(addons): require a Playwright spec for every addon; add 87 per-addon specs
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:16:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 43eb831de0 fix(e2e): run Playwright with --network host in swarm
In swarm the apps are reachable via the node host-mode proxy ports; share the node network namespace so the Playwright container resolves the canonical domains via the node dnsmasq and reaches them, instead of the compose --add-host host.docker.internal path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3b60b36fb2 fix(web-app-peertube): repair the SeaweedFS object-store e2e test for PeerTube v8.2.1
The test timed out: the publish-button locator no longer matched the v8.2.1
manage-video wizard, and the inline 78-byte MP4 stub was structurally invalid,
so PeerTube rejected the upload as "Video file unreadable" and left the Save
button disabled.

- test-seaweedfs.js: target the v8.2.1 save control (.save-button > button),
  upload PeerTube's real video_short.mp4 fixture, and assert a version-tolerant
  saved marker.
- add fixture roles/web-app-peertube/files/playwright/fixtures/video_short.mp4.
- test-e2e-playwright: stage role-local files/playwright/fixtures/ next to the
  tests (the *.js glob ships neither binaries nor subdirectories), so specs load
  a fixture via path.join(__dirname, "fixtures", <name>).
- docs: document the fixtures/ folder and the per-scenario test-*.js split in the
  playwright spec contract and the runner README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 70ad33bc13 fix(web-app-peertube): repair the SeaweedFS object-store e2e test for PeerTube v8.2.1
The test timed out: the publish-button locator no longer matched the v8.2.1
manage-video wizard, and the inline 78-byte MP4 stub was structurally invalid,
so PeerTube rejected the upload as "Video file unreadable" and left the Save
button disabled.

- test-seaweedfs.js: target the v8.2.1 save control (.save-button > button),
  upload PeerTube's real video_short.mp4 fixture, and assert a version-tolerant
  saved marker.
- add fixture roles/web-app-peertube/files/playwright/fixtures/video_short.mp4.
- test-e2e-playwright: stage role-local files/playwright/fixtures/ next to the
  tests (the *.js glob ships neither binaries nor subdirectories), so specs load
  a fixture via path.join(__dirname, "fixtures", <name>).
- docs: document the fixtures/ folder and the per-scenario test-*.js split in the
  playwright spec contract and the runner README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:07:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 05f6a97bd8 refactor(stack_host_template): generalise IS_STACK_HOST template wrapper across the codebase
Renames the local action plugin from `conf_template` to
`stack_host_template` so the name carries the actual semantics
(IS_STACK_HOST gate, not nginx config), broadens the lint guard to
every bare `template:` task, and migrates every consumer found in the
audit.

Plugin / tests / lint:
  * plugins/action/conf_template.py -> plugins/action/stack_host_template.py
    (drop nginx-specific docstring; trust_as_template gate unchanged)
  * test_conf_template.py            -> test_stack_host_template.py
  * test_conf_template_migration.py  -> test_stack_host_template_migration.py
    Scope expanded from `.conf.j2` to every `template:` task; marker
    renamed to `nocheck: stack-host-template`.

Migrations (~50 tasks now using stack_host_template:):
  * sys-svc-compose/04_files.yml: compose.yml, .env, Dockerfile,
    compose.override (every per-app compose-tree file)
  * web-app-keycloak: sso_proxy upstream-config (fixes the friendica
    CI failure that prompted this refactor) + realm-import
  * web-app-taiga, web-app-matrix, web-app-opentalk, web-app-pgadmin,
    web-app-moodle, web-app-listmonk, web-app-littlejs, web-app-fediwall,
    web-app-wordpress, web-app-discourse, web-app-bigbluebutton,
    web-app-jenkins, web-app-jitsi, web-app-mobilizon, web-app-suitecrm,
    web-app-decidim, web-app-dashboard, web-app-nextcloud, web-app-xwiki,
    web-app-mailu, web-app-magento, web-app-odoo, web-app-prometheus,
    web-app-matrix, web-app-peertube, web-app-mailu (+ existing `conf_template:`
    callers updated to the new name)
  * sys-front-tls, svc-db-openldap, sys-svc-webserver-core,
    sys-svc-rdbms (dedicated DB env + initdb), sys-front-inj-css,
    sys-front-inj-dashboard, sys-front-inj-all, sys-stk-front-proxy,
    web-svc-cdn-css, web-svc-xmpp, web-svc-legal, web-svc-simpleicons,
    web-opt-rdr-domains

Nochecks (per-host or already-structurally-stack-host):
  * sys-svc-mail-smtp/postfix, sys-svc-sshd, sys-svc-container/daemon.json
  * sys-timer, sys-service, sys-svc-mail-msmtp, sys-svc-journalctl,
    sys-daemon, sys-svc-compose/utils/_render_one_template (delegate_to
    stack host with peer slurp dance)
  * dev-locales, dev-nix, desk-*, user, user-backup, user-workstation
  * svc-storage-nfs-server/exports, test-e2e-playwright stage dirs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 15:45:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9d8a4155a2 fix(test-e2e-playwright): stat repo paths on controller, not on STACK_HOST
The role's block-level `delegate_to: STACK_HOST` made five `stat:` tasks
run on the manager, which has no repo checkout — `playbook_dir`-based
paths always returned `exists=false` and the subsequent `fail:` tasks
aborted runs like web-app-baserow / web-app-akaunting with
"Missing Playwright env template". Override delegate_to to localhost on
each repo-path stat so the existence check runs where the files
actually live; CA-cert stats keep their STACK_HOST scope because those
files belong to the host.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 20:57:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8ab37347b1 feat(test-e2e-playwright): add host.docker.internal:host-gateway to sidecar runs
Gives Playwright sidecars a stable name for the docker host so role specs
can reach services whose port is bound on the host (e.g. Mailu's SMTP
submission on 465/587 in the Zammad email-channel test) without
attaching to that service's docker network.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:53:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5196283f6c refactor(test-e2e-playwright): bake @playwright/test into a shared node_modules
Per-role Playwright runs no longer call `npm install` against
registry.npmjs.org on every invocation. A new `01_setup.yml` runs once
per host/version, populates `_shared/node_modules/` inside the Playwright
container, and stamps the installed version in `.installed-version`.
Each `02_run_one.yml` mounts that tree read-only at `/e2e/node_modules`
and the container command becomes `npx --no-install playwright test`.
This eliminates the npmjs.org TLS-handshake flake that took out the
keycloak playwright in a recent variant-2 update, saves ~4 s per role,
and gives version bumps a single rebuild point.

Three retry layers cover transient registry failures: Ansible (3× with
5 s delay), npm fetch (5× with 10-60 s backoff), and image pull (3×).

Stage-base / shared-dir / stamp / node_modules paths live as SPOT
constants in `vars/main.yml` so call sites never reconstruct them.

Renames `tasks/run_one.yml` → `tasks/02_run_one.yml` (and adds
`01_setup.yml`) to make the include order visible by filename; updates
cross-references in `AGENTS.md`, the two `docs/contributing/...` pages,
`dump_disk_diagnostics.sh`, and the lint test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 11:23:24 +02:00