Commit Graph
28 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 55ef9647d1 refactor(test-e2e-playwright): drop the orphaned lowercase wait keys
defaults/main.yml declared test_e2e_playwright_wait_enabled, _retries and
_delay with param docs. Nothing read them: 02_run_one.yml uses the uppercase
TEST_E2E_PLAYWRIGHT_WAIT_* names from vars/main.yml as retries, delay and in
the readiness condition.

These were not overridden defaults but a separate, unread namespace, so they
read as knobs while turning nothing. After removing them the lowercase names
have no occurrence left in the repository, while the uppercase ones are used
in three places.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:00:17 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 97fae6d3db fix(e2e): resolve the deployed role set from the inventory, not the spec list
Both halves of the Playwright harness used "ships a Playwright suite" as a
proxy for "is deployed". The proxy is structurally wrong in opposite
directions, and each error hid itself.

discover_playwright_roles.py finds roles by globbing
`templates/playwright.env.j2`, so a role without that template can never
appear in TEST_E2E_PLAYWRIGHT_APPS no matter how deployed it is.
addon_env_flags resolved its bridge-partner check against exactly that list,
so every addon declaring `bridges: [litellm]` (5) or `bridges: [ldap]` (4)
gated itself off: svc-ai-litellm and svc-db-openldap ship no suite. The
affected specs reported as skipped, which is indistinguishable from passing
in a `failed=0` PLAY RECAP -- discourse-ai, AIEditingAssistant and ai-engine
have never once executed. Measured: `ls roles/svc-ai-litellm/files/playwright`
does not exist and the role is absent from the runner's "Playwright-enabled
roles" line, while `grep svc-ai-litellm ~/inventories/localhost-2/devices.yml`
returns a group entry. Two sources, opposite verdicts.

The gate now reads `group_names`, the round's actual closure. It stays
fail-open when that cannot be resolved: `_resolve_deployed_roles` returns None
for an empty set, so contexts without an inventory skip bridge gating exactly
as before rather than flipping every bridged addon false. The change is
monotone -- TEST_E2E_PLAYWRIGHT_APPS is the discovered roles intersected with
`running`, and `running` is a subset of `groups`, so a flag can only move
false -> true and no spec that ran before is newly skipped.

The second half is the mirror image. test_e2e_playwright_only_roles defaulted
to `lookup('deployment').groups`, the whole round closure, so `apps=` deploying
three roles still ran every spec in the round; specs for roles left on stale
state then failed and masked the result of the deploy that actually ran. It is
`.running` now: the whitelist when a targeted deploy sets one, the inventory
groups otherwise, leaving full runs unchanged.

The two lookups deliberately pull in opposite directions. Addon gating asks
whether the partner exists anywhere in the deployment and needs the full
closure; run scoping asks what was just deployed and needs the subset.
Conflating them was the shared defect.

defaults/main.yml keeps the same knobs and values apart from only_roles; its
mid-file narration moved into a Param header so the comments policy passes on
a file that is now touched.

Verified: 23 unit tests including a guard that fails on the old path
(group_names unread -> gating skipped -> true, where the test demands false),
and `make quality` green on test-lint, test-unit, test-integration and
test-external, read from the per-target table rather than the exit code.
Not yet verified live: the three unskipped specs have still never run, and
`group_names` is per-host, so a swarm delegate whose membership differs from
the stack host is unexercised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 12:27:51 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3f7face0aa feat(test-e2e-playwright): route .onion apps through Tor + defaults->vars refactor
Per-app PLAYWRIGHT_PROXY: an app whose canonical URL is a .onion (services.tor primary/exclusive) has its browser routed through the node Tor SOCKS proxy (socks5://host.docker.internal:9050); clearnet apps stay direct. A SOCKS readiness gate probes the hidden service and skips (not fails) the app's run when Tor cannot reach it. TEST_E2E_PLAYWRIGHT_PROXY is removed since the per-app logic supersedes the global knob. Non-overridable tuning constants moved defaults->vars (UPPERCASE); only genuinely overridable knobs (only_roles/skip_roles/keep/VARIANT_INDEX) remain in defaults.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e6538e4f61 feat(svc-net-tor): Tor onion-service provider (full onion node)
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.

- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
  key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
  INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
  .onion-identity/hs (idempotent). The onion is a deploy-time input;
  no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
  installs the minted keys, torrc with one node HiddenService + a SOCKS
  proxy restricted to loopback/DiD ranges. files/test.sh probes every
  served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
  cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
  skip the clearnet DNS sanity check for onion; raise nginx
  server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
  navigation/action timeouts so specs reach .onion over Tor and fail
  fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.

Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 12:38:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36ebf34b64 Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:13:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cf765ee00c refactor: single /var/lib SPOT + swarm Playwright artifact upload
SPOT: declare the var-lib base once in default.env as INFINITO_DIR_VAR_LIB; the env build derives INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR from it; reference it everywhere (utils/paths.py, scripts via ${...:?}, tests, the dev inventory's DIR_VAR_LIB injection) instead of hardcoding the literal. group_vars keeps the literal default; the env-lookup lives in the dev inventory per the no-ansible-env-lookups lint rule. pull_specific_host.py --folder is now required and the role passes {{ DIR_BACKUPS }}.

Swarm e2e: collect_playwright_reports.sh docker-cp's the reports off the manager plus a workflow upload step, so swarm Playwright traces reach CI artifacts (was compose-only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6d417aa9a4 fix(test-e2e-playwright): keep reports on e2e timeout via a Playwright globalTimeout
The Ansible task timeout hard-kills the Playwright sidecar at TEST_E2E_PLAYWRIGHT_RUN_TIMEOUT, discarding every junit/html/trace and aborting the play before the archive step. Set Playwright's own globalTimeout a buffer (60s) below that cap so it self-terminates with reports, the task returns rc\!=0 instead of being killed, and the diagnostics get archived for the next run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d2f5efa7fe feat(test-e2e-playwright): parallel workers via plugin + tunable timeouts; split role vars by precedence
Add plugins/lookup/playwright_workers.py computing a conservative parallel-worker count (CPU/RAM-derived, app-under-test concurrency-capped, reduced on CI) and wire test_e2e_playwright_workers + test_e2e_playwright_fully_parallel + test_e2e_playwright_test_timeout into playwright.config.js (read from env) and 02_run_one.yml (passed via -e). fullyParallel stays false so parallelism is file-level only and intra-file ordering assumptions stay safe. Raise run_timeout to 3600 for addon-heavy roles (nextcloud ~50 specs).

Split the role's variables by Ansible precedence and intent: derived/fixed constants live in vars/main.yml (UPPERCASE, high precedence, not operator-tuned), operator-tunable knobs live in defaults/main.yml (lowercase, overridable). VARIANT_INDEX stays UPPERCASE with a single definition. Update references across tasks/templates/inventory/docs accordingly. Includes a unit test for playwright_workers and an autoformat pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:03:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 fc2ac7597a refactor(lookup): rename status -> deployment, add runtime-aware deployed
`status` was too vague for what the lookup returns (apps in current
play). Rename to `deployment` and add two new fields:

* `runtime` — RUNTIME variable value
* `deployed` — runtime-aware authoritative set: equals `running`
  (whitelist or groups) for ephemeral CI runtimes (act, github),
  falls back to `groups` for persistent runtimes (host, dev). This
  matches reality: in CI the container is fresh so only whitelisted
  apps exist; in production a partial deploy still leaves prior
  apps live, so groups is the right set.

Switch the two webserver/csp healthchecks to `lookup('deployment').deployed`
so they only probe domains for apps that actually exist in the current
deploy. In production behaviour is unchanged.

Update remaining consumers (cli/build/include, web-app-keycloak,
test-e2e-playwright) for the rename. Existing `status` semantics
(`whitelist`, `running`, `groups`, `all`) preserved under the new
name.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5196283f6c refactor(test-e2e-playwright): bake @playwright/test into a shared node_modules
Per-role Playwright runs no longer call `npm install` against
registry.npmjs.org on every invocation. A new `01_setup.yml` runs once
per host/version, populates `_shared/node_modules/` inside the Playwright
container, and stamps the installed version in `.installed-version`.
Each `02_run_one.yml` mounts that tree read-only at `/e2e/node_modules`
and the container command becomes `npx --no-install playwright test`.
This eliminates the npmjs.org TLS-handshake flake that took out the
keycloak playwright in a recent variant-2 update, saves ~4 s per role,
and gives version bumps a single rebuild point.

Three retry layers cover transient registry failures: Ansible (3× with
5 s delay), npm fetch (5× with 10-60 s backoff), and image pull (3×).

Stage-base / shared-dir / stamp / node_modules paths live as SPOT
constants in `vars/main.yml` so call sites never reconstruct them.

Renames `tasks/run_one.yml` → `tasks/02_run_one.yml` (and adds
`01_setup.yml`) to make the include order visible by filename; updates
cross-references in `AGENTS.md`, the two `docs/contributing/...` pages,
`dump_disk_diagnostics.sh`, and the lint test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 11:23:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 79faaf21e7 refactor(ansible): move INFINITO_* env lookups out of group_vars/roles into dev inventory
\`lookup('env', 'INFINITO_*')\` calls in group_vars / roles couple
deployment-agnostic Ansible code to default.env, which only exists in
dev/CI. Production deploys never source .env, so the lookups silently
fall back to their group_vars defaults and the inventory's intent gets
lost on the way (see the IP4 host_vars-overlay incident fixed in
8980b00b1).

Migrated keys (group_vars/roles -> inventory):
- DOMAIN_PRIMARY            (00_general.yml)
- BUILDKIT_ENABLED          (00_general.yml)
- CA_CONTAINER_ENABLED      (02_tls.yml)
- defaults_networks.internet.{ip4,ip6,dns}  (08_networks.yml)
- TEST_E2E_PLAYWRIGHT_KEEP  (roles/test-e2e-playwright/defaults/main.yml)

group_vars and the role default now carry literal fallback values;
the dev/CI inventory at \`inventories/development/default.yml\` pulls
the live values from .env via straight \`lookup('env', 'INFINITO_*')\`
calls without any \`| default(...)\` filter, because load.sh guarantees
the keys are set whenever .env exists.

The convention is enforced by the new lint test
\`tests/lint/filesystem/env/test_no_ansible_env_lookups.py\`, which
scans group_vars/, tasks/, roles/, and playbook.yml for any
\`lookup('env', 'INFINITO_*')\` against a default.env key and fails
with a pointer to the dev inventory as the injection site.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 17:57:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a77a570409 refactor(env): consolidate INFINITO_* namespace and centralize path/image SPOTs
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:

* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
  env var → defaulted to infinito.localhost, which never resolved
  because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
  INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
  (env-builder, role default, run_one.yml docker exec, playwright.config.js,
  lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
  the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
  compose forwards it as build-arg, volume mount path, and container env;
  every Python CLI + shell helper + CI workflow now reads it strictly
  (os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
  /opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
  INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
  utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
  container) through compose.yml's environment block.

Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 02:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0aa34b903e feat(playwright): persist artifacts per variant+pass, add Time column
- Matrix-deploy orchestrator now pushes `VARIANT_INDEX` (0-based round
  id) alongside the existing `ASYNC_ENABLED` extra var for both PASS 1
  (sync) and PASS 2 (async) runs. Test-e2e-playwright's archive step
  consumes both to derive a per-(variant, pass) destination so a
  subsequent round cannot overwrite the previous one's reports.
- New `roles/test-e2e-playwright/files/archive_playwright_reports.sh`
  moves the playwright-junit.xml / playwright-report/ / test-results/
  outputs into `variant-<N>/<sync|async>/` under the role's reports
  dir, idempotently (re-running the same tuple replaces rather than
  fails). `run_one.yml` invokes the script via `ansible.builtin.script`
  so the YAML stays a single 7-line task.
- `VARIANT_INDEX: 0` lives in `defaults/main.yml` to keep the variable
  defined for dev / non-matrix invocations. Role defaults are not
  considered SPOT, so the `| default(0)` belt-and-suspenders inside
  the script invocation stays valid against the spot-default lint.
- Summary script now renders an 8-column table: Time | App | Variant |
  Pass | Test | Status | Duration | Message. Per-test ISO timestamps
  are derived from the JUnit `<testsuite timestamp>` attribute plus the
  cumulative duration of preceding cases; file mtime is the fallback
  when the attribute is absent. App / Variant / Pass are extracted
  from the artefact path layout produced by the archive script.
- Matrix-deploy unit tests refreshed to expect the new
  `extra_ansible_vars` shape; summary unit tests gain coverage for
  `_to_iso_z`, `_parse_iso_timestamp`, the four supported path
  layouts, and the file-mtime fallback (55 tests total).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 15:30:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1f665b8bcc chore(test-e2e-playwright): trim role description; widen ONLY_ROLES to all host groups
- meta/main.yml: cut the wordy auto-discovery description to a one-liner ("Runs end-to-end Playwright tests against deployed applications.").
- defaults/main.yml: `TEST_E2E_PLAYWRIGHT_ONLY_ROLES` now reads `lookup('status').groups` (the full host-deployed set) instead of `lookup('status').running` (the CLI `--id`-narrowed subset). Playwright now exercises every dashboard-discoverable app on the host regardless of the deploy's `--id` scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:24:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 944e782d84 refactor(applications): introduce status lookup + rbac_app_ids filter; rename allowed_applications → APPLICATIONS_WHITELIST
- plugins/lookup/status.py: new lookup returning {whitelist, running, groups, all}, cached per (whitelist, groups). Consumers replace direct group_vars references.
- web-app-keycloak/filter_plugins/rbac_per_app.py: add `rbac_app_ids` (intersect input list with invokable apps; sorted, deduped).
- web-app-keycloak/tasks/update/05c_per_app_mappers.yml: drop two set_fact tasks; consume KEYCLOAK_RBAC_{ALL,RUN}_APP_IDS as constants from vars/main.yml. Split orphan-purge scope (all host-deployed) from mapper-sync scope (current run).
- Rename Ansible-side `allowed_applications` → `APPLICATIONS_WHITELIST` (group_vars/all default, deploy CLI `-e` injection, generated include `when:` clause, debug task, test asserts). Python kwarg name unchanged (internal contract).
- test-e2e-playwright: `TEST_E2E_PLAYWRIGHT_ONLY_ROLES` consumes `lookup('status').running`; `TEST_E2E_PLAYWRIGHT_APPS` moved from set_fact to vars/main.yml constant.
- 01_constructor debug: collapse to single `status:` block over the lookup result.
- web-app-{opencloud,opentalk}/vars: drop dead `group_names | default([])` + fix latent `| lower`-on-list precedence bug.
- New lint `test_no_jinja_default_on_ansible_magic_var.py`: forbid `| default(...)` on Ansible magic vars (group_names, inventory_hostname, …); meta-scan + 7 fixture positive cases.
- Unit tests for `rbac_app_ids` (10) and `status` lookup (8).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:39:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d65d9b6765 fix(test-e2e-playwright): cap Playwright sidecar runtime + block generic personas on Friendica
Two changes that together stop one badly-behaved role from stalling
the whole deploy:

1. `roles/test-e2e-playwright/tasks/run_one.yml` — pass `timeout:
   {{ TEST_E2E_PLAYWRIGHT_RUN_TIMEOUT }}` to the `command` task. New
   default `1200` (20 min) lives in `defaults/main.yml`. Without an
   outer Ansible-side cap, a single hung test could burn 5 min ×
   3 retries × N specs before either `--max-failures` or the
   deploy supervisor noticed; friendica's run today sat at 27 min
   before the operator killed it manually.

2. `roles/test-e2e-playwright/defaults/main.yml` — append
   `--max-failures=5` to the default Playwright command. Once five
   tests have failed, the sidecar stops instead of chewing through
   the rest of the retry budget for tests that are obviously broken.

3. `roles/web-app-friendica/templates/playwright.env.j2` — declare
   `PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`.
   Friendica authenticates against its own local DB / LDAP via the
   `ldapauth` addon, not OIDC. The generic `runBiberFlow` /
   `runAdminFlow` helpers drive an OAuth2-proxy + Keycloak round-trip
   that Friendica cannot reconcile, so the generic persona tests
   wedge on the Keycloak sign-in form. The bespoke admin/biber tests
   above own the actual login flow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 576fad84ff refactor(images): move role images from defaults/main.yml to meta/services.yml
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.

`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 14700e0658 feat(test-e2e-playwright): add PLAYWRIGHT_KEEP_ALL artefact-retention toggle
Forwards `PLAYWRIGHT_KEEP_ALL` from host env / GitHub Actions repo var
through `TEST_E2E_PLAYWRIGHT_KEEP_ALL` (inventory-overridable) into the
container and flips trace / screenshot / video to `on` for every test
when truthy.

SPOT for the propagation chain lives in
`docs/contributing/actions/testing/playwright.md#artefact-retention-`;
CI Configuration and the agent iteration loops link to it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:31 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4350e2f993 chore(vars): prune dead vars and add nocheck markers (lint cleanup)
Resolve every remaining mismatch flagged by the new
test_role_and_group_vars_used lint:

- group_vars/all/02_tls.yml — mark TLS_SELFSIGNED_SCOPE as
  `nocheck: unused-var`; the self-signed cert generator consumes it
  outside the YAML/Jinja scope.
- group_vars/all/08_networks.yml — mark PORT_BANDS as
  `nocheck: unused-var`; the port-allocation plugin reads it via
  Python, not YAML.
- roles/sys-stk-front-proxy/vars/main.yml — delete the role-local
  one-liner whose only var (`front_proxy_domain_conf_dst`) was never
  consumed.
- roles/test-e2e-playwright/defaults/main.yml — drop
  TEST_E2E_PLAYWRIGHT_IMAGE_DISTRO; the image-derivation comment
  that referenced it is gone too.
- roles/web-app-opentalk/templates/playwright.env.j2 — switch
  OPENTALK_OIDC_ENABLED to render the inventory-resolved var
  directly instead of duplicating the lookup.
- roles/web-app-discourse/tasks/04_network.yml — trailing-newline
  fix (incidental autoformat).

After this commit the lint passes cleanly across the role tree and
the two `group_vars` markers stand as the single canonical place a
contributor finds when researching "why is X exempt".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 11:28:22 +02:00
Environment TestandClaude Sonnet 4.6 c47ac1c4cb feat(playwright): make images.playwright.version the SPOT for version
- Add image_version filter plugin to extract bare semver from image tag
- Derive TEST_E2E_PLAYWRIGHT_VERSION via image_version filter
- Convert files/package.json to templates/package.json.j2 using the
  derived version so image tag and devDependency stay in sync
- Use lookup('image', 'playwright', 'ref') in run_one.yml instead of
  building the image ref manually from package.json
- Add unit tests for image_version filter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 12:50:42 +02:00
Environment TestandClaude Sonnet 4.6 aa33256cde feat(mirror): split resolver output, move images to defaults, add lookup plugin
- resolver: output both applications: (config/main.yml compose images) and
  images: (defaults/main.yml images) keyed by source_file
- image_discovery: scan defaults/main.yml instead of vars/main.yml for images:
- mirror_overrides: write mirror URLs to applications.*.compose.services.*
  (mirror_policy-aware) and images_overrides.*.* (always merged, if_missing)
- roles/sys-ctl-hlth-csp, roles/test-e2e-playwright: move images: from
  vars/main.yml to defaults/main.yml for correct Ansible variable precedence
- plugins/lookup/image.py: new Ansible lookup plugin that prefers
  images_overrides over role-local images defaults
- docs: add artefact/image.md and artefact/mirror.md as SPOTs; add
  code/role/image.md contributor guide; link from ci.md
- tests: update test_util, test_mirror_overrides; add test_resolver,
  test_publish (404/401 coverage), test_image (lookup plugin)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 11:17:42 +02:00
kevinveenbirkenbach aef39f7542 Refactor central Playwright runner files 2026-03-18 22:16:46 +01:00
kevinveenbirkenbach 1f9a0d9778 feat(test-e2e-playwright): finalize role-based playwright runner and tests 2026-03-17 03:57:35 +01:00
kevinveenbirkenbach f482e7e63a Replace Cypress E2E role with Playwright 2026-03-17 01:57:57 +01:00