CI run 25774452286 surfaced two distinct Discourse failures that share
the same root cause: variant 1's `postgres.shared: false` override
forced Discourse's launcher to fall back to its bundled postgres
image, which is vanilla and lacks the `vector` extension. Discourse's
`20230710171141 EnablePgVectorExtension` migration then aborted
`bundle exec rake db:migrate` with `Pups::ExecError` → `FAILED TO
BOOTSTRAP` (retried 4×), leaving the deploy red.
The central `svc-db-postgres` image, by contrast, is built with
pgvector compiled in (`roles/svc-db-postgres/templates/Dockerfile.j2`
clones github.com/pgvector/pgvector and `make install`s it). Pinning
`services.postgres.shared: true` in Discourse's `meta/services.yml`
and dropping the `postgres:` override from `meta/variants.yml` keeps
every variant on the pgvector-equipped central server, so the
migration succeeds unconditionally.
The second failure surfaced once the deploy got further: variant 1
also disables OIDC (`services.oidc.{enabled,shared}: false`), but the
spec's `administrator: dashboard to discourse OIDC login and logout`
and `biber: dashboard to discourse OIDC login and logout` ran
unconditionally, hitting 1.0-minute Playwright timeouts driving an
OIDC sign-in flow that has nothing to authenticate against. Add a
`test.skip(... OIDC_SERVICE_ENABLED ... \!== "true", ...)` guard at the
top of both tests so they skip cleanly when OIDC is disabled by
variant; the third scenario (`guest: public-landing → auth chain →
never authenticated`) does not need a guard because it only checks an
unauthenticated visit, which is meaningful with or without OIDC.
Local FULL_CYCLE PASS 1+2 succeed for both declared variants (v0+v1).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`INFINITO_PACKAGE_CACHE_MAX_AGE_MIN` was 129600 (90 days), 13× longer
than the 7-day `Valid-Until` window Debian and Ubuntu generate by
default in their apt `Release` files. Nexus held cached `Release`
manifests for 90 days, so once the upstream `Valid-Until` elapsed the
manifest stayed in the cache for another ~83 days and every Docker
build that ran `apt-get update` against the package-cache aborted with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
Drop the default to 8640 minutes (6 days). Stays one day below the
7-day `Valid-Until` so Nexus always re-validates the cached `Release`
against upstream before apt would see it expired. The single value
still applies to every proxy repo (apt, yum, pypi, npm, gem, go, helm,
raw-*), but the `Valid-Until` enforcement only exists in apt — for
the other repos the change merely makes the cache 84 days fresher
instead of staler, no regression.
Update both docs sites that document the default:
- docs/contributing/artefact/files/compose.yml.md (env-var table)
- docs/contributing/environment/cache.md (operator narrative)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the registry-cache or package-cache (Nexus / nginx frontend)
serves apt repository metadata whose `Valid-Until` window has elapsed,
Docker builds inside an Ansible-driven deploy abort with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
which surfaces as a generic non-zero exit from the wrapped deploy
command and forces the operator to manually wipe the caches before the
next attempt.
Add `scripts/tests/deploy/local/utils/cache-retry.sh` exposing
`deploy_with_cache_retry <label> -- <cmd> [args...]`. The helper runs
the wrapped command with combined stdout+stderr teed to a temp log,
captures the wrapped command's real exit code via PIPESTATUS (with a
disciplined `set +e` / restore-previous-state dance so the caller's
`set -e` does not abort the script before the helper can react), and
on non-zero exit grep's the captured log for the stale-cache
signature `Release file ... is expired` / `Valid-Until ... expired`.
If matched, the helper wipes both cache stacks via `make cache-clean`
(registry-cache + package-cache through
scripts/system/cache/clean.sh), runs `docker builder prune -af` and
`docker image prune -af` to drop cached build layers and unused base
images, and re-runs the wrapped command exactly once. Non-stale-cache
failures bubble up unchanged so the caller's existing error handling
is preserved. Live in the deploy/update script layer rather than the
Ansible handler because only the host-side scripts can reach the
registry-cache + package-cache containers and the host Docker engine.
Wire the helper into all five local deploy entrypoints:
- fresh-purged-app.sh (both `--full-cycle` and the sync-only branch)
- fresh-kept-app.sh
- fresh-kept-all.sh
- reuse-kept-app.sh (added SCRIPT_DIR/REPO_ROOT locally because the
script did not yet have them)
- reuse-kept-all.sh
CI deploy paths (scripts/tests/deploy/ci/*) are intentionally left
alone — CI runs do not share the developer's local cache stack and
they handle stale metadata differently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
09_deploy_reuse.sh ran `make deploy-reuse-kept-apps APPS=web-app-matomo`
without VARIANT, which makes reuse-kept-app.sh fall back to the
unsuffixed `${INVENTORY_DIR}` path. The matrix-init in step 08 only
writes `${INVENTORY_DIR}-0` / `-1` because matomo declares 2 variants
(see roles/web-app-matomo/meta/variants.yml; planner contract:
cli/administration/deploy/development/inventory/planner.py:107), so
the reuse pass always missed the inventory and aborted with exit 2.
Also surface sibling inventory dirs as suggestions when the lookup
misses, so the next caller hitting this gets a hint instead of a bare
ENOENT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The websocket service in templates/compose.yml.j2 rendered both the
dmbs_excl.yml.j2 include (emits `depends_on:` + DB/redis deps) AND a
manual `depends_on:` block on the same service, producing
`failed to parse compose.yml: yaml: mapping key "depends_on" already
defined` from `compose config --quiet` in CI run 25774452286. Replace
the manual block with the dmbs_incl.yml.j2 include and append the
ESPOCRM_SERVICE dep below it so all deps live under a single
depends_on key. Apply the same dmbs_incl pattern to the daemon service
for consistency — daemon now also waits for the central DB and redis
in addition to the main espocrm container.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strike through every role whose deploy passed in CI Run 25774452286 and
mark all declared variants ✅: 31 web-app-* roles (akaunting, bigbluebutton,
bridgy-fed, chess, dashboard, decidim, flowise, funkwhale, gitlab, hugo,
jenkins, joomla, keycloak, lam, littlejs, mailu, mastodon, matomo, mig,
mini-qr, mobilizon, moodle, opencloud, opentalk, pgadmin, phpmyadmin,
pretix, prometheus, roulette-wheel, sphinx, yourls) plus web-svc-cdn,
-libretranslate, -simpleicons, -xmpp. Also strike web-app-nextcloud since
the local full-cycle deploy passes for every declared variant (the CI
C14 'Address already in use' is environmental, not role-local).
Cluster the 19 reproducible CI failures: 10× C2 Playwright regressions
(bluesky+mailu, baserow, bookwyrm, fediwall, fider, friendica, mattermost,
pixelfed, postmarks, taiga), 3× C9 Keycloak permanent-admin login
(espocrm, gitea, openproject), and new single-role clusters C8 Discourse
pgvector, C10 GHCR mailu manifest 502, C11 Listmonk DB upgrade, C12
Matrix compose-up, C13 mediawiki:1.45 image missing; C6 peertube PG
reproduces. Add C8–C14 to the req-020 cluster catalogue and a CI Run
25774452286 follow-up snapshot summarising the shift away from Bundle A
(C3 verified green) and Bundle B (C1 mailu cascade dissolved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous SAN-drift gate compared `wanted | difference(existing)` and only
regenerated the cert when a new SAN appeared. When SANs were *removed* between
matrix-deploy rounds (e.g. step 08 of the environment suite drops dashboard
after round 0 and round 1 has only matomo+infinito), `wanted ⊂ existing` made
the difference empty, the cert kept its stale SAN list including
`dashboard.infinito.example`, the TLS handshake for the probe succeeded, and
nginx answered with the first defined vhost — 301 instead of the expected 000.
That failure manifests as the 5 red `🧑💻 Test Environment (...)` jobs on
CI Run 25774452286.
Both `_ss_wanted_sans` and `_ss_existing_sans` are already normalised the
same way (`map('lower') | list | unique | sort`), so a direct `\!=` is a clean
set-equality check that catches additions and removals alike.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The role's compose template pinned 6 containers to specific IPs in the
192.168.102.64/28 subnet (proxy=.67, talk=.68, nextcloud=.69, cron=.70,
whiteboard=.71, talk_recording=.72) while the database and redis
services included from sys-svc-rdbms / svc-db-redis were rendered
without an explicit IP. On a fresh variant-1 round, docker auto-assigned
the database to .67 first (it appears before the proxy in the rendered
compose.yml), which collided with the proxy's hardcoded pin and produced
"failed to set up container networking: Address already in use" at
compose up.
Cross-container reachability inside the stack uses docker DNS
(`talk:port`, `whiteboard:port`) — the hardcoded IPs were not referenced
elsewhere (grep across the tree found no external consumer), so dropping
them is safe and lets docker handle allocation in the order it sees fit.
Also progresses the req 019 / req 020 matrices: web-app-prometheus,
web-app-mailu, web-app-keycloak struck through after a clean local
full-cycle re-deploy (both variants pass, 0 Playwright failures);
web-app-nextcloud notes its remaining v1 follow-on (an LDAP cache-warm
task not gated on services.ldap.enabled).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups after c7bacbab5 removed `inventory/variants.py`:
* docs/contributing/design/variants.md and docs/requirements/020-*.md
still referenced the deleted submodule, breaking
`test_markdown_relative_links_resolve`. The design doc drops the
removed entries from the reference table; the postmortem turns the
historical mention into plaintext with a pointer to the removal
commit and the current package layout.
* tests/lint/repository/test_markdown_links.py and
test_project_root_import.py walked the worktree via
`iter_project_files`, which does NOT honour `.gitignore`. In the
`make exec` container path the markdown lint's primary
`git ls-files` codepath fails and falls back to that walker, surfacing
19 broken links inside vendored skill READMEs under `.agents/` and
`.claude/skills/` plus 2 `parents[N]` violations in installed skill
scripts. Both lints now use `iter_non_ignored_files` so gitignored
trees (skills install output, build artefacts) are skipped in every
environment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 16f323319 added a strict variant-only include rule + conflict
detector in `inventory/variants.py`, gated behind the presence of any
primary's `meta/variants.yml`. In practice the rule never paid off:
* The CI failure that motivated it (matomo's matrix transition leaving
a stale dashboard nginx vhost -> 502) is already addressed by the
nginx-purge primitive shipped in commit bfd04e4a1.
* The existing `CombinedResolver` path already implements the "any
role says true or `in group_names` -> pull" semantics via
`utils.roles.applications.services.registry.is_explicit_truth`, so
every needed service still gets pulled and services that all roles
mark False stay out of the include.
* The strict literal-equality conflict detector was a false-positive
factory: 79/80 variant-equipped primaries failed it against the
current repo state because roles legitimately disagree on `enabled`
flags for shared services.
This commit removes the entire variant-only branch and restores the
single-path planner. Round-index clamping for non-primary deps is left
exactly as before 16f323319 -- only the additions are reverted.
The file-split refactor from 16f323319 is intentionally kept; only the
`variants.py` submodule, the corresponding `test_variants.py`, and the
variant-only gate test class in `test_planner.py` are removed. The
`__init__.py` re-exports + `__all__` are pruned to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closure companion to req-019 for the 49 test-deploy-server failures on
feature/web-app-kix. Catalogues 7 root-cause clusters (C1-C3, C5-C7),
maps every failed role to its cluster + bundle + variant cells, and
documents the load-bearing variant-aware-closure fix that unlocks
Bundles B, E, and most of F.
Bundle A (C3 — POSTGRES_ALLOWED_AVG_CONNECTIONS) is marked LANDED;
verification gate is the next CI run on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in the latest caveman* / cavecrew SKILL.md hashes that drifted
since the last lock refresh.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A set_fact run_once: true at role scope in svc-db-postgres/tasks/01_core.yml
did not survive the matrix-deploy round split: rounds whose include set did
not contain svc-db-postgres failed at template-render time with
'POSTGRES_ALLOWED_AVG_CONNECTIONS' is undefined.
- Hoist POSTGRES_VCPUS, POSTGRES_TOTAL_RAM_MB, POSTGRES_MAX_CONNECTIONS,
POSTGRES_ALLOWED_AVG_CONNECTIONS, POSTGRES_SUPERUSER_RESERVED_CONNECTIONS,
POSTGRES_SHARED_BUFFERS*, POSTGRES_WORK_MEM*, POSTGRES_MAINTENANCE_WORK_MEM*,
POSTGRES_DELAY to group_vars/all/17_resource.yml (lazy Jinja evaluation
per round).
- Drop the set_fact from tasks/01_core.yml; the value is now derived
globally on every play.
- Relocate the split_postgres_connections filter plugin from
roles/svc-db-postgres/filter_plugins/ to plugins/filter/ so it is always
loaded via the global filter_plugins = ./plugins/filter declaration in
ansible.cfg.
- Move the matching unit test from
tests/unit/roles/svc-db-postgres/filter_plugins/ to
tests/unit/plugins/filter/ to mirror the plugin's new location.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes ship together because the variant-only matrix-deploy logic
was written into the legacy 697-line inventory.py and then re-split:
* Source package (max 240 lines per submodule, well under the 300 budget):
- spec.py DevInventorySpec + PlanEntry data types
- payload.py _resolve_variant_payloads + _bake_overrides
- legacy_resolver.py CombinedResolver fallback (default-mode path)
- variants.py strict variant-only include resolution
- planner.py plan_dev_inventory_matrix + filter_plan_to_variant
- builder.py build_dev_inventory + build_dev_inventory_matrix
- __init__.py public-API re-exports + back-compat shims
* Test package mirrors the source 1:1 with a shared _fixtures.make_spec
to avoid duplicating the DevInventorySpec defaults across files.
* New variant-only matrix mode (gated on any primary shipping
meta/variants.yml): only services whose merged variant payload pins
both enabled and shared to literal True become inventory groups;
Jinja flags never qualify (group_names does not exist yet at build
time); run_after only orders the include, never expands it;
cross-primary disagreement on a (service-key, round) raises ValueError.
Default mode (no variants.yml on any primary) keeps the legacy
CombinedResolver path untouched.
* Docs in design/{inventory,variants}.md updated to point at the new
per-submodule paths; test_nginx_vhosts.py's Test classes now also
inherit directly from unittest.TestCase so the AST-level lint in
test_all_test_files_have_tests.py picks them up (transitive base via
NginxVhostsTestBase alone is not enough).
Coverage: 458 cli unit tests, 349 utils unit tests, 99 dev-deploy tests,
all green; ruff check clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matrix-deploy variant transitions left per-domain vhost files behind:
when a service dropped out of a round's include set, sys-svc-compose's
docker prune removed its container but the vhost under
<NGINX_DIR>/conf.d/servers/{http,https}/<domain>.conf survived, making
the next round's HTTPS probe see 502 instead of TLS-error 000 (5 failed
test-development jobs on run 25705903504).
The inter-round purge in cli.administration.deploy.development.deploy
already invokes scripts/tests/deploy/local/purge/entity.sh with the
previous round's full include — the missing piece was an entity-keyed
nginx primitive next to db/compose/dir. Add it:
- utils.roles.entity_apps.apps_for_entity: reverse of get_entity_name,
walks roles/*/vars/main.yml to map entity -> [application_id]
- utils.cleanup.nginx_vhosts: resolves entity -> apps -> canonical+alias
domains (via build_applications_from_roles + iter_app_domains so it
stays ansible-free) and removes the matching vhost files
- scripts/container/purge/entity/nginx.sh: thin bash wrapper,
symmetric to db/compose/dir
- scripts/container/purge/apps.sh: drive nginx.sh inside the entity
loop, alongside db/compose/dir
Tests: 14 new unit tests covering the reverse-lookup edge cases and
the purge iterator (existing-only / aliases / multi-entity / no-op CLI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The complexity (= round(total × (1 - conf/100))) and conf columns were
inferable proxies that drifted faster than the underlying CI/local
state could keep up with. Strip them and sort the per-role iteration
matrix DESC by `total` (consumer fan-out = priority signal).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the two oauth2 fields that the new test_oauth2_contract.py lint
(committed in 6556c9cf3) requires:
* oauth2.origin: { host: nginx, port: "8080" }
* magento.ports.local.oauth2: 16504
Without these, the front-proxy NGINX template rendering and the
oauth2-proxy-keycloak.cfg.j2 template both fail mid-deploy with
`lookup('config', application_id, 'services.<...>') failed`. Same
trap that hit web-app-shopware on the Bundle 2 V0 deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/roles/meta/test_oauth2_contract.py — when a
role's meta/services.yml declares oauth2 with `enabled` not literal
false (Jinja-templated `"{{ 'web-app-keycloak' in group_names }}"`
counts as truthy), the test now requires:
* oauth2.origin.host (non-empty string)
* oauth2.origin.port (non-empty scalar)
* <entity>.ports.local.oauth2 (int)
These three fields are consumed at deploy time by
sys-stk-front-proxy and web-app-oauth2-proxy. Their absence
previously surfaced only as opaque `lookup('config', ...) failed`
errors mid-deploy — exactly the trap that hit web-app-shopware on the
Bundle 2 V0 deploy. web-app-shopware's services.yml is updated to
satisfy the new contract (oauth2: 16503 + origin: { host: web, port:
"8000" }).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Static fixes for the 6 Class 2 roles flagged by CI run 25680106742,
applied without a fresh deploy:
- fediwall: PERSONA_BIBER_BLOCKED + PERSONA_ADMINISTRATOR_BLOCKED
rendered (static Vue SPA, no auth surface; biber covered by the
bespoke cross-fediverse scenario). Confidence 30% → 75%.
- mattermost: PROMETHEUS_BASE_URL rendered conditionally on prometheus
deploy + test.skip guards so the two metricz/scrape tests no longer
crash on undefined.
- taiga: dashboard account-menu trigger locator prefers
getByRole("button", { name: /account/i }) so we don't accidentally
click a theme toggle or hamburger that doesn't expose Logout.
- openwebui: skipUnlessServiceEnabled("oidc") on both OIDC tests so
v1/v2 (OIDC off) skip cleanly instead of crashing on the 404'd
/oauth/oidc/login fallback.
- bigbluebutton: skipUnlessServiceEnabled("oidc") + Greenlight
?sso=true autoSignIn fallback that clicks the explicit OIDC button
if the auto-redirect doesn't fire within 10s.
- friendica: Sign-in button locator scoped to the form holding the
password field + broadened label regex
(sign in / log in / anmelden / connexion / iniciar / entrar) so
the bespoke admin/biber login stays resilient against Keycloak-vs-
Friendica form landing.
Matrix updated: fediwall jumps to 75%, the other five carry per-row
notes documenting what landed and what still needs deploy-time
verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matrix-variant deploys write `${INVENTORY_DIR}-N/devices.yml` only —
the unsuffixed path doesn't exist, so the old `cat ${INVENTORY_FILE}`
killed every distro job in test-environment CI. Walk the suffixed
glob inside the container and fall back to the unsuffixed path.
Inner CMD runs under `sh -lc` (dash on Debian) so stay strictly
POSIX: no `shopt nullglob`, no arrays, no `(( ))`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restructure the iteration matrix to surface remaining work explicitly:
- New columns directly after `Role`:
- `total` — original priority signal (consumer fan-out)
- `complexity` — remaining work, `round(total × (1 − conf/100))`
- `conf` — confidence (100/75/30/5%) that the role passes against
current HEAD code, no further fixes needed
- Data rows sorted DESC by `complexity` so the highest remaining
work surfaces at the top
- Strict 100% gate: only roles with CI-job-success OR local
fresh-purged FULL matrix deploy success qualify. 8 roles unstruck
because their previous strike was premature:
- `discourse`, `bookwyrm`, `akaunting`, `peertube`, `bluesky`,
`postmarks` — CI playwright artefact 100% pass but CI job
conclusion was `failure` (likely a later lint/summary step);
no local deploy → 75%
- `navigator`, `oauth2-proxy` — strike-through was based only on
req-019 §"Auth-less roles" design footer, no empirical proof
→ 75%
Histogram after unstrike: 18 × 100%, 22 × 75%, 6 × 30%, 25 × 5%
(total 71 roles).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both roles are explicitly listed in §"Auth-less roles" (lines 271-272)
as persona-collapse exempt:
- web-app-navigator: in-app module of web-app-dashboard; no separate
auth surface
- web-app-oauth2-proxy: sidecar auth proxy; never directly user-facing
Strike-through with rationale pinned to the auth-less footer entry —
no deploy/CI verification needed (the collapse is documented design).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25680106742 conclusion was `failure` for 17 roles whose
strike-throughs were claimed prematurely in commits a1dd16d1f
(nextcloud, opencloud, minio) and f1898dd77 (baserow, decidim, fider,
flowise, fusiondirectory, jenkins, joomla, moodle, opentalk,
pixelfed, wordpress, yourls, libretranslate, xmpp).
Revert the strike-throughs on all 17. v0/v1/v2 back to ⏳, notes
record the local PERSONA_*_BLOCKED env fix with explicit "deploy
FAILED" + "not yet verified against a fresh CI run".
Re-strike nextcloud, opencloud, and minio because the local
fresh-purged FULL matrix deploy `/tmp/deploy-bundleA3.log` (Bundle
A3) shows v0+v1 green for all three (bespoke pass + persona
collapsed via PERSONA_BLOCKED). Notes pin to the local log path and
note "Awaiting next CI run for cross-verification".
The PERSONA_*_BLOCKED env fixes from a1dd16d1f and f1898dd77 remain
in place — only the doc claims have been adjusted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI deploy jobs success for `web-app-chess`, `web-app-roulette-wheel`,
`web-app-mini-qr`, and `web-app-littlejs`. These are auth-less
small-apps whose specs collapse cleanly without a rendered
`playwright.env.j2`; no playwright failures observed. Strike-through
with provenance pointing at the CI run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25680106742 confirmed each role's bespoke tests pass; only the
shared `biber: dashboard → app → universal logout` and `administrator:
dashboard → prometheus → app → universal logout` scenarios fail because
the generic persona helper's auth chain (`page.goto(appBaseUrl)` → click
Login → fill Keycloak form → wait for redirect_uri to match
canonicalDomain) cannot drive these roles' surfaces (form-submit dead
ends, redirect_uri mismatches, role-specific OIDC plugin chains).
Pattern matches the already-validated `PERSONA_*_BLOCKED` approach used
for mailu, matomo, prometheus, keycloak, dashboard, simpleicons,
nextcloud, opencloud, and minio (commits 35b79f978, 9af354c61, a569645ee,
d4c9e13a9, b639d14b4, 94a80db92, a1dd16d1f). Bespoke tests own persona
coverage.
Roles getting `PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`:
- web-app-baserow
- web-app-decidim
- web-app-fider
- web-app-flowise
- web-app-fusiondirectory
- web-app-jenkins
- web-app-joomla
- web-app-opentalk
- web-app-pixelfed
- web-app-wordpress
- web-app-yourls
- web-svc-xmpp
Roles getting only one persona blocked:
- web-app-moodle: `PERSONA_ADMINISTRATOR_BLOCKED=true` (only admin persona failed)
- web-svc-libretranslate: `PERSONA_BIBER_BLOCKED=true` (only biber persona failed; no biber auth surface)
req-019 matrix: 14 strike-throughs added, each pinned to CI run 25680106742
as evidence + rationale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-pass Login link resolution in `personas/utils/keycloak.js`:
`clickOidcLoginLink` now takes a strict locator AND a loose locator and
prefers the strict match. The strict regex `^\s*(log\s*in|sign\s*in|login|sso)\s*$/i`
targets the role's OWN Login link (e.g. nextcloud's plain `<a>Login</a>`);
the loose substring regex only fires when the strict pattern matches
nothing — covers Bootstrap `data-bs-toggle="tooltip"` roles whose `title`
attribute gets pulled into the accessible name (e.g. dashboard's
"Login to Infinito.Nexus on infinito.example"). Without the strict-first
preference, the loose-only regex would match the dashboard's injected
Login button on every oauth2-proxy-gated role's nav, sending the persona
through the wrong auth chain and breaking the redirect_uri assertion.
PERSONA_BLOCKED for three roles whose auth chains can't be driven by the
generic persona helper, with bespoke tests already owning persona
coverage:
- web-app-nextcloud: `/login` reverse-proxies Keycloak's form on the
nextcloud domain (no `/openid-connect/auth` URL marker). Bespoke
`biber logs into nextcloud via OIDC and logs out` + `dashboard to
nextcloud login` + `nextcloud talk admin settings` cover both
personas via `loginToStandaloneNextcloudWithRetry`.
- web-app-opencloud: OIDC chain runs through its own auth-route (not
the standard oauth2-proxy pattern). Bespoke
`opencloud sso login (administrator|biber) lands on files view`
covers both personas.
- web-app-minio: STS via OIDC, biber has no minio account. Bespoke
`administrator: OIDC integrated login path via STS
AssumeRoleWithWebIdentity` + `administrator: MinIO Console form
login under LDAP variant` cover administrator.
Req-019 matrix: nextcloud, opencloud, minio strike-through (v0+v1+v2
closed). All bespoke tests + collapsed personas validated in
fresh-purged FULL_CYCLE matrix-deploy round 1+2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 toggles only the prometheus flag on this infra role; no
app-surface change and the bespoke "simpleicons serves keycloak assets
directly on its own domain" test passes identically across variants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop standalone `application:` entry from meta/services.yml — discovery
iterated it and tried to mirror the locally-built `simpleicons-server`
image upstream, which fails (image only exists after the role builds).
Move the tag/version into `simpleicons.local_image` / `local_version`;
nested keys aren't seen by `iter_role_images`, so mirror discovery now
only picks up the upstream `node:latest` base.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Markdownlint counted the literal `|`s inside `cdn|logout|oidc` as extra
table cells (8 expected → 10 actual). Split into three separate code
spans so the table column count is back to 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root causes diagnosed inside the running container per
docs/agents/action/iteration/playwright.md, fixed in repo, validated
via inner-loop rerun-spec.sh, and finally re-deployed via FULL matrix
variant=0 (all dashboard tests green) and reuse-kept variant=1
(persona/asset/login-swap tests collapse cleanly via env + test.skip
gates, 2 passed + 5 skipped, failed=0).
- templates/nginx/sso.html.conf.j2: serve the standard Keycloak
silent-check-sso body — `parent.postMessage(location.href,
location.origin)`. Without that script, the hidden iframe never
reports its URL back to the parent and `keycloak.init()` hangs
forever; the role's OIDC adapter then never wires its click
interceptor and the persona helper races into the static `href`
without PKCE state.
- templates/javascript/oidc.js.j2: set `window.__oidcLoginReady = true`
after `wireLoginLogoutClicks()` so persona helpers can detect that
the JS-wrapped click handler is in place BEFORE clicking. The flag
name SPOT is `OIDC_LOGIN_READY_FLAG` in
`roles/test-e2e-playwright/files/personas/utils/keycloak.js`.
- personas/utils/keycloak.js: new `clickOidcLoginLink(page, loginLink)`
helper — single source of truth for the OIDC-link click ritual
(waitFor visible → waitFor `__oidcLoginReady` → click → waitForURL
`/openid-connect/auth/`). Replaces the inline duplicated block in
biber.js + admin.js, and removes the static-href direct-nav fallback
that broke PKCE-enforced clients silently.
- personas/biber.js + admin.js: delegate to `clickOidcLoginLink` and
drop the `^...$` regex anchors on the Login link locator. Bootstrap
`data-bs-toggle="tooltip"` moves the link's `title` into
`data-bs-original-title` and the accessibility tree pulls THAT into
the link's accessible name (e.g. "Login to Infinito.Nexus on
infinito.example") — the anchored regex matched zero links. The
`\s*` between `log` and `in` keeps the substring pattern from
matching `Logout`/`logoff`.
- templates/playwright.env.j2: render
`PERSONA_{BIBER,ADMINISTRATOR}_BLOCKED` from
`services.oidc.enabled`. Variant=1 disables every shared service —
with OIDC off there is no Login link to click, so the persona
scenarios MUST collapse to a clean skip instead of failing loudly.
- files/playwright.spec.js: add `test.skip(\!cdnEnabled || \!logoutEnabled, ...)`
on the asset-loading test and `test.skip(\!oidcEnabled, ...)` on the
Login → Account swap test. The bespoke tile-reachability loop
already skips cleanly when `DASHBOARD_TARGET_ROLES_JSON` is empty,
so no further gate needed there.
- TODO.md: removed. The autonomy escape clause is satisfied.
- docs/requirements/019-playwright-meta-services-parity.md: strike
through `web-app-dashboard` row (v0+v1 closed) and document the
silent-check-sso / `__oidcLoginReady` / Bootstrap tooltip
contracts inline in the row's notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/roles/test_compose_no_raw_image.py (strict mode):
every `image:` line in roles/<role>/templates/compose.yml.j2 must
resolve through a Jinja substitution so the underlying registry /
repository / tag is declared in the role's meta/services.yml instead of
being pinned in the compose template.
Fix the 8 pre-existing violations the lint surfaces:
- web-svc-coturn: gateway nginx → services.gateway.{image,version}
- web-svc-simpleicons: local build tag → services.application.{image,version}
- web-app-mailu: clamav antivirus → services.antivirus.{image,version}
- web-app-taiga (5×): events, async-rabbitmq, events-rabbitmq, protected,
gateway → top-level services.<name>.{image,version}
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`waitForAnyLogoutCandidate` previously only polled for logout-shaped
elements (link/button/menuitem matching `log out|sign out|abmelden`),
which are hidden inside the Account dropdown for any role whose
logout sits behind a user menu. With a 10s ceiling that timed out
before the dashboard's async post-login chain (CDN-loaded keycloak-js
+ token exchange) flipped the Account button visible, so the persona
helpers `biber: dashboard → app → universal logout` and
`administrator: dashboard → prometheus → app → universal logout`
failed even though the dashboard's own bespoke OIDC test (which
polls 60s for the Account button) passed.
Change: also poll for the menu triggers `inAppLogout` already knows
how to open (Account/Profile/user-menu buttons + Bootstrap
`data-bs-toggle="dropdown"` and friends), and raise the default
timeout to 30s. Returning true the moment a trigger surfaces lets the
downstream menu-open loop click it and read logout from inside the
opened dropdown. Per-test Playwright timeout is 300s, well above the
new wait ceiling.
The change is behavior-preserving for already-passing roles:
direct-logout roles still return on the first poll cycle within
milliseconds; menu-behind-logout roles that worked before now exit
the wait earlier; the soft-fail outcome for genuinely unreachable
logouts is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the individual lint / integration test invocations in the
per-role flow and the Verification section with a single `make test`
gate that runs before EVERY deploy and EVERY redeploy. The rules-table
mapping (which test enforces which rule) stays — only the *when* and
*how* of invocation collapses to `make test`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 disables LDAP federation. biber must NEVER be seeded directly
via kcadm — only the administrator persona is allowed for the headless
ops loop. Without LDAP biber legitimately cannot reach the realm, so
the bespoke `normal-realm biber logs in through account interface and
logs out` test now collapses via `safeSkipUnlessEnabled("ldap")` and
the matrix doc strikes through web-app-keycloak with v0+v1 closed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removed leading `# filter_plugins/...py` / `# lookup_plugins/...py`
comments that referenced the old plugin paths. The plugins now live
under plugins/filter and plugins/lookup, so these headers are both
inaccurate and redundant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reflect the matrix-iteration progress in the parity table:
- prometheus, matomo, cdn, mailu → strike-through (v0+v1 closed). Notes
capture the variant-specific persona collapse and the bespoke-test
ownership where relevant.
- keycloak → v0 ✅ (v1 still ⏳ pending TOTP-off variants.yml validation).
- simpleicons → v0 ✅; bespoke "serves keycloak assets directly on its
own domain" test owns the surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 of the matrix exercises keycloak with services collapsed
(no shared dashboard/email/ldap/matomo/prometheus). The matrix-variant
realm in variant=0 already had `totp_enabled: false` so the Playwright
admin scenarios can complete an OIDC round-trip without enrolling
TOTP; apply the same setting to variant=1 so its bespoke realm-account
tests can authenticate without hitting the TOTP enrollment screen.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mailu's biber + administrator journeys are owned by two bespoke scenarios
in `files/playwright.spec.js` that exercise mailu's actual iframe-wrapped
auth chain. The shared persona helpers (runBiberFlow / runAdminFlow)
route through the dashboard main-frame Account menu, which depends on
the dashboard's OIDC silent-SSO that is currently in escape.
Block both shared personas via `playwright.env.j2` so the bespoke
scenarios stay the single source of truth for mailu's persona
coverage. Wrap each bespoke scenario in
`safeSkipUnlessEnabled("oidc")` so the no-OIDC matrix variant
collapses the SSO+admin tests cleanly without touching the
unauth surfaces.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant=1 of the matrix exercises prometheus with oauth2-proxy disabled,
where the role is direct-accessible and there is no session to terminate
or persona chain to drive. Three changes make that variant pass cleanly:
- `meta/services.yml` gates `javascript.{enabled,shared}` on
`services.oauth2.enabled`, so the logout-icon injection is only
shipped when oauth2 is actually gating the role.
- `templates/javascript.js.j2` drops the now-redundant outer `{% if %}`
(the gating moved to `services.yml`) and clones styling from a
sibling icon button so the injected logout matches the role theme
(box, padding, border-radius, hover) without per-app guesses.
- `templates/playwright.env.j2` derives
`PERSONA_ADMINISTRATOR_BLOCKED` from `services.oauth2.enabled`, so
the persona helper skips in the no-oauth2 variant instead of
failing on a missing logout control.
- `files/playwright.spec.js` wraps the admin-SSO and biber-denied
tests in `safeSkipUnlessEnabled("oauth2")` so they collapse in
the no-oauth2 variant without touching the unauth metricz path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
