`web-app-mailu/tasks/03_manage_user.yml` and `03b_create_user_token.yml`
run `compose exec admin flask mailu user <...>` and `compose exec admin
curl http://localhost:8080/api/v1/token` immediately after the existing
wait blocks in `02_wait_for_database.yml` (mariadb domain table + admin
flask CLI ready). Under CI load the Mailu admin container is reachable
on the compose network, but two further runtime preconditions are not
guaranteed when those tasks fire:
1. The flask CLI imports succeed (`flask routes` returns 0) before
Mailu's admin code has finished establishing the lazy Redis
connection that `flask mailu user` needs to commit the user.
Symptom (run 26480944120 job 77980186049, infinito-nexus/core
run 26480946718 job 77980074367, web-app-suitecrm):
redis/connection.py:639 ConnectionRefusedError: [Errno 111]
during `flask mailu user no-reply infinito.example -- <pw>`.
2. The admin gunicorn HTTP server on :8080 is not yet listening when
`03b_create_user_token.yml` curls `http://localhost:8080/api/v1/
token`. Symptom (run 26480946718 job 77980074254, web-app-fediwall):
curl: (7) Failed to connect to localhost port 8080 after 0 ms.
Add two further wait blocks after the existing flask-routes wait, both
following the same probe/rescue pattern (300-line container log dump
on timeout) the file already establishes:
* Redis: `compose exec -T redis redis-cli ping` until stdout
contains "PONG". Matches Mailu's own healthcheck command verbatim
(svc-db-redis/templates/service.yml.j2 line 24) and depends on no
extra auth -- Mailu's env.j2 declares no REDIS_PASSWORD and the
injected svc-db-redis service runs with no `--requirepass`.
* Admin HTTP: `compose exec -T admin curl -sS -o /dev/null -w
'%{http_code}' http://localhost:8080/sso/login` until rc=0 and the
captured status code matches `^[1-5][0-9][0-9]$`. Any HTTP response
(200 / 302 / 404 / 5xx) proves gunicorn is bound; the regex form
makes the check robust against an upstream rename of the SSO
endpoint.
Restructure for clarity:
* Split the original `02_wait_for_database.yml` into
`02_wait_for_mariadb.yml` (mariadb domain table + flask CLI ready;
flask init touches the DB) and `03_wait_for_redis.yml` (Redis PING
+ admin HTTP :8080; the HTTP API requires Redis-backed sessions).
* Bump every subsequent task file by one step
(03_manage_user -> 04_manage_user, 04_generate-... -> 05_..., etc.)
so the directory order still encodes execution order at a glance.
* Update the cross-`include_tasks` references in `01_core.yml`,
`04_manage_user.yml` and `04a_manage_user_token.yml`. Also bump
the trailing comment in `01_core.yml` that pointed at
`03_manage_user.yml` to the new `04_manage_user.yml`.
* Hoist the recurring `lookup('container', application_id,
'directories.instance')` (12 occurrences plus a 6x indirection via
a per-include `MAILU_DOCKER_DIR` alias previously defined in
`01_core.yml`) into a single `MAILU_INSTANCE_DIR` constant in
`vars/main.yml`, matching the existing `MAILU_VOLUMES_DIR` shape.
All `chdir:` values now resolve to the same string with no extra
indirection.
`make autoformat` and `make test` (188 tests, 158s) pass on the
restructured tree.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`startMattermostSsoFlow` polled `/login` in a `while (true)` loop with
`waitUntil: "domcontentloaded"` and a 10s `isVisible` window for
`#input_loginId`. Under CI load Mattermost v11+'s React bundle takes
15-30s to parse and mount the login form. Each loop iteration's
`page.goto` aborted the previous in-flight hydration, so the SPA
restarted from scratch on every retry and the form mounted only when
the iteration's 10s window happened to align with a hydration that
slipped through.
Run 26451797037 job 77878640850 (web-app-mattermost ubuntu) timed out
at 300s in this state. The Playwright trace shows biber's loop ran 80+
iterations over 156s before `#input_loginId` finally became visible;
admin's loop then had only 132s of budget left, never broke out, and
the test hit the outer `testTimeout: 300_000`. Both contexts'
`error-context.md` page snapshots confirmed the form was present at
the moment of timeout, ruling out an actual Mattermost-side failure.
Replace the loop with a single navigate sequence:
* `waitUntil: "load"` waits for the full `load` event, giving the
React bundle time to parse and run instead of returning at
`domcontentloaded` before any of it has executed.
* One explicit `page.locator(...).waitFor({ visible, timeout: 60_000 })`
spans the observed 15-30s hydration tail with comfortable headroom.
* The wrapping `for (attempt = 0; attempt < 2; ...)` adds exactly one
retry on transient navigation failures (network blip, mid-deploy
container restart) without reintroducing the abort-self race the
bare loop had — only one in-flight hydration at a time.
The PASS 1 (sync) path that already succeeded in the failing run keeps
working because the new control flow is strictly simpler than the
loop. The failing PASS 2 (async) path now has a deterministic upper
bound: at most two 60s navigations + two 60s `waitFor`s per call site
= 240s ceiling per `startMattermostSsoFlow`, comfortably below the
test-level 300s timeout even with biber + admin both invoking it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`web-opt-rdr-www` generates a redirect vhost
`<NGINX_DIR>/conf.d/servers/{http,https}/www.<domain>.conf` for every
deployed app's canonical and alias domains. Those redirect vhosts are
not part of any app's `server.domains.{canonical,aliases}` list, so the
entity-keyed purge primitive in `utils/cleanup/nginx_vhosts.py` did
not yield them and `iter_vhost_files_for_entity` left them on disk.
Between matrix-deploy rounds this leaked the redirect vhosts of the
previous round's broader app set into the next round's narrower one.
On run 26451828677 job 77879571696 (web-opt-rdr-www on debian) the
round 1 -> round 2 transition
round 1/2 inv=github-0 apps=[15 apps incl. web-svc-cdn,
web-svc-simpleicons, web-svc-css, ...]
round 2/2 inv=github-1 apps=[web-opt-rdr-www]
dropped the bare `cdn.infinito.example.conf` etc. via the existing
purge but kept the matching `www.cdn.infinito.example.conf`,
`www.icon.infinito.example.conf`, ... behind. Round 2 only redeploys
web-opt-rdr-www, whose `www_redirect_domains` lookup now only emits
its own `www.w3redirect.<primary>` redirect, so nothing reissues the
stale www-prefixed vhosts.
`sys-ctl-hlth-csp` discovers domains by scanning .conf filenames in
the nginx servers dir, so it still probed the stale `www.<x>.conf`
hosts. nginx served them with a cert whose SAN no longer covered the
hostname (the per-domain cert path baked into the conf had been
removed alongside the bare vhost), the TLS handshake returned
`net::ERR_SSL_UNRECOGNIZED_NAME_ALERT`, the CSP probe exited 11, and
`sys-service-terminator : Apply state` failed after 10 retries.
Extend `iter_vhost_files_for_entity` to also yield the
`www.<domain>.conf` variant for every domain it already yields,
skipping domains that already begin with `www.` so a role that
declares its canonical with a leading `www.` (web-opt-rdr-www's own
`www.w3redirect.<primary>`) does not produce `www.www.<x>.conf`. The
existing two- and three-step purge primitives (db / compose / dir /
nginx) and the orchestrator in `scripts/container/purge/apps.sh` keep
their entity-keyed contract; only the per-entity yield set widens.
Coverage:
- test_yields_www_redirect_variant: per-app www-prefixed file is
yielded alongside the bare vhost.
- test_www_prefixed_domain_not_double_prefixed: a www-prefixed
canonical does not synthesize `www.www.<x>` even when such a file
exists on disk (defense in depth against a future regression that
prepends unconditionally).
- test_removes_www_redirect_variant: end-to-end purge removes both
bare and www variant, leaves an unrelated www-prefixed file alone.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The include_role vars block referenced `_load_app_id`, a host fact
created by `set_fact` at the top of load_app.yml. Host facts live in
the non-persistent fact cache (vars/manager.py:405) and survive across
include frames, so every nested `include_tasks: utils/load_app.yml`
call overwrote `_load_app_id` for the whole play.
Inside the coturn deploy chain (web-svc-coturn -> sys-stk-full ->
sys-stk-backend -> sys-svc-compose), sys-svc-container.Setup Swapfile
triggers a nested load_app.yml(svc-opt-swapfile) BEFORE
sys-svc-compose/04_files.yml renders coturn's env.j2. The nested call
set_fact'd `_load_app_id='svc-opt-swapfile'` and left it as a host
fact. When env.j2 lazy-evaluated `COTURN_REALM -> domain ->
lookup('domain', application_id)`, the outer include_role's vars block
resolved `application_id` through `{{ _load_app_id }}` and read the
mutated value, yielding
lookup('domain'): failed to resolve domain for 'svc-opt-swapfile':
application_id 'svc-opt-swapfile' not found in domains mapping
(job 77879571693 on run 26451828677, debian, play 3)
The fix is two-step:
1. Switch the include_role to reference `load_app_id` directly.
`load_app_id` is the caller's `include_tasks vars:` argument,
attached to the TaskInclude object as task-scoped vars (Level 414
include params in vars/manager.py). It walks the `_parent` chain
via `task.get_include_params()`, and `_ACTION_ALL_INCLUDES`
(constants.py:101) covers both include_tasks and include_role, so
the outer TaskInclude's vars stay reachable from any depth. Each
nested include_tasks frame has its own TaskInclude.vars and cannot
mutate the outer frame's `load_app_id`.
2. Remove the `set_fact: _load_app_id` step entirely. The host fact
was the only mutation source for this variable in load_app.yml,
and dropping it eliminates the bug class for the whole loader.
`grep` across the repo confirms `_load_app_id` is unused elsewhere
and no role does `set_fact: application_id|service_role|service_item
|load_app_id`, so the include-params resolution chain stays immune
to host-fact races.
The 7 current callers of load_app.yml are verified safe:
- 6 pass literal strings (sys-svc-webserver-core, web-app-bluesky,
sys-stk-front-base, web-opt-rdr-www, sys-svc-container,
web-app-dashboard)
- 1 (sys-service-loader/load_service.yml) passes
`"{{ service_role }}"`, which resolves via outer-loop-scoped
`service_item.role` and is itself only ever set via include-tasks
vars or loop_control, never via set_fact.
Reuses the same defense-in-depth direction as commits f32266d53
(tls lookup defensive templating) and 7b9cb5617
(current_play_redirect_domains defensive templating): close the path
where a mutable side channel feeds into a load-bearing lookup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Ran `python -m cli.contributing.requirements.archive` after marking all
remaining acceptance criteria as `[x]` in:
- 012 - Package Cache (Nexus 3 OSS)
- 013 - Sub-Beta-to-Beta Lifecycle Promotion for `web-*` Roles
- 015 - OpenCloud and OpenTalk IAM integration
- 015 - Self-built Moodle image
Their headings are appended to the `## Archive` block of
`docs/requirements/README.md` and the per-requirement files are deleted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ran `python -m cli.contributing.requirements.archive` to move the
following fully-checked-off requirement files into the `## Archive`
block of `docs/requirements/README.md` and delete the per-requirement
files:
- 001 - Nextcloud Talk TURN relay
- 002 - Service Registry Refactoring
- 003 - Reduce `applications` and `users` to lookup-only access
- 004 - Generic RBAC role auto-provisioning via OpenLDAP
- 005 - WordPress Multisite support for OIDC-driven RBAC
- 006 - Service-gated Playwright tests
- 007 - WordPress to Discourse post round-trip
- 015 - KIX Service Management integration
- 016 - web-app-hugo
Cross-references from the still-open requirements (015-moodle,
015-opencloud, 017, 018, 019, 021) rewritten from the deleted file
paths to \`README.md#archive\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* TC003: move `pathlib.Path` into `TYPE_CHECKING` blocks across the
new `requirements/archive` package and its completeness lint test.
* TC001: same treatment for `VaultHandler` in
`credentials/emit.py` (annotation-only).
* SIM105/S110: replace `try/except/pass` in
`current_play_redirect_domains.py` with
`contextlib.suppress(Exception)`.
* `archive/__main__.py`: mark `README_PATH.read_text(...)` with
`# nocheck: cache-read` — CLI rewrites the README in the same flow,
mirroring the `changelog/archive` pattern.
* Drive-by ruff-format condensations in the archive unit tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* `python -m cli.contributing.requirements.archive` moves the H1 of
every fully-checked `docs/requirements/NNN-*.md` into the
`## Archive` block of the directory README and deletes the file.
Files with any remaining `- [ ]` are skipped. Split SRP-wise into
discovery / inspect / readme / __main__.
* New lint `test_requirements_completeness` warns per unchecked
acceptance criterion (GitHub annotation) and fails when any
requirement is ready for archival, pointing at the CLI. Keeps the
requirements directory small so AI agents process less context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The plugin's previous
domain_primary = str(vars_.get("DOMAIN_PRIMARY") or "")
domain_homepage = str(vars_.get("DOMAIN_HOMEPAGE") or "")
read inventory values that may carry Ansible 2.18+'s
``_AnsibleTaggedStr`` with the ``TrustedAsTemplate`` tag — exactly
the shape ``inventories/development/default.yml:20`` produces with
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}"
A bare ``str(value)`` casts the tag away and leaves the literal Jinja
text in the plugin output. The output then flows into
``CURRENT_PLAY_REDIRECT_DOMAINS`` via the set_fact in
``tasks/stages/01_constructor.yml`` and lands in
``roles/web-opt-rdr-domains/tasks/main.yml`` loop items as
item={'source': "{{ lookup('env','INFINITO_DOMAIN') }}", 'target': "{{ … }}"}
The ``lookup('tls', domain, 'enabled')`` call in
``roles/web-opt-rdr-domains/tasks/redirect-domain.yml:5`` then
receives the literal Jinja string and raises
The lookup plugin 'tls' failed: tls: application_id
'{{ lookup('env', 'INFINITO_DOMAIN') }}' not found in domains mapping
(job 77797371504 on run 26428080957). That CI ran on SHA d1410c0be,
which already shipped this plugin — so the upstream replacement of
the inline two-stage set_fact merges with the lookup plugin did not
fix the underlying trust-tag-strip bug; it just relocated it from
Jinja to Python.
Add ``_resolve_str`` that runs the value through ``self._templar``
before string-coercion. Idempotent for already-resolved strings,
exception-safe (falls back to ``str(raw)`` if the templar is
unavailable, e.g. on hosts without rendered vars). Both
``DOMAIN_PRIMARY`` and ``DOMAIN_HOMEPAGE`` route through it.
Defense-in-depth with the existing ``plugins/lookup/tls.py``
defensive templating (commit f32266d53): even if a future code path
hands ``lookup('tls', …)`` an untemplated string, the tls plugin
itself now templates the first term. Either layer alone suffices for
the rdr-domains failure; both together guarantee the symptom cannot
re-surface from a single regression elsewhere.
Coverage: two regression tests in
``tests/unit/plugins/lookup/test_current_play_redirect_domains.py``
— ``test_jinja_tagged_domain_primary_is_templated`` proves the
templar call happens and produces the resolved domain, and
``test_templar_failure_falls_back_to_raw_value`` proves a raising
templar does not crash the plugin. The 10 existing tests stay green.
``make test`` green across all 5 targets.
Class follow-up: 11 further lookup plugins read inventory values
without ``templar.template`` first (see git log for ``vars_.get``
call sites). Not in scope for this fix; logged for a separate
audit pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Six ``uri_retry`` tasks under ``roles/web-app-xwiki/tasks/`` carried
their XML payload as inline ``body: |`` blocks (CheckExtension Groovy
page, InstallExtensions Groovy page, the administrator user-page
PUT/POST pair, and the post-Fix-#8 XWikiAdminGroup PUT/POST pair).
Inline XML mixes Ansible YAML indentation with Jinja control flow
(``{% raw %}``-escaping for ``{{groovy}}`` delimiters), defeats
editor XML support, and silently rots when a future contributor
copy/edits the snippet without realising the surrounding ``body: |``
context.
Split by concern:
- ``templates/xml/*.xml.j2`` — six XML files that embed Jinja:
``user_page``, ``user_xwikiusers_object``,
``xwiki_admin_group_page``, ``xwiki_admin_group_member_object``,
``install_extensions_page``, ``main_webhome_page`` (the last one
was already a template; moved here for symmetry with the new five).
- ``files/xml/check_extension_page.xml`` — fully static; rewritten
without ``{% raw %}{{groovy}}{% endraw %}`` because the file is
served via ``lookup('file', …)`` (no Jinja pass), so ``{{groovy}}``
reaches XWiki literally.
Task references migrate to ``body: "{{ lookup('template',
'xml/<name>.xml.j2') }}"`` (or ``lookup('file', 'xml/<name>.xml')``
for the static case), normalising the previously bespoke long-form
``roles/web-app-xwiki/templates/main_webhome_page.xml.j2`` lookup in
the bargain.
SPOT-extract the magic string ``XWikiAdminGroup``:
- ``XWIKI_ADMIN_GROUP_PAGE: "XWikiAdminGroup"`` in vars/main.yml,
documented as the page-name part used by REST URLs, the
XAR-shipped page, the LDAP ``group_mapping`` argument, and the
OIDC ``groups.mapping`` argument.
- Five call sites updated to ``{{ XWIKI_ADMIN_GROUP_PAGE }}`` or
bare ``XWIKI_ADMIN_GROUP_PAGE`` inside list contexts:
``tasks/03_administrator.yml`` (×4 — two task names, the PUT URL,
the POST URL), ``templates/xml/xwiki_admin_group_page.xml.j2``,
``templates/compose.yml.j2`` (LDAP group_mapping), and
``templates/xwiki.properties.j2`` (OIDC groups.mapping, in a
commented example).
Lint guard: new ``tests/lint/ansible/tasks/test_no_inline_xml_body.py``
scans every ``roles/`` and ``tasks/`` YAML for ``body: |`` (or
``body: >``) folded scalars whose first content line starts with
``<`` and fails with a pointer at the matching ``templates/xml/...``
or ``files/xml/...`` alternative. JSON / shell / SQL bodies stay
allowed. Per-task opt-out: ``# nocheck: inline-xml-body`` on the
``body:`` line or the line above. Five unit tests cover the
scanner: matches XML body, ignores JSON body, honours same-line
nocheck, honours line-above nocheck, repo-wide sweep clean after
this refactor.
XWiki v17.10.8 distribution-merge behaviour was cross-checked
against upstream (``AbstractXarEntryType.upgradeType =
THREEWAYS``); the empty ``<member/>`` placeholder in the
XAR-shipped ``XWikiAdminGroup`` cannot overwrite our explicitly-added
member (different GUID / number).
``make test`` green across all 5 targets.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`XWIKI | Ensure user is in XWikiAdminGroup` POSTed
`<XWiki.XWikiGroups>{<member>XWiki.administrator</member>}</…>` to
`POST /rest/wikis/xwiki/spaces/XWiki/pages/XWikiAdminGroup/objects`,
hit HTTP 404 31 retries in a row, and aborted the deploy (job
77797115264 on run 26428083128, debian/no-SSO variant).
XWiki's REST `…/pages/<name>/objects` resource resolves the parent
page first; a 404 means the page document does not exist yet. The
admin-group page ships in
`xwiki-platform-distribution-ui-base/.../XWikiAdminGroup.xml` and is
materialised by the XWiki Distribution Job on first boot. The
Distribution Job did not finish materialising it before the
administrator-bootstrap step fired in this run — log evidence:
`XWiki.administrator` user-page POST 201 at 08:24:41 (line 487473),
group POST 404 at 10:09:27 (line 487745), no
`XWikiAdminGroup` PUT/import in between.
Mirror the pattern already used two tasks above for the user page:
PUT-create `XWiki.XWikiAdminGroup` (idempotent — returns 200/202 when
the Distribution Job won the race, 201 when we beat it) before POSTing
the membership object. The new PUT + the existing POST live inside a
single `block:` gated by `not (XWIKI_SSO_ENABLED | bool)` — the
existing `XWIKI_SSO_ENABLED` SPOT in vars/main.yml subsumes the
duplicated `not LDAP and not OIDC` clause on the POST task.
Distribution-Job-overwrite risk assessed against XWiki v17.10.8
upstream source:
- `AbstractXarEntryType.upgradeType` defaults to
`UpgradeType.THREEWAYS` (3-way merge, not OVERWRITE/SKIP). Verified
at `xwiki-platform-xar-model/.../type/AbstractXarEntryType.java:38`.
- `DocumentMergeImporter.importDocument` passes the existing document
as merge-current; the XAR template ships an empty `<member/>`
placeholder with a hardcoded GUID
(`46d387f2-4d8d-446d-a618-8bfcb655ba18`), so our REST-created
XWikiGroups object (auto-assigned number, fresh GUID) is treated as
a separate object and survives a later XAR re-install. The empty
placeholder also means the XAR install has no member value to
overwrite with.
- Combined risk: ~5% that a future XWiki release changes the upgrade
type or the XAR-shipped XWikiAdminGroup content; logged for review
but not blocking.
Confidence: ~92%. Remaining ~8% covers (a) future XWiki XAR re-runs
with non-default conflict actions, (b) custom merge-conflict configs
in the deployed flavor, (c) unbedachte XWiki internal listeners.
Variant scope: no-SSO only (`XWIKI_SSO_ENABLED == false`). The OIDC
and LDAP variants delegate group membership to the SSO mapper and
skip this task entirely, so the fix is a no-op there.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The inventory validator at cli/administration/inventory/validate
walked `recursive_keys(host_vars) ⊆ recursive_keys(application_defaults)`
and flagged any extra key with "Missing default for <app>: <key>".
After commit 10cb02461 ("bake only variants.yml overrides into
matrix-deploy --vars") host_vars carry every key the active
variants.yml overlay declares. Since 29377e1bd
(`_build_application_defaults` reads only `meta/<topic>.yml`, not
variants.yml — to keep the variant-0 service-key leak closed), the
defaults' allow-set is artificially narrower than the role contract:
roles/web-opt-rdr-www declares `services.dashboard.{enabled,shared}`
exclusively in `meta/variants.yml`, so the validator rejects the
baked host_vars with three "Missing default" entries before
ansible-playbook even starts (job 77797115291 on run 26428083128,
distro=debian; distro-agnostic per the matrix).
Widen the validator's legal-key set per app to
recursive_keys(defaults[app]) ∪ ⋃ recursive_keys(variants[app][i])
variants.yml IS part of the role contract; the validator's previous
"defaults only" view was incomplete. Runtime
`_build_application_defaults` stays narrow (so 29377e1bd's
variant-0-leak fix stays in place) — only the validator widens. Typos
not declared by any variant remain caught: a new
`test_key_typo_in_variants_still_rejected` unit test pins that.
Package split (KISS/DRY/SPOT), mirroring the credentials/ refactor:
`validate/__main__.py` carried six orthogonal concerns plus
orchestration in 175 LoC. Split:
- `validate/keys.py`: `recursive_keys` (pure helper).
- `validate/loaders.py`: `load_yaml_file` (with vault-block stripping)
and `load_inventory_files` (directory scan).
- `validate/applications.py`: `compare_application_keys` (the widened
variants-aware logic).
- `validate/users.py`: `compare_user_keys`.
- `validate/hosts.py`: `validate_host_keys`.
- `validate/__main__.py`: thin orchestrator — argparse + load defaults/
variants/users + invoke compare_* + exit code. 75 LoC.
`validate/__init__.py` re-exports the public API
(`main`, `compare_application_keys`, `compare_user_keys`,
`load_inventory_files`, `load_yaml_file`, `recursive_keys`,
`validate_host_keys`, `PROJECT_ROOT`). External callers
(`tests/unit/cli/administration/inventory/validate/test_main.py`,
`tests/unit/cli/administration/inventory/validate/test_hosts.py`)
keep working unchanged via this re-export.
`__main__.py` uses ABSOLUTE imports (`from cli.administration.inventory.
validate.applications import ...`) instead of relative because it's
invoked as a subprocess by tests and by the deploy entrypoint; the
sys.path bootstrap at the top of the file makes `cli.…` resolvable
in that mode.
Test layout follows the split — one test file per module:
- `test_keys.py` (6 tests: empty/non-dict/flat/nested/deep/prefix).
- `test_loaders.py` (8 tests: vault stripping, missing file, broken
YAML, top-level + recursive directory scan, empty-block skip).
- `test_applications.py` (9 tests: all-keys-match, unknown app, extra
key, credentials skip, variants legal, variants union, no-variants
fallback, source path in error, typo still rejected).
- `test_users.py` (5 tests: match, unknown user warn, missing default,
password skip, credentials skip).
- `test_hosts.py` (renamed from test_inventory_hosts.py — unit tests
for `validate_host_keys`).
- `test_main.py` (renamed from test_inventory.py — end-to-end
subprocess tests for `main()`, including the
`test_variants_only_key_is_accepted` regression guard for this fix).
Coverage: 36 tests, all green. `make test` green across all 5 targets.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mattermost v11.7.1's `Root.tsx` redirects every fresh browser context
that lacks `localStorage.__landingPageSeen__` to `/landing#/` (the
"open in app vs view in browser" splash) — verified against upstream
source webapp/channels/src/components/root/root.tsx:146 and the key
name in webapp/channels/src/utils/constants.tsx:808 at tag v11.7.1.
The redirect is purely client-side (React Router `history.push`), so
localStorage stamped via Playwright's `addInitScript` before the very
first navigation suppresses it.
The bounce broke job 77797115305 on run 26428083128 (fedora distro):
`test-biber-dm-administrator` opens two fresh `browser.newContext()`s
that go through `startMattermostSsoFlow`. The first SSO test passes
(default `page` fixture inherits no relevant state either, but the
admin user's first provisioning seemingly papers over the missing
flag for that one context). The DM test's second context bounces to
`/landing#/` indefinitely, the existing 120s deadline + outer 300s
test budget expire across all three retries, and the role-runner
reports `Playwright failed for roles: ['web-app-mattermost']`.
Adds a single `page.addInitScript` in `startMattermostSsoFlow` that
writes `localStorage.__landingPageSeen__ = "true"` on every document
load. The existing bounce-loop stays as a safety net for the case
where Mattermost ever moves the flag — the fix is purely additive.
Verification:
- Flag name `__landingPageSeen__`: `StoragePrefixes.LANDING_PAGE_SEEN`
in upstream webapp/channels/src/utils/constants.tsx:808.
- Redirect gate: `BrowserStore.hasSeenLandingPage()` short-circuits
the redirect at root.tsx:146 — `localStorage.getItem(...)` on the
same key.
- Playwright contract: `addInitScript` evaluates on every page load
before any page script (per Playwright docs).
- Confidence: ~95%. The remaining 5% covers (a) `appDownloadLink`
unset (redirect never fires — irrelevant) and (b) Playwright
ordering edge cases (standard pattern, low risk).
The clean architectural follow-up (Playwright `storageState` reuse
to skip the second SSO round-trip entirely) lives elsewhere; this
commit is the focused unblocker.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The credentials baker at cli/administration/inventory/credentials/__main__.py
walked `schema_creds.items()` one level deep and called `str(value)` on
every leaf. For nested credential groups like
``credentials.recaptcha = {key, secret}`` (declared in
roles/web-app-espocrm/meta/schema.yml and
roles/web-app-listmonk/meta/schema.yml) this collapsed the inner dict
into its Python repr, vault-encrypted that blob, and wrote it as a
single scalar at ``credentials.recaptcha``. The runtime ``config``
lookup then choked on the descent into ``recaptcha.key``:
The lookup plugin 'config' failed: Expected dict for 'key', got
_AnsibleTaggedStr at 'key'
Full path so far: applications['web-app-espocrm'].credentials.recaptcha.key
Current object: "{'key': '{{ CAPTCHA.RECAPTCHA.KEY }}', 'secret':
'{{ CAPTCHA.RECAPTCHA.SECRET }}'}"
(jobs 77797371397 espocrm and 77797371442 listmonk on run 26428080957.)
Introduce ``emit_credentials`` (new module: ``credentials/emit.py``)
that walks the schema recursively. Scalar leaves still vault-encrypt
via ``to_vault_block``; dict nodes recurse into a nested CommentedMap
via ``ensure_map``. The walker also tracks dotted full-key paths so
the existing ``--force``-mode override-apply logic sees the same dotted
keys it always saw.
Auto-healing for previously-corrupted host_vars: ``ensure_map``
overwrites non-CommentedMap placeholders, so the next bake replaces a
stringified ``credentials.recaptcha`` blob with the correct nested
structure. Operators with corrupted entries simply re-run the
credentials generator; no manual cleanup needed.
Package refactor (KISS/DRY/SPOT):
The 450+ LoC ``__main__.py`` was carrying four orthogonal concerns
(vault, --set parsing, recursive emission, prompts) plus orchestration.
Split by concern:
- ``credentials/vault.py``: ``is_vault_encrypted``,
``is_ruamel_vault``, ``to_vault_block`` (+ private helpers
``_vault_body``, ``_make_vault_scalar_from_text``).
- ``credentials/overrides.py``: ``parse_overrides``, ``override_for``.
- ``credentials/emit.py``: ``ensure_map``, ``emit_credentials``.
- ``credentials/prompts.py``: ``ask_for_confirmation``.
- ``credentials/__main__.py``: argparse + ``main`` + two thin mode
handlers (``_run_snippet_mode``, ``_run_default_mode``) + the
force-override helper.
``credentials/__init__.py`` re-exports the public API
(``main``, ``ask_for_confirmation``, ``parse_overrides``, etc.) so
external callers (notably ``tests/unit/cli/administration/inventory/
test_credentials.py``) keep working unchanged.
Test layout follows the split:
- ``tests/unit/cli/administration/inventory/credentials/test_vault.py``
(20 tests: vault detection, body extraction, scalar construction,
to_vault_block paths, plus the VaultHandler subprocess coverage).
- ``tests/unit/cli/administration/inventory/credentials/test_overrides.py``
(13 tests: parse_overrides + override_for incl. dotted nested keys).
- ``tests/unit/cli/administration/inventory/credentials/test_emit.py``
(10 tests: ensure_map placeholder replacement, recursive emission,
none → empty, skip_existing semantics, override precedence,
dotted-path tracking, the on-disk corruption recovery case).
- ``tests/unit/cli/administration/inventory/credentials/test_prompts.py``
(5 tests: ask_for_confirmation yes/no/garbage/case/whitespace).
``tests/unit/cli/administration/inventory/test_credentials.py`` keeps
only the end-to-end ``main()`` integration coverage (including
``test_nested_credentials_dict_recurses_instead_of_stringifying`` —
the explicit regression guard for the original failure mode).
Coverage: 47 new per-module unit tests + the 5 existing main()
integration tests, all green. ``make test`` green across all five
targets.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI job 77797371504 on run 26428080957 crashed with:
tls lookup: application_id '{{ lookup('env', 'INFINITO_DOMAIN') }}'
not found in domains mapping
`DOMAIN_PRIMARY` is defined in inventories/development/default.yml:20
as `"{{ lookup('env', 'INFINITO_DOMAIN') }}"`. On Ansible 2.18+ that
string carries a `TrustedAsTemplate` tag (it is an
`_AnsibleTaggedStr`). The constructor at tasks/stages/01_constructor.yml:64
wrapped it in `[ DOMAIN_PRIMARY ] | flatten | map('string') | first`
— the explicit `str()` cast strips the trust tag. The resulting plain
string flowed into `CURRENT_PLAY_REDIRECT_DOMAINS.source` as the
literal text `{{ lookup('env', 'INFINITO_DOMAIN') }}`, was passed
through `domain: "{{ item.source }}"` in
roles/web-opt-rdr-domains/tasks/main.yml:14 (one render pass that
produces untrusted content), and finally reached `as_str(terms[0])`
in plugins/lookup/tls.py:39 unchanged. The tls plugin never templated
its arg, so `resolve_term` looked up the literal Jinja string in the
domains mapping and raised.
Two layers, both necessary:
1. plugins/lookup/tls.py:39 — defensively template `terms[0]` through
`self._templar` before string-coercion. Mirrors the idiom in
plugins/lookup/unit_name.py:33 and plugins/lookup/email.py:61. Any
future caller that hands the plugin an unresolved expression now
gets it resolved before the domains-map lookup.
2. tasks/stages/01_constructor.yml:64-65 — drop the
`[ X ] | flatten | map('string') | first` ritual for
`DOMAIN_PRIMARY` and `DOMAIN_HOMEPAGE`. Both are scalar strings in
every known inventory (group_vars/all/00_general.yml:41,43 +
inventories/development/default.yml:20); the over-defensive
wrapper was the trust-tag stripper. Direct reference preserves the
tag through `CURRENT_PLAY_REDIRECT_DOMAINS` and every downstream
consumer.
Downstream consumers audited; all transparent to `_AnsibleTaggedStr`
(it inherits from `str`):
- merge_mapping filter: content-based dict-key compare
- domain_mappings filter: takes `DOMAIN_PRIMARY` directly, no cast
- items2dict(key_name='source', value_name='source'): hashable
- generate_all_domains filter: `isinstance(item, str)` matches
- sys-stk-front-base `when: ... | length > 0`: only counts
- web-opt-rdr-domains -> tls lookup (the failing path): now templates
Coverage: new `test_term_with_unresolved_jinja_is_templated` in
tests/unit/plugins/lookup/test_tls.py verifies the lookup feeds its
first term through the templar. The 12 existing tls tests stay
green. `make test` green across all 5 targets.
Class follow-up (not addressed here): the
`| string` / `| map('string')` cast still strips `TrustedAsTemplate`
elsewhere if a future commit reintroduces it; 12 other lookup
plugins do not yet defensively template their args. A lint guard and
plugin-wide audit will come in separate PRs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pruned per project convention "no unnecessary code comments":
- module docstring: 23 lines → 1 line (the file name + symbol already
state intent; full incident write-up belongs in the commit log)
- wipe_database_entries docstring: 11 lines → 2 lines
- six inline rationale comments in tests/__main__ removed (test
names already encode the intent)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
After 10cb02461 switched `payload._resolve_variant_payloads` from
`get_variants` to `get_variant_overrides_only`, matrix-deploy bakes
only the raw `meta/variants.yml` slice into host_vars (typically just
`services:`). Lookups that read `variables['applications']` raw no
longer see content from `meta/rbac.yml`, `meta/server.yml`, etc.
`rbac_group_path` was the last lookup still going through the raw
host_vars read. For joomla (whose `meta/rbac.yml` declares the
`editor` role) the post-10cb02461 host_vars carried no `rbac` block
at all, the lookup walked an empty `rbac.roles` map, and the deploy
crashed with "rbac_group_path: role 'editor' is not declared under
applications[web-app-joomla].rbac.roles. Declared roles:
['administrator']" (job 77797371433 on run 26428080957).
Route `_get_application` through `get_merged_applications` so the
meta-merged view (services + server + rbac + volumes + schema
credentials + variant overrides) is restored. This mirrors the
existing pattern in plugins/lookup/{config,tls,nginx,domain}.py and
keeps the mirror-override intent of 10cb02461 intact: the bake step
still emits only the variant overrides; the merged cache assembles
the full view at lookup time.
Test updates:
- TestRbacGroupPathLookup.setUp patches get_merged_applications to
read from variables['applications'] so the existing 14 tests stay
hermetic without disk-reading roles/*/meta/*.
- New TestRbacGroupPathReadsMergedApplications regression case
exercises the exact 10cb02461 scenario: host_vars only carry a
variants.yml slice, the merged cache supplies the rbac block, the
lookup returns the correct path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Matrix-deploy variant transitions wipe a consumer's compose stack and
its named DB volume (`compose down -v` in
scripts/container/purge/entity/compose.sh) but `databases.csv` at
$FILE_DATABASE_SECRETS survives the purge. When the next round drops
the consumer from its inventory, the stale baudolo-seed row remains
behind. The subsequent backup run reads that row, attempts
`mariadb-dump -u<entity> -p<old_pw>` against the freshly initialised
RDBMS user that no longer exists, and the systemd unit fails with
"Access denied for user 'matomo'@'127.0.0.1' (using password: YES)"
(job 77797356000 on run 26428080957: matomo dump in
svc-bkp-container-2-local variant 1, where round 1 seeded matomo but
round 2 only deploys postgres + mariadb + svc-bkp itself).
Add `utils.cleanup.databases_csv` mirroring the existing
`utils.cleanup.tokens` pattern: parse the baudolo-seed CSV format
(';'-separated, four columns instance/database/username/password, with
header — verified against baudolo.seed.__main__:check_and_add_entry)
and strip every row whose `database` or `username` column matches an
entity derived from the supplied app ids via
utils.roles.entity_name.get_entity_name. Header and blank lines are
preserved verbatim; the file is rewritten only when at least one row
was removed.
Wire the helper into scripts/container/purge/apps.sh right after the
token-store wipe so every matrix-deploy inter-round purge invokes it
alongside the other cleanups. Operator-triggered purges
(`INFINITO_APPS=<app> bash scripts/container/purge/apps.sh`) also get
the cleanup — that is a behaviour change relative to before, where
the row would survive an operator-triggered entity teardown.
Coverage: 12 unit tests in tests/unit/utils/cleanup/test_databases_csv.py
covering the happy path, the multi-app case, the
`database='*'`/username-match case, the unknown-app no-op, header and
blank-line preservation, the all-rows-removed case, malformed short
rows, and the legacy-header-less file. `make test` green across all 5
targets.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- web-opt-rdr-www canonical: rdr-www.{DOMAIN_PRIMARY} -> w3redirect.{DOMAIN_PRIMARY}.
- New lookup `www_redirect_domains` (role-scoped): returns full {source, target}
mapping list, resolves DOMAIN_HOMEPAGE target for the role's own canonical via
lookup('domain', 'web-opt-rdr-www'). REDIRECT_WWW_DOMAINS now carries dicts;
tasks/main.yml and _01_cloudflare_edge_redirect.yml updated to consume dicts.
- New lookup `current_play_redirect_domains`: replaces the two inline
set_fact merges in tasks/stages/01_constructor.yml. Merges user-supplied
redirect_domain_mappings, primary-redirect (conditional on web-opt-rdr-domains
in deployed), and per-app domain mappings (inlined from the now-removed
domain_mappings filter).
- domain_redirect_mappings filter removed; web-app-keycloak RedirectUrisTest
pollution of utils.roles.applications.config in sys.modules fixed via
tearDownClass cleanup.
- csp_skip_domains lookup: csp.health.skip opt-out path removed (architecture
no longer needs it).
- Lint test_only_web_app_roles_integrate_dashboard expanded from web-app-* to
web-*; test_role_and_group_vars_used now scans .py files too so vars consumed
only inside plugins (DOMAIN_HOMEPAGE, AUTO_BUILD_ALIASES) are seen.
- New role-scoped variants.yml on web-opt-rdr-www with dashboard on/off.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Since 576fad84f (2026-05-10) moved role images from
`defaults/main.yml` into `meta/services.yml`, matrix-deploy's variant
baker has been writing the FULL effective config (services.yml +
variants.yml deep-merged via `get_variants`) into host_vars through
the `--vars` JSON consumed by `apply_vars_overrides`. That populates
`applications.{role}.services.{svc}.image` (and `.version`, .cpus,
etc.) with the upstream value, which then silently disables
`apply_mirror_overrides` for every role with an explicit image
declaration: the default `if_missing` policy short-circuits when the
field is already non-blank, so the rendered compose / Dockerfile FROM
keeps pulling from the upstream registry rather than the GHCR mirror.
The regression went unnoticed because Docker Hub usually serves the
pull, but it eventually surfaced as a BuildKit 429 on
`docker.io/library/php:8.2-apache` during the suitecrm Dockerfile
build (job 77602110531 on run 26362676803). Nextcloud, moodle,
mariadb consumers, postgres consumers, simpleicons, phpmyadmin,
phpldapadmin, jenkins, baserow, fider, ... all share the same latent
defect — only suitecrm happened to hit the rate limit first.
Fix the root cause instead of papering over per-role with
`mirror_policy: force`: introduce `get_variant_overrides_only` next
to `get_variants` and switch payload.py to it. The new function
returns the raw `meta/variants.yml` entries per role — no merge with
`meta/services.yml`. matrix-deploy now bakes only the variant-
specific overrides into host_vars; the role's services.yml fields
flow through the normal Ansible lookup chain at deploy time, leaving
`image`/`version` blank in host_vars so `apply_mirror_overrides`
populates them from `mirrors.yml` under its default `if_missing`
policy.
Behavior preserved for the other `get_variants` consumers:
* utils/manager/inventory.py:104 (schema-driven credentials) keeps
the full merged variant payload.
* planner.py:58 (variant-count only) is unaffected.
* legacy_resolver.py:42 (CombinedResolver runtime services_overrides)
keeps the full merged view.
Coverage:
* 3 new tests in test_applications.py for the new helper (single
empty variant when no meta/variants.yml, raw override without base
merge, cache invariance).
* test_payload.py + test_builder.py mocks repointed from `get_variants`
to `get_variant_overrides_only`.
* `make test` green across all 5 targets (lint, unit, integration,
external, test-lint) — 188 tests total.
Side-effects assessed:
* Repo-wide grep confirms no template / task accesses
`applications.X.services.Y.image` directly via hostvars — every
consumer goes through `lookup('config', …)` which still deep-merges.
* Roles previously pulling upstream now resolve via the GHCR mirror
whenever the image is already synced (the `images-mirror-missing.yml`
workflow keeps the destination populated).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the nextcloud / friendica layout so each kix scenario lives in
its own `test-*.js` and `_shared.js` carries only the logic consumed
by more than one file. `playwright.spec.js` becomes a thin orchestrator
that pulls each test module's `register(shared)` hook.
Layout
playwright.spec.js orchestrator (test.use, beforeEach, requires)
_shared.js cross-cutting: env (appBaseUrl,
canonicalDomain, oidcIssuerUrl, logoutUrl,
oauth2Enabled, ldapEnabled), performKixLogin,
runKixLoginLogoutFlow, beforeEach, persona
+ service-gating re-exports
test-kix-root-tls-hsts.js `kix root emits TLS+HSTS`
test-login-administrator.js admin OAuth2-proxy + Keycloak + KIX-LDAP
flow (ADMIN_USERNAME/PASSWORD local)
test-login-biber.js same flow for biber + the Keycloak admin
REST helper `ensureUserInGroup`
(KEYCLOAK_*, KIX_USER_GROUP_PATH,
BIBER_USERNAME/PASSWORD local)
test-guest-persona.js `guest: public-landing → auth chain`
test-biber-persona.js `biber: app → universal logout`
Per the strict shared-vs-local split, only `appBaseUrl`,
`canonicalDomain`, `oidcIssuerUrl`, `logoutUrl` and the two service
flags stay in `_shared.js`; admin credentials, biber credentials and
the Keycloak admin API env live in their consuming `test-*.js`.
`ensureUserInGroup` moves with its sole caller (biber) and keeps its
`api.dispose()` cleanup.
Behavioral delta: env validation is now per-consumer — TLS / persona /
admin-login tests no longer fail in `beforeEach` for missing BIBER_*
or KEYCLOAK_*; they validate the env they actually touch. Matches the
nextcloud/friendica diagnostic pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Variant 1 of web-app-kix disables every dynamic service flag
(css, dashboard, email, ldap, logout, matomo, oauth2, prometheus —
meta/variants.yml:32-56). With `services.ldap.enabled=false`,
svc-db-openldap is not in the deploy closure, so the top-level
`LDAP.SERVER.DOMAIN` chain in group_vars/all/12_ldap.yml:14-16
resolves via `lookup('config','svc-db-openldap', ...)` and raises
hard the moment any task touches it.
tasks/01_core.yml rendered `zzz_auth_ldap.xml.j2` unconditionally,
the template referenced `KIX_LDAP_HOST` etc., the lazy `LDAP.*`
chain fired, and the deploy aborted before any companion subapp
ever reached Playwright (job 77602110489 on run 26362676803 has
zero variant-1 artefacts across the entire 14-app bundle, while
gitlab's variant-1 produced at least three).
Introduce `KIX_LDAP_ENABLED` in vars/main.yml as the single source
of truth (matches OPENTALK_OIDC_ENABLED / OPENTALK_TURN_ENABLED in
web-app-opentalk), and gate both consumers on it:
* the LDAP-config render task — skipped when LDAP is off
* the compose mount of the LDAP config file — emitted only when
LDAP is on, so the backend container does not try to mount a
missing path
Matches the persisted operator preference: auth services stay
optional — fix code, not variants.
Static check only (per operator request). Confidence ~70% that
this clears the immediate variant-1 failure point; possible
follow-up issues (oauth2-proxy absence, etc.) would only surface
once the kix log reaches further than the previous abort.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The biber sso-login spec timed out at 300s per attempt (688.751s
across 3 retries) on job 77602110523 of run 26362676803. Trace
analysis showed a single action accounted for the entire test
budget: `Press "Enter" on input[name=password]` ran for 312.33s.
The race the existing comment already flagged: biber's first OIDC
hop triggers OpenCloud's lazy user provisioning, Keycloak unmounts
the password form sooner than admin's already-existing path, and
Playwright's `.press("Enter")` actionability retry waits for the now
detached input to come back. Without an action-level timeout the
retry inherits the test-level 300s budget, the `.catch(() => {})`
fires too late, and the subsequent banner assertion crashes with
"Target page, context or browser has been closed".
Bound the press to 5s so the catch can swallow the failure
immediately; the `leftKeycloak` promise (60s) then waits for the
real navigation, which the page-snapshot confirms always reaches
the Files view in time. Admin keeps its 2.96s happy path (5s is an
upper bound, not a baseline delay).
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The biber-LDAP-login test asserted that biber's URL after logout
matches `/realms/.+/protocol/openid-connect/logout`, but its
variant gating (test.skip when `oidc || \!ldap`) means the test only
runs in nextcloud variant 2 — where meta/variants.yml explicitly
sets `services.logout.enabled=false`. Without web-svc-logout in
scope, no universal-logout JS is injected, the native Nextcloud
logout never hits Keycloak's end-session endpoint, and the assertion
times out 100% of the time (job 77602110528 on run 26362676803).
The combination `oidc=false + logout=true` is semantically
impossible: universal-logout coordinates logout across an OIDC realm
and has nothing to do when no OIDC session exists. The Keycloak
end-session assertion is structurally never reachable from this
test's scope.
Mirror the existing post-logout verification in
test-login-biber-oidc.js (which passed in variant 0): navigate to
/login and assert no authenticated Nextcloud shell is visible. That
check is variant-agnostic and captures the real semantic intent —
"after clicking Log out, biber is no longer signed in" — without
depending on the universal-logout injection chain.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`_apply_one_role_special_rules()` in utils/manager/inventory.py used
`bool(resolve_database_service_key(...))` to decide whether to
auto-generate a `database_password` credential for a role. The
resolver raises when multiple RDBMS keys are enabled — which is the
correct contract for compose_volumes / container_depends_on /
plugins.lookup.database (they need an unambiguous single key) but the
wrong contract for inventory's credential-generation pass: variant 1
of svc-bkp-container-2-local intentionally enables both mariadb and
postgres to exercise the engine-side dump path against both engines
at once, and the deploy aborted with `ValueError: multiple direct
database services are enabled: mariadb, postgres` during inventory
generation (job 77602095377 on run 26362676803).
Add a non-raising sibling helper `has_single_database_service()` that
returns True iff exactly one RDBMS service is enabled, and switch
inventory's call site to it. Multi-DB roles now skip the auto-
generated `database_password` (which was a phantom credential for
svc-bkp anyway — no schema.yml, no cross-role lookup; the role's own
`04_seed-database-to-backup.yml` receives `database_password` as a
caller-passed task var, not as a self-credential lookup).
`resolve_database_service_key` keeps its strict raise semantics for
the compose_volumes / container_depends_on / database-lookup callers,
which still need a single unambiguous key.
Add unit coverage for both functions (3 + 4 tests) in
tests/unit/utils/roles/applications/services/test_database.py.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The biber+admin side-by-side scenario asserted on `a[href*='/logout']`
to verify admin's session survives biber's logout, but in variant 2
(ldap-only, no oauth2-proxy) Friendica's frio theme renders the logout
link as `href="logout"` (relative, no leading slash). The substring
match `/logout` therefore never hit, the assertion timed out, and the
deploy failed at `Fail if there were Playwright failures` (job
77602110488 web-app-friendica plus 77602110466 web-app-fediwall which
re-runs the same spec via variant-2 bundling on run 26362676803).
The trace.zip of the failing assertion shows the page IS fully
authenticated — banner with the Administrator user-menu button, the
New post link, full sidebar navigation — and `id="topbar-first"
class="topbar" role="menubar"` is present in all four captured HTML
snapshots. Variant 0 (oauth2+ldap+oidc) passed with the old selector
because the oauth2-proxy round-trip emits `/logout` substrings in the
HTML that variant 2's direct ldap flow does not.
Switch the assertion to the same composite selector both Friendica
login helpers (loginViaOauth2ProxyAndFriendica, loginViaFriendicaDirect)
already rely on:
"#topbar-first, #navbar-apps-menu, a[href*='/logout']"
`#topbar-first` matches in both variants (verified in trace.zip),
keeps a[href*='/logout'] as a third OR option for forward
compatibility, and uses toBeVisible to match the login helpers'
`waitFor({ state: 'visible' })` semantics. Session-bleed detection
(the real intent of the test) stays intact: if admin's session were
killed by biber's logout, /network would redirect to /login where
none of the three selectors match.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ejabberd's HTTP listener binds only the explicit request_handlers
declared in configuration.yml.j2 (/admin, /api, /.well-known/host-meta,
/bosh, /ws). Bare `/` is intentionally unmapped — ejabberd responds
with HTTP 404 by design, while the actual XMPP traffic flows over
TCP 5222/5269 and is not HTTP-testable.
sys-ctl-hlth-webserver applies its DEFAULT_OK = [200, 302, 301] when
an app does not override `server.status_codes`, so the role's HTTPS
front fails the post-deploy health gate with `Got 404` and
sys-service-terminator aborts the deploy (job 77602110559 on run
26362676803).
Override `status_codes.default` to [404] — consistent with the role's
own Playwright baseline at roles/web-svc-xmpp/files/playwright/
playwright.spec.js, which asserts `/admin/` returns < 500 (not /),
and with the pattern already used by web-svc-html (403/200),
web-app-prometheus (200/302/405), web-app-jenkins (200/302/403) etc.
Failure modes remain detectable: ejabberd down still surfaces as 502
from nginx, which the [404]-only expectation rejects.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`generate_all_domains(include_www=True)` synthesizes `www.X` for every
canonical domain X and pushes them into CURRENT_PLAY_DOMAINS_ALL, but
they are never registered in the global canonical_domains_map or in
applications[*].server.domains. The strict `tls`/`cert` lookups
therefore raised `domain 'www.X' not found (domains/applications)`
when web-opt-rdr-www iterated REDIRECT_WWW_DOMAINS and called
`lookup('tls', 'www.X', 'enabled')` in the redirect-domain.yml TLS
gate — failing job 77602110568 on run 26362676803 (and cascading
through sys-front-tls/vars/main.yml, sys-front-tls/tasks/02_http.yml
and sys-front-tls-selfsigned/tasks/01_core.yml, all of which call
`lookup('tls', application_id, …)` with the same leaked www.X).
Add a single fallback branch to `resolve_term`: if the strict lookup
fails AND the term starts with `www.`, retry both resolver paths
(legacy reverse + applications) against the bare domain. The fix is
strictly additive — every previously successful term and every non-www
miss preserves its current behavior.
Coverage:
* test_www_prefix_falls_back_to_bare_domain
* test_www_prefix_with_unknown_bare_still_raises
* test_non_www_unregistered_still_raises (regression guard)
Verified statically via the in-container `resolve_term`:
www.infinito.example -> (svc-prx-openresty, infinito.example)
www.rdr-www.infinito.example -> (web-opt-rdr-www, rdr-www.infinito.example)
The global self-signed cert SAN list (cert.py:193) is built from
CURRENT_PLAY_DOMAINS_ALL, so it already covers the www.X variants —
nginx serves the redirect vhost with that cert and the TLS handshake
on the hlth-webserver check matches.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In variant 1, sys-stk-front-base loads web-opt-rdr-domains for the
redirect set; its `include_tasks` loop sets `domain` from each
`item.source`. Ansible's include_tasks vars precedence (22) outranks the
task-level `vars:` block (17) on the subsequent "Deploy NGINX config"
task in sys-stk-front-proxy, so role-specific proxy.conf.j2 templates
that reference `{{ domain }}` directly render with the leaked redirect
source instead of the deployed app's domain. nginx then has no server
block for the intended SNI → sys-ctl-hlth-webserver fails with
TLSV1_UNRECOGNIZED_NAME and the deploy aborts at sys-service-terminator.
The generic vhost template `sys-svc-proxy/templates/vhost/basic.conf.j2`
already carries the canonical workaround (lines 1-2):
{% set application_id = front_proxy_application_id | default(application_id) %}
{% set domain = front_proxy_domain | default(domain) %}
Apply the same rebind to the role-specific proxy.conf.j2 of web-svc-html
(both rebinds missing) and web-svc-file (application_id rebind already
present, domain rebind missing). `front_proxy_domain` is set_fact-frozen
at the start of sys-stk-front-proxy from the outer load_app context and
verified correct in the run log (line 390447, html job in variant 1).
Unblocks 4/5 cluster-2 jobs on run 26362676803:
* web-svc-html (77602110571) — direct
* web-svc-file (77602110558) — direct
* web-svc-asset (77602110556) — variant-1 bundle [asset, file]
* web-svc-legal (77602110552) — variant-1 bundle [legal, html]
web-svc-xmpp (77602110559) has a separate failure mode (HTTP 404, not
SNI mismatch) and is not addressed here.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both roles declared `required_by: categories: [web]`, which the
called-services verifier matches against `categories_of(role_id)` =
{top-level, top+subcat} — so `web-opt-rdr-domains` (top: web, sub:
web-opt) triggered the contract even though its 301-redirect vhost
template is self-contained and intentionally bypasses both the front
proxy stack and the global injection layer.
Narrow the contract to the two subcategories that actually render
proxy.conf.j2 and depend on the inj-all server/location snippets:
web-app and web-svc. Verified against the live `required_role_ids()`:
web-app-bluesky and web-svc-asset still mark both roles required;
web-opt-rdr-domains no longer does.
Unblocks job 77602110578 on run 26362676803. Static check only (per
operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
meta/variants.yml already deploys variants 1 and 2 without web-svc-coturn,
but controller.toml.j2 rendered [turn]/[stun] unconditionally and
vars/main.yml evaluated `lookup('config', 'web-svc-coturn', 'credentials.
auth_secret')` for those variants too. Since coturn is out of scope there,
its schema-declared `credentials.*` are unset and the lookup raises in
"Render OpenTalk controller config" (job 77602110542 on run 26362676803).
Gate the two TOML sections on a new `OPENTALK_TURN_ENABLED` flag that
reads opentalk's own `services.coturn.enabled`. The cross-role
`OPENTALK_TURN_HOST/PORT/SECRET` lookups remain lazy in vars/main.yml —
controller.toml.j2 is their only consumer and now skips them when
TURN_ENABLED is false.
Static check only (per operator request).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LibreTranslate's landing page inlines a `<script>` at line 90 to wire up
the language picker and translate form. Without `'unsafe-inline'` in
script-src-elem, sys-ctl-hlth-csp reports
`[CSP DOM] Blocked URI: inline` on libretranslate.infinito.example and
aborts the deploy in variants that disable the css/cdn injectors
(variant 1+). Same class as 27e2574d4 (bluesky) and 66745645e
(fusiondirectory).
Static check only (per operator request); pattern parity with the two
prior CSP fixes that were verified in-container.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Flatten records from all summaries and sort by ISO-Z timestamp so the
step-summary table is chronological across apps/variants/passes instead
of grouped per file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FusionDirectory's breezy theme inlines a `<script>` block on the login
page to drive form validation. Without `'unsafe-inline'` in
script-src-elem, sys-ctl-hlth-csp reports
`[CSP DOM] Blocked URI: inline` on fd.infinito.example and aborts the
deploy in variants that disable the css/cdn injectors (variant 1+).
Verified in-container by hand-patching the rendered nginx vhost,
reloading nginx, and re-running hlth-csp: fd.infinito.example reports
`No CSP or network blocks detected`.
GitLab 19.0.0-ee.0 puma crash-loops with
`NameError: uninitialized constant Gitlab::Redis::ALL_CLASSES` whenever
omniauth is not configured. The instrumentation/redis.rb autoloads
before lib/gitlab/redis.rb has the chance to define the constant; with
OIDC enabled the omniauth gem eager-loads gitlab/redis first and
silences the race. Without OIDC (variant 1/2) the race is deterministic
and the deploy aborts at hlth-webserver because lab.git returns 502.
- New plain `files/Dockerfile` prepends `require_relative "../redis"`
to `lib/gitlab/instrumentation/redis.rb`. Image/version arrive via
`ARG GITLAB_IMAGE` / `ARG GITLAB_VERSION` forwarded from compose
`build.args`, so `meta/services.yml` stays the single source of truth.
- `templates/compose.yml.j2` now builds `GITLAB_CUSTOM_IMAGE` locally
instead of pulling, with `pull_policy: never` and `build.args`.
- `vars/main.yml` adds `entity_name` and `GITLAB_CUSTOM_IMAGE`.
- `meta/services.yml` pins GitLab to `19.0.0-ee.0` (current latest
stable on Docker Hub) to make the patch reproducible.
Verified in-container: hand-applying the same require_relative makes
puma boot listening; `INFINITO_VARIANT=1 make compose-deploy
mode=update apps=web-app-gitlab` builds the patched image and Playwright
passes 2 / 4 skipped for gitlab variant 1.
The bluesky social-app bootstraps via an inline script tag and pulls
runtime config from `app-config.workers.bsky.app` and live events from
`live-events.workers.bsky.app`. Without these CSP relaxations the page
loads under variants that disable the css/cdn injectors (variant 1+),
sys-ctl-hlth-csp reports `[CSP DOM] Blocked URI: inline` on
script-src-elem and `Blocked URI: https://*.workers.bsky.app/config` on
connect-src, and the final-run service aborts the deploy.
Verified in-container by hand-patching the rendered nginx vhost,
reloading nginx, and re-running hlth-csp: web.bluesky.infinito.example
reports `No CSP or network blocks detected`.
FD has no native OIDC support; after the oauth2-proxy + Keycloak chain
settles, the user lands on FD's own breezy-themed login form. The
previous `body.toBeVisible()` assertion was flagged as hidden by the
theme's FOUC-prevent CSS, masking the otherwise-successful redirect
chain. Target a stable FD-rendered marker (the form's "Sign in" button)
so the OIDC variant-0 test proves what it intends to.
