The note about dropping `PERSONA_*_BLOCKED` once upstream adds an
accessible label on Bookwyrm's profile-avatar dropdown is already
captured in `templates/playwright.env.j2` (the env file has the
same rationale as a comment alongside the actual flags). Keeping
a parallel TODO.md just creates drift risk.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the role's `sys-stk-full` deploy block into `tasks/01_core.yml`
(matching the pixelfed / keycloak / openproject convention), with
`tasks/main.yml` reduced to a single guarded `include_tasks: 01_core.yml`.
No behavior change — the role still runs once per playbook via the
`run_once_web_app_bookwyrm` flag, and `sys-stk-full` still flushes
handlers on return so containers are up before any follow-up step.
The refactor exists so future bootstrap tasks (install-mode dismissal,
admin provisioning, oauth2 wiring) have a clean docking point next to
the stack include without re-litigating the file layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25890032686 v0+v1 of `web-app-nextcloud` deployed cleanly
(PLAY RECAP `failed=0`); only v2 fail'd in
`tasks/plugins/user_ldap.yml:69` with `Failed opening required
'/var/www/html/lib/versioncheck.php'`.
Source-code review against the Nextcloud `stable33` upstream:
- `lib/versioncheck.php` exists in the source tree (726 B, sha
dd7790ff…).
- `console.php:14` loads it via `require_once __DIR__ . '/lib/versioncheck.php'`
— `__DIR__` resolves at compile time to the absolute directory of
`console.php`; cwd is irrelevant.
- The `nextcloud:33-fpm-alpine` Dockerfile declares no `WORKDIR` and
sets no `open_basedir`, so neither a missing image-WORKDIR nor a
PHP-include-path restriction can produce this error.
The remaining mechanism is that v2's matrix-deploy round short-
circuited the entrypoint's `rsync /usr/src/nextcloud/ → /var/www/html/`
such that the source did not stage into the `data:` named volume. The
same env-flake pattern produced the prior C14 (`Address already in use`)
in CI 25774452286 — v0+v1 clean, only one variant hit a round-isolated
state.
Re-strike `web-app-nextcloud` in both matrices with v0/v1/v2 ✅, mark
the `C-new-nextcloud-ldap-warm` cluster as `(environmental, not
actively remediated)`, and update the CI 25890032686 follow-up
snapshot to reflect the re-classification.
Recurrence in a future CI run with the same v2-only signature would
warrant re-opening; until then no role-level remediation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI Run 25890032686 web-app-fediwall job 76093401474 failed the
`each wall surfaces biber's posts according to its servers list`
test with a 300s timeout. The Playwright error-context page snapshot
showed the Keycloak form filled (username `biber`, password active)
but the Sign In button never clicked.
The inline `fillKeycloakLoginIfPresent` looked for the submit button
via `input#kc-login, button#kc-login, button[type='submit'],
input[type='submit']`. Modern Keycloak themes render the button as
`<button>Sign In</button>` with no `id="kc-login"` and no explicit
`type="submit"` — none of those selectors match, the locator returns
zero hits, the `.click()` never fires, the test sits on the login form
until the 300s timeout.
The shared `performKeycloakLoginForm` in
`roles/test-e2e-playwright/files/personas/utils/keycloak.js` prepends a
role-based locator (`getByRole("button", { name: /sign in|login|log in/i })`)
that catches this exact rendering, with the legacy CSS selector list as
fallback. It is the helper every other role's spec uses.
Drop the inline function, import `performKeycloakLoginForm` from
`./personas`, and call it directly inside `loginToMastodonViaOidc`.
The call site no longer branches on a "is the form there?" boolean —
the helper `waitFor`s the username field itself and throws on timeout
with a clearer message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI Run 25890032686 web-app-odoo job 76093401665 failed at
roles/web-app-odoo/tasks/04_ldap.yml:54 (Create new LDAP configuration)
with `invalid input syntax for type integer: "uid"`, LINE 12 of the
INSERT being the `user` column.
Odoo 19's `res.company.ldap.user` is a `Many2one('res.users', ...)` —
i.e. an integer FK to the *template user* to copy when auto-provisioning
LDAP-authenticated users. It is NOT the LDAP attribute name, which lives
inside `ldap_filter` (e.g. `(uid=%s)`). The role was sending the string
`'uid'` (from `ODOO_LDAP_USER_ATTR`) into the integer FK column.
Per `addons/auth_ldap/models/res_company_ldap.py` @ 19.0:
- `user` has no `required=True` flag, so the column is nullable
- `_get_or_create_user` falls back to `SudoUser.create(values).id`
when `conf['user']` is falsy, materialising users with the defaults
from `_map_ldap_attributes` (name, login, company_id, optional email)
Drop the column from both INSERT and UPDATE, drop the matching key from
`ODOO_LDAP_CONFIG`. `ODOO_LDAP_USER_ATTR` remains because it still backs
`ODOO_LDAP_FILTER`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the bespoke ``_emit`` that hand-rolled ``::warning file=...::``
directives written to ``sys.__stderr__`` and switches to the shared
``utils.annotations.message.warning`` helper so the PR-side annotation
shape matches every other lint:
::warning title=Deprecated pkgmgr role include,file=X,line=Y::body
Before, GitHub rendered the annotation with no title column and a body
that duplicated the location prefix (``X:Y: deprecated 'pkgmgr' ...
Line: name: pkgmgr-install``). After, the title column carries the
rule name, ``file=`` / ``line=`` carry the location, and the body
contains just the message plus the offending snippet in parentheses.
The local pytest warnings-summary line keeps its ``path:line: body``
shape via ``warnings.warn`` so terminal output stays scannable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OpenProject 17.4 (released 2026-05-13, picked up by our `version: "17"`
rolling tag) added a hard boot check that refuses to start when
`SECRET_KEY_BASE` is the image default `"OVERWRITE_ME"`. Web container
crashed within seconds, leaving `rails db:migrate` to fail with
rc=137 (compose exec killed alongside the dying container).
Declare `credentials.secret_key_base` (sha256-generated 64-char hex) in
the role's `meta/schema.yml` and render it as `SECRET_KEY_BASE=…` in
`templates/env.j2`. Same pattern already used by web-app-mastodon.
Persistent across deploys so cookies / encrypted DB columns stay
readable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Modules imported by tests run inside the same pytest process as the
tests themselves. Raw ``path.read_text(...)`` calls in those modules
bypass the cache the tests just populated. ``roles/`` stays excluded
(deployed scripts have no ``utils/`` on PYTHONPATH); ``utils/cache/``
stays excluded (cache implementation itself).
Five conversions to ``utils.cache.files.read_text``:
- ``utils/handler/yaml.py`` (YAML loader, read-only)
- ``utils/annotations/summarize.py`` (CI log scanner, with
``UnicodeDecodeError`` fallback replacing ``errors="replace"``)
- ``cli/meta/roles/applications/resolution/combined/__main__.py``
(single-shot CLI JSON input)
- ``cli/core/help.py`` (read_folder_title + read_folder_description;
same README is read many times across CLI help iteration)
Eight ``# nocheck: cache-read`` markers, each annotated inline:
- ``utils/claude/settings_sort.py`` (read then rewrite sorted)
- ``utils/docker/version_updater.py`` (read then rewrite config)
- ``cli/administration/deploy/development/network.py`` (tests mock
``Path.read_text`` directly; caching would bypass the mock)
- ``cli/administration/deploy/development/log_utils.py`` (append flow)
- ``cli/contributing/changelog/archive/changelog_md.py``,
``__main__.py``, ``package_mirror.py`` (file mutated in same flow)
Two callers refactored to variable extraction so ruff's 88-char wrap
keeps the ``# nocheck`` marker on the ``read_text(...)`` AST span line
(``cli/.../network.py`` and ``package_mirror.py:105``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Widens the cache-read lint guard from ``tests/lint/`` to every ``.py``
under ``tests/``. Integration tests show the same compounding cost
pattern (one role's compose template / nginx conf / scrape fragment
read across many shape tests) that motivated the original lint for
lint/.
Converts 91 violations across 28 test files:
- **Cache (utils.cache.files.read_text)** for project-tracked reads
and single-read tempdir fixtures, e.g. test_prometheus.py (32x),
test_keycloak_import_flow.py (11x), test_dashboard_oidc.py (3x),
test_changelog_md.py (8x), test_package_mirror.py (8x), plus
test_logging.py / test_summarize.py / test_version_updater.py /
test_get_url_retry.py / test_pull_specific_host.py.
- **Cache + UnicodeDecodeError handler** for project-walk scanners
that previously used ``errors="replace"`` / ``errors="ignore"`` to
defend against non-UTF-8 files (compose volumes, password quoting,
template image, mailu dependency, URL reachability). Wraps each
read in ``try/except (OSError, UnicodeDecodeError): continue``,
which has the same observable effect (skip undecodable files).
- **Refactor away the binary heuristic** in test_no_raw_docker.py:
``read_bytes() + b"\x00" not in raw`` becomes
``read_text() / except UnicodeDecodeError``. Same files skipped,
with cache benefit.
- **Variable extraction** in test_changelog_md.py / test_package_mirror.py
for cases where ``self.assertEqual(path.read_text(...), expected)``
exceeded ruff's 88-char limit. Ruff was wrapping the marker onto a
line outside the AST span of the read_text call.
- **Compute-from-setup** for two test methods that previously read
``self.changelog`` twice (before/after) to verify a no-op:
``before`` is now derived from the known input, dropping the first
read.
Five ``# nocheck: cache-read`` markers remain, each annotated inline
with the reason: idempotency tests where the second read must hit
disk to catch non-idempotent regressions, and tests with intermediate
``write_text`` whose subsequent read would return stale cache data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`openssl req -x509` without explicit `-addext` produced a CA cert
without an X.509v3 keyUsage extension. Python 3.13's stricter ssl
module refuses to load the system trust bundle if any anchor lacks
keyUsage, so every outbound HTTPS — including hits against the
cache frontend that proxies codeload.github.com — fails with
`CA cert does not include key usage extension`. mediawiki's
OpenIDConnect extension download failed on this in CI.
Add `keyUsage=critical,keyCertSign,cRLSign`,
`basicConstraints=critical,CA:TRUE`, and `subjectKeyIdentifier=hash`
to the CA self-sign so every freshly generated CA is acceptable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* sys-svc-compose/templates/networks.yml.j2: declare the per-role
`default` network as `external: true`. The network is pre-created
by `sys-svc-compose/tasks/utils/network/create.yml` via
`community.docker.docker_network` (no compose labels), so the
second `compose up` after an inter-round purge refuses with
`network <name> was found but has incorrect label com.docker.compose.network
set to "" (expected: "default")` — observed on web-app-matrix in
CI run 25868178119. External lifecycle keeps compose out of the
label race and matches the actual architecture (we manage the
network ourselves via Ansible).
* web-app-odoo/tasks/04_ldap.yml: rename SQL column `company_id` to
`company` in both the UPDATE and the INSERT. Odoo 19's
`auth_ldap` model declares `company = fields.Many2one('res.company', ...)`
(no `_id` suffix), so the actual DB column is `company`. CI run
25868178119 failed with `column "company_id" of relation
"res_company_ldap" does not exist`. Verified against
https://github.com/odoo/odoo/blob/19.0/addons/auth_ldap/models/res_company_ldap.py.
Python named-arg keys keep their `company_id` name (`ODOO_LDAP_CONFIG.company_id`
in vars/main.yml stays as is).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25868178119 — both jobs failed Playwright with the same
signature as web-app-kix's pre-fix state:
Error: no in-app logout control reachable on the current
authenticated surface
The generic `runBiberFlow` / `runAdminFlow` helpers cannot find an
in-app logout control on:
* Taiga's post-login UI (the bespoke `dashboard to taiga login and
logout` test at spec line 543 drives Taiga's own logout flow).
* Mattermost's `/landing#/` post-login surface (the bespoke
`dashboard to mattermost: sso login, verify channel view, logout`
test at spec line 273 drives Mattermost's own logout flow).
Declare `PERSONA_BIBER_BLOCKED=true` + `PERSONA_ADMINISTRATOR_BLOCKED=true`
per Rule 11 so the generic flows skip cleanly. Same pattern already
applied to web-app-{kix,nextcloud,postmarks,bluesky,fediwall}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Counterpart to `test_env_keys_used.py` — that one catches dead env
declarations (env declared, spec never reads), this one catches the
inverse failure mode (spec reads, env never declares).
The kix `DASHBOARD_BASE_URL` regression from CI run 25868178119 made
every persona test crash on the preamble assertion
`expect(dashboardBaseUrl, "DASHBOARD_BASE_URL must be set").toBeTruthy()`.
The existing env→spec lint didn't catch it because the failure was
in the opposite direction. This test scans `roles/<role>/files/playwright.spec.js`,
extracts every `process.env.X` / `process.env["X"]` / `readEnv("X")`
reference, and fails loudly if X is not declared in the sibling
`roles/<role>/templates/playwright.env.j2`.
Scope is narrowed to ONLY `playwright.spec.js` — other JS files
under `<role>/files/` (e.g. `login-broker/server.js` in web-app-bluesky)
are runtime app code whose `process.env` reads come from compose env,
not the test contract.
Excludes Node.js / Playwright / test-runner runtime names
(`NODE_ENV`, `NODE_TLS_REJECT_UNAUTHORIZED`, `PLAYWRIGHT_KEEP_ALL`,
`PWDEBUG`, etc.) via `_RUNTIME_ENV_NAMES`.
Verified: passes against the current 44 specs that ship an env
template, fails with a precise diagnostic when `DASHBOARD_BASE_URL`
is removed from `roles/web-app-kix/templates/playwright.env.j2`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two reasons the kix Playwright run was failing all 5 tests:
1. `DASHBOARD_BASE_URL` was missing from the env. Every persona reads
it via the role spec's preamble assertion
(`expect(dashboardBaseUrl, "DASHBOARD_BASE_URL must be set")`),
so the absence short-circuited all five tests before any auth
step ran. Render the same dashboard URL the rest of the persona
surfaces use — `lookup('tls', 'web-app-dashboard', 'url.base')`.
2. The generic `runBiberFlow` (test 174: `biber: dashboard → app →
universal logout`) cannot drive KIX's native LDAP login form
that appears AFTER the Keycloak OIDC round-trip — that LDAP-form
step is covered exclusively by the bespoke `runKixLoginLogoutFlow`
tests at lines 157+161. Without the block flag the generic biber
lands on the LDAP login form, never reaches the KIX UI, and fails
on the missing in-app logout control. Declare `PERSONA_BIBER_BLOCKED=true`
per Rule 11 so the generic flow skips cleanly.
In-container verified before deploy: 4 passed + 1 skipped, 0 failed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25868178119 — `Deploy web-app-postmarks` job passed (deploy +
Playwright), confirming the prior C2 Playwright regression is resolved
by the persona-helper fast-fail (commit a6ad5687a) and the postmarks
biber RBAC block via `PERSONA_BIBER_BLOCKED=true` (commit f9deebed3).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`3c1d5d821` solved the cross-round backbone-abriss problem by making
`include` equal the union of every variant's closure for every round.
Side-effect: every matrix round then ALSO deployed the full union, so
`08_deploy_performance.sh` saw `dashboard.infinito.example` still
answering HTTP 200 after a matomo-only `deploy-fresh-purged-apps` —
the deploy step was no longer matomo-only at all.
Split the two concerns instead:
- `include` (PlanEntry[3]) goes back to the per-round variant-closure.
Both `init`/`builder` (`--include` for the inventory bake) and
`deploy` (`--id` for ansible) consume it. Each round therefore
deploys ONLY its own variant-closure again — the pre-`3c1d5d821`
semantics.
- `purge_set` (PlanEntry[4], new) carries the union across all rounds.
`deploy.py` uses it for the inter-round wipe instead of the previous
round's include. Result: between rounds the WHOLE matrix footprint
is purged, not just the previous round's variant slice — a variant
pinning `services.<X>.shared: false` cannot leave its bundled
provider on the host for a sibling round that wires the same service
shared. The backbone-abriss `3c1d5d821` was hunting stays fixed:
whichever round needs the backbone re-deploys it immediately after
the wipe (purge runs at the start of each round, not the end).
`previous_round_include` tracking is gone. The first iteration skips
the purge (nothing to wipe yet), which also covers `total_rounds == 1`
and `--variant N` pinned single-folder modes cleanly.
Unrelated, but staged together because it is part of the same
investigation:
- `roles/dev-nix/templates/nix.conf.j2`: enable `keep-derivations` /
`keep-outputs`, so `nix-collect-garbage -d` (run by the aggressive
prune step) does not drop the build closure of pkgmgr's flake; this
was making every deploy re-fetch the build inputs from
cache.nixos.org and overshoot pkgmgr's update timeout.
- `roles/pkgmgr/tasks/01_core.yml`: raise the pkgmgr update timeout
from 600s to 1800s and lower the retry count from 6 to 3, so a slow
fetch on a single try no longer pushes total wallclock past three
hours.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the no-op `docker restart` recovery with `POST
/service/rest/v1/repositories/{name}/invalidate-cache` against each
apt-format proxy repo (Debian, Ubuntu, …) discovered dynamically
from Nexus' REST API. Container restart never re-validated cached
content; the new path makes Nexus actually re-fetch from upstream
on the next request and keeps repo definitions, blobstore, and
admin credentials intact.
Co-located changes:
- fresh-purged-app.sh: source scripts/meta/env/all.sh so the cache
override env (incl. INFINITO_PACKAGE_CACHE_FRONTEND_IP) is set
for the deploy CLI — without this, apt traffic bypasses Nexus.
- utils/nexus/list_apt_proxy_repos.py: stdin JSON → stdout name list,
exit 1 on malformed input / no matches; used by the bash helper
for discovery (parser lives outside the heredoc for testability).
- tests/unit/utils/nexus/test_list_apt_proxy_repos.py: 10 cases
covering filter rules, empty/invalid input, non-dict entries,
missing/empty names, and a realistic live payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `tests/lint/repository/test_pkgmgr_deprecated.py` and the matching
`pkgmgr-deprecated` rule in the suppression catalog. The check never
fails the suite; each unmarked `name: pkgmgr` / `name: pkgmgr-install`
line is surfaced as a `DeprecationWarning` locally and as a GitHub
Actions `::warning file=…,line=…::…` annotation in CI. The warning
message points at the available pip migrations (`kpmx`, `docoli`,
`git+infinito-nexus-core`). Per-line opt-out via
`# nocheck: pkgmgr-deprecated`.
Also compacts the suppression catalog page: drops the examples
section, replaces prose-style bullet lists with tables, shortens
catalog `Effect` cells, and aligns with `documentation.md` (no em
dashes, no soft-wraps inside sentences, link texts use filenames
instead of full paths).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 08 (`deploy-fresh-purged-apps … FULL_CYCLE=true`) iterates matomo's
two variants and skips the final inter-round purge, so the host ends on
variant 1 (mariadb.shared=false → embedded `database` service). The
previous pin to VARIANT=0 forced a variant flip (shared=true) on a host
still wired for variant 1, which left the freshly-rendered matomo
compose.yml referencing `database` without an image and tripped the
`compose config --quiet` validation handler with:
service "database" has neither an image nor a build context specified:
invalid compose project
Pinning the reuse pass to VARIANT=1 redeploys onto the same variant the
matrix last left running, which is the intended `reuse of existing
state` semantic from variants.md ("the last state remains available for
inspection or follow-up specs").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The three playwright.env.j2 templates rendered
`APP_BASE_URL=lookup('tls', 'web-app-dashboard', 'url.base')` — a
stray copy-paste from the dashboard-tile flow that put the persona
helper on the dashboard login instead of the role's own surface.
After the dashboard's Keycloak round-trip the URL stayed on
dashboard.infinito.example, the canonical-domain poll
(`taiga.kanban.*`, `mattermost.*`, `social.*`) timed out at 60s and
the universal-logout test failed with
`Expected redirect back to <app>.infinito.example after Keycloak login`.
Point each `APP_BASE_URL` at the role's own canonical URL (the
project-wide convention used by baserow, bluesky, bookwyrm, akaunting,
hugo, jenkins, mig, moodle, …). The persona helper now lands on the
role's surface, the OIDC chain redirects to that role's Keycloak
client, the assertion passes, and the in-app logout step has a real
authenticated surface to operate on.
A primary app's variant MAY pin `services.<X>.shared: false` to test
the bundled-provider path. That removes <X>'s shared provider from
that variant's closure; the inter-round purge then tears down a
backbone the next variant still expects, leaving consumers in the
following round on an empty dashboard (Index of /), a missing
postgres (5432 Connection refused), or a torn-down mailu.
Pre-compute every variant's closure once, then union them and use the
union as the include for every round. Variants keep changing HOW each
consumer is wired (baked host_vars per round); the deployed app set
stays the superset so purge → redeploy stays round-independent.
Postmarks gates its entire web UI through oauth2-proxy + Keycloak, and
biber is not a member of `web-app-postmarks-administrator` (the only
allowed group). Every biber request therefore lands on the oauth2-proxy
403 page with no in-app surface, and the universal-logout helper polls
for a logout marker until the 5-minute test-level timeout fires.
Render `PERSONA_BIBER_BLOCKED=true` in `templates/playwright.env.j2` so
the biber scenario skips cleanly per req-019 Rule 11 instead of
masquerading as a 300s hang.
The guest scenario's `page.goto` previously swallowed every navigation
error via `.catch(() => null)`. When the role's public surface was
unreachable, the subsequent CSP probe in `readCspString` ran another
HTTP request without an explicit timeout and fell back to Playwright's
30s default. Stacked on top of the stalled goto this consumed the 60s
test-level limit and surfaced as `Test timeout of 60000ms exceeded`,
masking the real cause (backend unhealthy).
Re-throw with a precise message when goto rejects so the CI artifact
points at the unreachable URL inside the 30s navigation timeout, and
cap `page.request.get` inside `readCspString` at 10s so a single
stalled CSP fetch can no longer add 30s to the test budget.
The register is set by the same task's `command:` module, so `.stdout`
is always defined when `until:` evaluates. Satisfies the
`no_redundant_default_on_module_io` lint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The mediawiki install/update step shells out to a throwaway mariadb
client container that connects to the shared database over the docker
network. Because the shared database lives in a separate compose
project (svc-db-mariadb), compose-level
`depends_on: condition: service_healthy` cannot reach it from
mediawiki's compose; `dmbs_excl.yml.j2` already skips the dependency
when `database.shared` is true. The install therefore raced the
mariadb container's startup and tripped over
`real_connect(): (HY000/2002): Connection refused`.
Adds an ansible-side wait task that polls the database container's
own docker healthcheck via `container inspect --format
{{.State.Health.Status}}` until it reports `healthy`, mirroring the
existing `Wait until container exists` task in the same file. Sixty
2-second retries cover the typical mariadb startup window; if the
healthcheck never goes green the task fails so the deploy stops
instead of silently rolling into the install race.
The previous emptiness check ran `\dt` against the listmonk database and
treated the absence of `No relations found.` / `Did not find any relations.`
in the output as proof that listmonk's schema was already installed.
Postgres now phrases the empty-result message as `Did not find any
relation` / `No matching relations found`, so the check kept reporting
the schema as present even when only system tables were there. The
`./listmonk --upgrade` path then ran against a structurally empty
database and exited with `the database does not appear to be setup. Run
--install.`.
Probe the canonical `subscribers` table directly and update the match
strings to postgres's current phrasing so `listmonk_db_empty` flips to
true on a fresh database and the install path runs first.
Tightens tests/lint/ansible/roles/test_compose_resource_limits in two
steps. First, the iteration scope is reduced from "every role with a
meta/services.yml" to "every role whose folder name starts with an
invokable-category prefix" (resolved via plugins.filter.invokable_paths.
get_invokable_paths). Non-invokable categories like sys-* never ship a
top-level compose service of their own, so leaving them out removes a
class of structural noise. Second, the warn-only sentinel
(`assertGreater(len(findings), 0)`) is replaced with a hard failure
that lists every missing key — a missing limit is now a build break.
Declares min_storage / cpus / mem_reservation / mem_limit / pids_limit
on the 48 invokable primary entities that previously lacked one or more
of the keys. Container roles (svc-db-{memcached,redis},
web-app-{bigbluebutton,matrix,oauth2-proxy}) get sizing tiers matched
to their workload: BBB heavy (4 cpus / 6g / 4096 pids), Matrix mid-high
(1.5 cpus / 2g / 2048 pids), redis / memcached / oauth2-proxy small.
Native-on-host roles (desk-*, drv-*, gen-*, svc-bkp-*, svc-net-*,
svc-opt-*, web-opt-rdr-*, web-svc-legal, web-app-socialhome) carry
approximate footprints of the underlying package(s); the keys do not
bind a container at runtime (no compose template includes them) but
document expected host consumption and satisfy the strict lint.
Pre-existing curated min_storage values stay (BBB's 50GB recording
budget, redis 1GB, oauth2-proxy 1GB, drv-intel 0GB, …) — only
placeholder shapes ("0GB" / "1GB" with quotes) get overwritten.
Restructures tests/lint/ansible/ into five concern-keyed subdirectories
(jinja/, modules/, services/, shell/, sql/) plus the pre-existing roles/
and variables/. Each new subdir ships its own __init__.py exporting
PROJECT_ROOT and a scope-focused README.md following the project's
documentation conventions (RFC 2119 keywords, emoji-after-headline,
one-sentence-per-line, no historical context).
12 existing tests move into their concern bucket via git mv so history
follows the file. Adds tests/lint/ansible/variables/test_dotted_path_keys_exist.py
which catches the bug class where a Jinja dotted path's head resolves
via group_vars/all/ or the consuming role's own vars/defaults but a
deeper segment does not exist as a key in the dict tree (e.g. the
previous `LDAP.SERVER.REACH_DOMAIN` typo in web-app-odoo).
Touches three external references so cross-links keep resolving:
- docs/contributing/actions/testing/suppression.md (run-once, shared)
- roles/sys-svc-compose/tasks/utils/network/README.md (network-via-util)
- tests/lint/ansible/sql/test_no_inline_multiline_sql.py (self-path exclude)
Adds tests/lint/ansible/test_no_jinja_default_on_spot_path.py: scans every
Jinja `{{ ... | default(...) }}` block under roles/, tasks/, group_vars/,
identifies the actual default target via a paren-aware left-walk from the
matched `|`, and fails when the target's dotted path resolves cleanly
against any group_vars/all/*.yml dict tree. Role vars/defaults are
intentionally out of scope (Ansible loads them only when the owning role
is in the play, so cross-role defaulting is a legitimate caller guard,
not SPOT decoupling). Per-line opt-out: `# nocheck: spot-default`.
Fixes the 21 violations the test surfaces:
- group_vars/all/00_general.yml: drop `| default('')` on CAPTCHA self-checks
- group_vars/all/02_tls.yml: drop `| default(...)` on DOMAIN_PRIMARY in CA subject
- roles/web-app-{listmonk,keycloak,espocrm}/meta/schema.yml: drop defaults on CAPTCHA.*
- roles/web-app-suitecrm/templates/env.j2: map empty SECURITY -> 'none' via `or`,
not via the default filter (SPOT value `""` is always set)
- roles/web-app-opentalk/tasks/main.yml: drop default on MODE_ASSERT (always bool)
- roles/user-workstation/tasks/01_core.yml: drop default on MASK_CREDENTIALS_IN_LOGS
- roles/test-e2e-playwright/tasks/run_one.yml: drop defaults on TLS_MODE + CA_TRUST.*
Also corrects the dotted-path bugs in roles/web-app-odoo/vars/main.yml that
referenced non-existent LDAP keys (`LDAP.SERVER.REACH_DOMAIN`,
`LDAP.USERS.SERVICE_ACCOUNT.*`, `LDAP.ATTRIBUTES.ID`, `LDAP.SERVER.SSL`) to
the canonical paths actually defined in group_vars/all/12_ldap.yml.
openwebui, minio, xwiki, shopware, wordpress, suitecrm, snipe-it all passed
deploy + Playwright in CI Run 25797277810; struck through with every declared
variant flipped to ✅ in both matrices.
Commit `1e5a47f67` added `PROMETHEUS_SERVICE_ENABLED=` to
`roles/web-app-gitea/templates/playwright.env.j2` so the spec's
`beforeEach` no longer aborts with `Received: undefined`, but the
two prometheus-dependent tests in `roles/web-app-gitea/files/
playwright.spec.js` never consumed the new flag through the standard
service-gating helper. That tripped Rule 2 of req-019 — every
`<NAME>_SERVICE_ENABLED=` line in the env template MUST be consumed
by ≥1 `requireService` / `skipUnlessServiceEnabled` / `isServiceEnabled`
/ `isServiceDisabledReason` call in the spec — and
`tests/integration/roles/test_playwright_spec_env_gates.py` failed
the assertion in the pre-commit hook.
Import `skipUnlessServiceEnabled` from `./service-gating` (the
same import pattern that web-app-akaunting / baserow / bookwyrm /
bigbluebutton / fediwall already use), and call it as the first
statement of:
- `metricz endpoint exposes gitea metrics when prometheus is loaded as dependency`
- `prometheus scrapes gitea native metrics — job target is up`
so both tests skip cleanly when the variant disables prometheus.
The existing manual `test.skip(...)` fallbacks for 404 / missing-URL
cases remain in place as belt-and-suspenders.
`python3 -m unittest tests.integration.roles.test_playwright_spec_env_gates`
now passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lint walk fix in commit 115196f28 dropped `.agents/skills/` and
`.claude/skills/` from the python / shellcheck / generic walks but did
NOT extend the same exclusion to `markdownlint-cli2`, which has its
own glob config in `.markdownlint-cli2.jsonc`. As a result every
`pre-commit run make-lint-markdown` flagged 14 baseline findings in
vendored skill README / SKILL.md files (MD041 first-line-heading,
MD045 no-alt-text) — those ship from upstream skill repos and follow
their own conventions.
Add `.agents/skills/**` and `.claude/skills/**` to the `ignores` array.
Drop the two patterns that match zero files today:
- `tests/**/fixtures/**` — there is no `fixtures/` directory anywhere
under `tests/`; the actual fixture files are Python modules named
`_fixtures.py` (already covered by the markdownlint glob's `**/*.md`
restriction).
- `docs/changelog/archive/**` — there is no `archive/` directory under
`docs/changelog/`.
`.github/PULL_REQUEST_TEMPLATE/**` is kept (still matches the six
template files).
Verified: `make lint-markdown` now reports `0 error(s)` over
686 files (was 716 walked / 14 errors before).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several entries in both requirements docs pointed at the first-iteration
fix locations that were later reverted or relocated. Bring the matrix
notes and cluster catalogue in line with the actual committed state:
- gitea + matrix rows: replace stale reference to a
`roles/sys-svc-compose/handlers/main.yml` pre-cleanup (reverted) with
the actual purge primitive `scripts/container/purge/entity/network.sh`
+ global `docker network prune -f` in `apps.sh` (commit c6affc96f).
- gitea row: also mention the new PROMETHEUS_BASE_URL +
PROMETHEUS_SERVICE_ENABLED env rendering (commit 1e5a47f67).
- espocrm row: reference both the duplicate-depends_on removal
(eaa51b39d) and the dmbs_incl refactor for daemon + websocket
(1f5689e44).
- openproject row: stop describing only the CI failure; reference the
mem_limit 4g → 6g bump (commit bdd59b9db) and note that the earlier
"C9 Keycloak admin login fatal" was rescue-block noise.
- fusiondirectory row: mention the services.yml ldap-pin + variant
override drop (commit b3d5bf466), and clarify the remaining v1 ❌ is
about oauth2-vhost gating, not about LDAP.
- lam + phpldapadmin rows: note the same ldap-pin landed for them too.
Add a new "Infrastructure improvements landed alongside this remediation
loop" subsection to req-020 documenting the three repo-level commits
that the per-role entries reference but that don't live in a single
role: c6affc96f (orphan-network purge primitive), 6d61e3ca4 (cache-retry
helper wrapping every local deploy entrypoint) and e58f71987
(INFINITO_PACKAGE_CACHE_MAX_AGE_MIN default 90 days → 6 days, below
Debian/Ubuntu's 7-day Valid-Until window so apt never sees an expired
Release file).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI run 25774452286 surfaced openproject's deploy aborting at
`Run database migrations` with rc=137 (SIGKILL = cgroup OOM) after
9.5 minutes and 30 retries. The task runs the migration via the
already-running web container:
/usr/bin/compose exec web bash -lc \
"cd /app && bin/rails db:migrate RAILS_ENV=production"
OpenProject's Rails 8 boot peaks well above its steady-state working
set during class-loading + schema-walk on a fresh database, and 4g
was right at the edge — each retry rebooted the process, peaked,
and got killed before the migration finished.
Bump `web.mem_limit` to 6g (peak observed ~5g in local runs, plus
headroom). `mem_reservation` stays at 3g (cgroup soft floor); on
the 16 GB public CI runner this is comfortable — total nominal
across the stack lands near ~12 GB with ~4 GB headroom.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The gitea spec at `roles/web-app-gitea/files/playwright.spec.js`
opens with
test.beforeEach(() => {
expect(prometheusBaseUrl, "PROMETHEUS_BASE_URL must be set in the Playwright env file").toBeTruthy();
expect(gitEaBaseUrl, "GITEA_BASE_URL must be set in the Playwright env file").toBeTruthy();
});
and contains a `metricz endpoint exposes gitea metrics when
prometheus is loaded as dependency` test that scrapes the prometheus
metricz surface for the gitea vhost. The Playwright env template
rendered `GITEA_BASE_URL` and `MATOMO_BASE_URL` but never
`PROMETHEUS_BASE_URL`, so every gitea Playwright run aborted in
`beforeEach` with `Received: undefined` — before any scenario got
a chance to execute.
Render the missing pair so the spec's `beforeEach` no longer
short-circuits:
- `PROMETHEUS_BASE_URL` from the `lookup('tls', 'web-app-prometheus',
'url.base')` so the metricz test has a target URL.
- `PROMETHEUS_SERVICE_ENABLED` from
`services.prometheus.enabled` so the spec can skip the
prometheus-scrape scenarios cleanly when the variant disables
prometheus.
Confirmed locally: deploy compose-up now reaches Playwright instead
of failing in beforeEach. Remaining gitea spec failures (`prometheus
scrapes gitea native metrics — job target is up`, biber /
administrator universal-logout timeouts) are independent issues that
need separate spec work and are tracked separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`docker compose down --remove-orphans` only deletes networks that
compose itself created (those carrying
`com.docker.compose.network=default` + `com.docker.compose.project=<name>`
labels). When the per-entity default network ends up unlabeled or
project-mislabeled — typical after an inter-round matrix-deploy
purge, or when a `community.docker.docker_network` Ansible call
materialised the network before compose ran — compose deliberately
leaves it untouched, and a subsequent `compose up` aborts with
network <entity> was found but has incorrect label
com.docker.compose.network set to "" (expected: "default")
The matrix-deploy CI rounds for web-app-matrix, web-app-gitea and
others reliably reproduce this.
Add a new entity-keyed purge primitive
`scripts/container/purge/entity/network.sh <ENTITY>` that runs after
the compose / dir / nginx primitives. It is idempotent: no-op when
the network does not exist, skips removal while active container
endpoints prove the network belongs to a shared service still in
use (svc-db-mariadb, svc-db-postgres, svc-db-openldap, svc-ai-ollama,
…), and otherwise drops the orphan network with `docker network rm`.
`docker network rm` failures are logged and tolerated — the next
`compose up` will surface the same problem with an actionable error.
Wire the primitive into `scripts/container/purge/apps.sh` immediately
after the existing `nginx.sh` call, so every entity-keyed purge
collects its own default network on the same pass.
Finally, after the entity loop, run a single bare `docker network
prune -f` to sweep orphan networks whose name does NOT correspond
to a purged entity (left over from renamed roles, abandoned
matrix-deploy rounds, etc.). The sweep is bare on purpose — no
`|| true` guard — so a daemon-side prune failure aborts the purge
loudly via the script's `set -euo pipefail`. Performance: ~13× faster
than calling prune inside `network.sh` per entity (one ~200 ms
listing pass vs. N × 200 ms for N entities).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three roles whose persistence layer IS LDAP — they cannot run without
it — had `services.ldap.shared` rendered dynamically as
`'svc-db-openldap' in group_names`, and their `meta/variants.yml`
explicitly pinned `ldap.{enabled,shared}: false` in the "everything-
off" variant. The combination meant the matrix-deploy variant-1
deploy actually tore LDAP out from under FusionDirectory / LAM /
phpLDAPadmin, leaving the upstream containers unhealthy and the
matrix-job red.
Fix the dependency declaration to match physical reality: pin
`services.ldap.{enabled,shared}: true` in each role's `meta/
services.yml` (with `# nocheck: dynamic-flag` rationale comments), and
drop every `ldap:` override from `meta/variants.yml` so each variant
inherits the pinned values. fusiondirectory's variants.yml already
carried an explanatory comment ("The LDAP marker stays on across both
variants because LDAP IS the storage backend") that contradicted the
actual variant body; align the body to the comment.
The split between "LDAP IS the storage" (storage-backend) and "LDAP
is an optional federation source" (consumers like Keycloak,
Nextcloud, Gitea, …) is intentional — only the storage-backend
roles get the pin; federation consumers keep the dynamic
`'svc-db-openldap' in group_names` resolution.
Local FULL_CYCLE v0 for `web-app-fusiondirectory` runs green with
this change. v1 + v2 fail later on an unrelated oauth2-proxy gating
issue (502 from openresty when oauth2 is disabled) and are tracked
separately; the ldap fix is a prerequisite for that work, not a
solution to it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
