Commit Graph
1311 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 bd233bdb96 fix(web-app-stalwart): migrate 0.16 stores with stalwart-migrate 1.0.4 and verify every account
stalwart-migrate 1.0.0 failed the final verify on echoserver (6 rows) and galaxyserver (2 rows), all x:BlockedIp entries that expire when the tool starts 1.x, so a final run could never pass; 1.0.0 to 1.0.3 also skip accounts without a change counter. The pin moves to 1.0.4, final runs with --verify-full instead of a sample of 16 accounts, and a failed final restarts 0.16 and prints the error and verify rows of the new report phase, since final-report.txt is only written by a passing run. The README describes how to read the rows and when to start over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 09:26:59 +01:00
kevinveenbirkenbachandClaude Opus 5.5 03217e415d Merge branch 'fix/web-app-nextcloud/swarm-init-gate' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:13:19 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a0c36d6e38 fix(web-app-nextcloud): evaluate occ status's exit code in the init gate
Gate pass 23 failed tests/lint/filesystem/shell/test_no_or_true.py on init-gate.sh: occ_status swallowed every exit code with || true. It now returns 0 with the status on success and maps 1 (occ error), 124 (timeout) and 255 (php fatal while the tree is copied or upgraded) to an empty status, the not-ready state both slots already handle. Any other code, such as 127 from a missing su, timeout or php, ends the gate with that code and a message instead of waiting forever. Both slots now call occ_status directly so that exit reaches the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:12:57 +01:00
kevinveenbirkenbachandClaude Opus 5.5 812d2329a7 Merge branch 'fix/web-app-nextcloud/swarm-init-gate' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:39:40 +01:00
kevinveenbirkenbachandClaude Opus 5.5 93e0e83d7b fix(web-app-nextcloud): fail fast when slot 1 finds a half-finished install
In swarm CI run 41206003651 (Nextcloud#11) the first occ maintenance:install failed on a config.php lock after the upstream entrypoint had already copied version.php. Upstream skips the install whenever version.php exists and never retries, so the replacement task served an uninstalled Nextcloud that occ status still reported as healthy, and slots 2 and 3 waited for installed: true until converge gave up after about 25 minutes.

Slot 1, and compose where TASK_SLOT is unset, now exits 1 with the cause and the recovery when version.php and occ exist, the init lock is free and occ status prints installed: false. The task crash-loops and the stack readiness check ends the deploy within minutes with that message in the service log. occ status runs with a 120 s timeout in both branches. Why the lock failed on NFS is not addressed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:32:08 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5861e8694c Merge branch 'fix/platform-admin/ensure-on-existing-installs' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:21:43 +01:00
kevinveenbirkenbachandClaude Opus 5.5 cc70de8f28 fix(platform-admin): create platformadmin on existing installs in six roles
Matomo, SuiteCRM, WordPress, MediaWiki, Joomla and Listmonk only realigned an existing admin login, so existing installs without platformadmin stopped the deploy. Each admin step now creates the account next to the old admin when it is missing and realigns it when it is an admin.

A platformadmin that exists without admin rights fails with its cause and remedy. MediaWiki alone promotes it, and only while SSO is the sole login and its confirmed email matches: there the account can only come from Keycloak's preferred_username. SuiteCRM keeps native login under SAML and lets users rename themselves, Listmonk's OIDC never creates that login, and Joomla's Keycloak plugin rewrites groups on every sign-in, so none of them promote.

Matomo, SuiteCRM and WordPress write through php://stdout and php://stderr: PHP 8.2 defines no STDOUT/STDERR for code fed on stdin. MediaWiki's account probe lives in files/sql/admin_state.sql and receives name and email as hex literals. A lint checks that each role's admin step creates a missing admin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:21:20 +01:00
kevinveenbirkenbach a697411cbe Merge branch 'feature/inventory-validate/pinned-user-passwords' into feature/store-i18n 2026-10-31 22:29:53 +01:00
kevinveenbirkenbachandClaude Opus 5.5 167cc22194 feat(inventory): fail validation on required users without a pinned password
A user the host's roles need but the inventory gives no password falls back
to a generated password that renders anew on every read, so password
converges set values nobody knows (echoserver and galaxyserver stopped at
web-app-matomo for that after passing validation). The validator now asks
provision's required_user_policies for each host's groups, merges the
host's users with Ansible's replace precedence and fails before the deploy,
naming the users and the provision command. Presence only: vaulted values
count, nothing is decrypted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 22:09:10 +01:00
kevinveenbirkenbachandClaude Opus 5.5 89063972a6 perf(test-e2e-playwright): order Playwright passes by meta/tests.yml run_after
A role's playwright.run_after names the roles its pass runs after; the new
playwright_order lookup reads it through role_tests (which now renders
Jinja with the queried role as application_id) and orders the round with
a stable topological sort, so only constrained roles move. web-app-docs
runs after every other invokable role: in CI run 41206003651 (Discourse#2)
its pass waited 3126 s for the docs server's Sphinx build while stalwart,
bigbluebutton, jitsi and prometheus (~400 s) queued behind it. Cycles,
self references, unknown roles and non-list values fail loudly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 21:41:55 +01:00
kevinveenbirkenbachandClaude Opus 5.5 edf75b49dc refactor(web-app-keycloak): group the RBAC REST scripts in files/python/rbac
Gate pass 15 rejected rbac_reconcile.py and rbac_realm_groups.py as a prefix group. They are now rbac/reconcile.py and rbac/realm_groups.py with mirrored tests; the shared rest_script runner copies a script from its subpath and stages and runs it under its file name next to keycloak_admin.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 19:40:58 +01:00
kevinveenbirkenbach 8a1c076f14 Merge branch 'perf/web-app-keycloak/rbac-groups-batch' into feature/store-i18n 2026-10-31 19:13:58 +01:00
kevinveenbirkenbachandClaude Opus 5.5 2ad6968246 test(web-app-weblate): keep the stand-ins next to the package tests
Gate pass 14 rejected the parent-relative imports (ruff TID252) the moved weblate_network tests used for _stand_ins; the helper now sits in the same test package and test_settings_override imports it downward.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 19:13:53 +01:00
kevinveenbirkenbachandClaude Opus 5.5 7d983fc66c perf(web-app-keycloak): reconcile the realm RBAC groups in one batched pass
The realm RBAC step ran one kcadm round trip per group path and member through ensure_group_path_members.sh, about 609 s per XWiki job. A Python helper now reads the realm's groups and memberships once, computes the missing paths and members, and applies only the difference in one run; each failing item still fails loud and names itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 19:07:32 +01:00
kevinveenbirkenbach 2b9773bd03 Merge branch 'fix/web-app-magento/two-factor-toggle-race' into feature/store-i18n 2026-10-31 19:01:06 +01:00
kevinveenbirkenbach fcc2441247 Merge branch 'fix/web-svc-collabora/onion-frame-ancestors' into feature/store-i18n 2026-10-31 19:00:59 +01:00
kevinveenbirkenbach e11e2817df Merge branch 'fix/web-app-docs/builder-timeout-and-prepared-glob' into feature/store-i18n
# Conflicts:
#	roles/web-app-docs/README.md
#	roles/web-app-docs/files/python/infinito_docs/build_queue.py
2026-10-31 19:00:53 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ced411d0de fix(web-app-magento): toggle the two-factor modules only when their state is wrong
Magento 2.4.8 wipes generated/ on every module:enable or module:disable, decided by the command name alone, and the role ran the toggle on every deploy. On CI Magento#0's update pass a Prometheus probe rendered a page during that wipe and wrote new generated classes into it, so rmdir failed and the deploy died. The role now reads module:status --enabled first and only flips the modules in the wrong state; on a fresh install the toggle runs before nginx starts routing to PHP, and a real flip also reloads nginx.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:58:00 +01:00
kevinveenbirkenbachandClaude Opus 5.5 bdc7408357 test(sys-svc-webserver-core): stub IS_SWARM_MODE for the access-log render
The access-log test from main (#1322) rendered nginx.conf.j2 with DEPLOYMENT_MODE, while store-i18n's template reads IS_SWARM_MODE, so every case failed with an undefined variable after the main merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:56:30 +01:00
kevinveenbirkenbachandClaude Opus 5.5 42983ded11 refactor(web-app-weblate): ship the network modules as the weblate_network package
Gate passes 12 and 13 rejected the prefix-grouped weblate_network_*.py modules, the stand_ins.py test helper and unmirrored test paths. The modules are now weblate_network/strategy.py and middleware.py, mounted as a namespace package under /app/infinito and referenced by their dotted package names; the helper is _stand_ins.py and the module tests mirror the package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:55:29 +01:00
kevinveenbirkenbachandClaude Opus 5.5 fa7555d2eb fix(web-app-docs): keep only the three release tags opened last on disk
Publishing tag v1.2.3 globbed prepared/v1.* and deleted every other v1.x tag's prepared tree, including one a parallel slot could be cloning from. With the exact glob each opened tag would keep its ~2.3 GB site and ~1 GB prepared tree forever, which 66 tags and a crawler following the version switcher turn into a full disk. Requests now stamp opened/<tag>, and after every build the builder moves out, under the slot lock, every tag beyond the three opened last that is neither queued nor building; latest and deployed always stay, and a dropped tag rebuilds on its next visit. The builder also clears scratch a killed predecessor left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:49:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5d2410f7e9 fix(web-svc-collabora): let every Nextcloud network sibling frame the editor
The onion Nextcloud page frames the clearnet Collabora editor, but the proxy's frame-ancestors only allowed 'self' and *.DOMAIN_PRIMARY, which matches neither the onion host nor its http scheme, so Chrome blocked the iframe and the editor websocket never opened. Collabora now adds every Nextcloud network origin, each with its own scheme, to its frame-ancestors through webserver_csp_whitelist_extra_by_app before the proxy stack renders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:42:09 +01:00
kevinveenbirkenbach e96dae47c2 Merge branch 'fix/web-app-weblate/onion-handover' into feature/store-i18n 2026-10-31 18:24:09 +01:00
kevinveenbirkenbachandClaude Opus 5.5 54eccfcee6 feat(web-app-weblate): serve clearnet and onion from one instance
Weblate built every absolute URL, the social-auth redirect and the CSRF origin from WEBLATE_SITE_DOMAIN, so the onion sibling either redirected to clearnet or failed the OIDC round trip. A settings override swaps in a network-aware social strategy and proxy middleware that take scheme and host from the request, drops the forced HTTPS redirect the http onion vhost cannot satisfy, and ALLOWED_HOSTS now lists every network sibling plus the container name and loopback. The two modules are separate because social_django loads the strategy at app-ready before defining load_strategy, which weblate.middleware imports back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:23:59 +01:00
kevinveenbirkenbach 1cae39e9e0 Merge branch 'fix/web-app-prometheus/cadvisor-oom' into feature/store-i18n 2026-10-31 18:02:31 +01:00
kevinveenbirkenbachandClaude Opus 5.5 06372544de fix(web-app-prometheus): keep cadvisor's scrape cost flat as containers grow
cadvisor ran with its defaults: every series of every cgroup carried the union of all containers' label keys (97 on echoserver) and every cgroup was re-read each second, so each scrape grew with containers times images and the 0.25-CPU cadvisor was OOM-killed in its 1g cgroup about every 13 minutes. It now stores only the compose and swarm identity labels and keeps housekeeping at 10s, below the 30s scrape interval; limits and metric groups stay unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:01:14 +01:00
kevinveenbirkenbach c8eb0d319a Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/sys-svc-webserver-core/README.md
2026-10-31 17:50:28 +01:00
kevinveenbirkenbachandClaude Opus 5.5 43bd6583e6 test(web-app-docs): reuse StopBuilderError in the timeout loop test and merge its nested with
Gate pass 9 on the merged store-i18n stopped at ruff N818 on the local Stop exception that the builder-timeout test defined next to the module's StopBuilderError from the build-before-clone tests, plus SIM117 on the nested with in the healthy-mirror test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:44:51 +01:00
kevinveenbirkenbach 69e0d967a0 Merge branch 'feature/swarm-inventory-extra-vars' into feature/store-i18n
# Conflicts:
#	Makefile
2026-10-31 17:19:25 +01:00
kevinveenbirkenbach afaf4eb659 Merge branch 'perf/ci/deploy-skips-unused-local-build' into feature/store-i18n 2026-10-31 17:18:08 +01:00
kevinveenbirkenbach 373330d040 Merge branch 'fix/web-app-docs/builder-timeout-and-prepared-glob' into feature/store-i18n
# Conflicts:
#	tests/unit/python/roles/web-app-docs/files/infinito_docs/test_build_queue.py
2026-10-31 17:18:02 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1e804115fc fix(web-app-docs): keep the builder alive through git timeouts and never wipe a healthy mirror
fetch() leaked subprocess.TimeoutExpired, which ended the builder thread while it kept the builder lock, so no replica built again. A clone killed at its timeout also left a HEAD that never resolves, and the HEAD-file check ran remote update on it forever. The loop and refs() now catch SubprocessError, fetch() re-clones when HEAD does not resolve, and the clone lands in repo.git.partial and replaces the mirror only after it succeeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:07:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 50b4fa38b2 perf(ci): skip the local stack image build the deploy jobs never run
On real GitHub Actions INFINITO_BUILD passed through from default.env as 1, so every deploy job built the infinito image locally (~5 min) and then pulled and ran the published ci image anyway (pull_policy always). An env handler now defaults INFINITO_BUILD to 0 there; act and jobs that set it themselves keep building.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 17:03:37 +01:00
kevinveenbirkenbach 620fafa6d1 feat(swarm): pass INFINITO_INVENTORY_EXTRA_VARS into the swarm-zombie inventory 2026-10-31 16:38:30 +01:00
kevinveenbirkenbachandClaude Opus 5.5 9f36a7ac6b perf(web-app-docs): build the deployed snapshot while the first mirror clone runs
run_builder fetched the git mirror before it claimed any build, so the deployed site, which copies the snapshot baked into the image and reads no mirror, waited for the first git clone --mirror (about 60-150 s on CI, 967 MiB with 1113 refs/pull/* heads). The builder now claims and submits DEPLOYED inside the pool before the first fetch; latest and the tags still wait for it, so none builds from an outdated head. The README no longer claims the fetch waits for a running build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 15:59:25 +01:00
kevinveenbirkenbachandClaude Opus 5.5 4592f2bf78 test(web-app-taiga): pass a neutral config-append marker in the static URL test
test_used_keys_declared reads every INFINITO_* literal in code as an environment key, so the marker string copied from vars/main.yml failed the lint without a registry entry. The marker only lands in the append's header comment, which the test does not inspect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 14:07:46 +01:00
kevinveenbirkenbach c976f51504 Merge branch 'fix/web-app-weblate/swarm-static-mountpoint' into feature/store-i18n 2026-10-31 13:27:52 +01:00
kevinveenbirkenbach d5a8997ff4 Merge branch 'fix/web-app-taiga/onion-design' into feature/store-i18n 2026-10-31 13:27:46 +01:00
kevinveenbirkenbach 5cb5acae6d Merge branch 'fix/lint/resource-budget-honours-pins' into feature/store-i18n 2026-10-31 13:27:42 +01:00
kevinveenbirkenbach ffcbd49883 Merge branch 'fix/web-app-docs/languages-404-duration' into feature/store-i18n 2026-10-31 13:27:39 +01:00
kevinveenbirkenbach 65bd855e24 Merge branch 'fix/prod/galaxyserver-branding-stall' into feature/store-i18n 2026-10-31 13:27:36 +01:00
kevinveenbirkenbach 394d58db85 Merge branch 'fix/prod/echoserver-ldap-users' into feature/store-i18n 2026-10-31 13:27:32 +01:00
kevinveenbirkenbach bb8c5236f7 Merge remote-tracking branch 'origin/main' into feature/store-i18n 2026-10-31 13:27:13 +01:00
kevinveenbirkenbachandClaude Opus 5.5 880eeeac5e fix(web-app-taiga): serve Django admin static files from the vhost's own origin
taiga-back's image settings build STATIC_URL from TAIGA_URL, the canonical clearnet domain, so the Django admin on the onion vhost linked base.css and its scripts to the clearnet host and the onion CSP blocked them (header background rgba(0, 0, 0, 0) in the design spec). STATIC_URL is now the same-origin path below FORCE_SCRIPT_NAME. The Jinja static() global of the e-mail engine keeps the canonical absolute URL, because a mail client resolves no same-origin path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 13:08:17 +01:00
kevinveenbirkenbachandClaude Opus 5.5 8d6040948c fix(sys-svc-webserver-core): match token routes on the decoded path
The access-log map read $request_uri, which keeps percent-encoding and
repeated slashes. A request such as /%70assword/reset/<token>,
/password%2Freset/<token> or //password//reset/<token> missed the
token-route expression, fell through to the plain-path branch and was
logged with the token in plain text, while the backend still decoded it
to its reset handler.

The map now reads $uri: decoded, slash-merged (merge_slashes is not
overridden anywhere) and without the query string, so the existing
expressions match every spelling of a token route. Trade-off: after an
internal redirect the log shows the internal URI; with the generic
vhost's error_page (ws_generic.conf.j2) a 5xx line now carries
/500.html instead of the client's path, with the status unchanged.
Nothing in the repository parses the proxy access log.

The unit test resolves the map against the variable it reads, modelling
$uri as nginx builds it, and adds the three encoded spellings above.
Negative control: with the map back on $request_uri, exactly those three
cases fail (3 failed, 8 passed).

Reported by CodeRabbit on #1322.

Part of #1192

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 12:46:31 +01:00
kevinveenbirkenbachandClaude Opus 5.5 76fc6992c4 fix(variants): apply a variant pin only while its carrier is in the round
Pins of every role applied at every round index, so a role the round did not deploy could pull a partner into a heavier variant (web-app-discourse#1 measured 35.46 GB against the 32 GB reservation cap). resolve_round now starts every role on its clamped index and recomputes the include and the pins in force until the assignment settles, raising VariantPinError on a cycle. The planner resolves each round once (39.1 s to 16.0 s for all rounds), affected judges every (role, index) with the round the planner deploys for it, and the resource budget lint measures the pin-aware rounds. The stale web-app-matomo pin above the primary-only docs entry is dropped, and test_variant_pin_fixed_point.py guards every planned round against unsettled pins and pins above a disabled entry. docs/contributing/artefact/files/role/variants.md explains the settle loop with diagrams.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 12:45:53 +01:00
kevinveenbirkenbachandClaude Opus 5.5 f0a70b8164 fix(web-app-docs): answer 404 for an untranslated language before the deployed build
The server answered 202 for every /deployed/<code>/ path until the deployed Sphinx build finished (about 80 min with -j1), so test-languages.js:61 outwaited its poll budget on attempt one and passed only on retry. The untranslated decision now reads the snapshot's meta/languages.yml and docs catalogs, the same files the build writes into the language index, so the 404 set is identical before and after the build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 11:58:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 49387850dc fix(deploy): check the deploy extra before running ansible-playbook
The dedicated deploy launched whatever ansible-playbook PATH found. A stale pkgmgr venv without Pillow crashed the galaxyserver deploy after 4h23m at the first branding task. The runner now verifies every distribution of the pyproject deploy extra in the running interpreter before make setup, exits naming the missing ones with the install command, and runs the ansible-playbook next to that interpreter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 11:56:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 299aa8e707 fix(svc-db-openldap): name the user whose record lacks LDAP entry fields
A user key that no role declares, such as an inventory leftover, gets no username, firstname, lastname, uid or gid derived. openldap_user_entries then crashed with a bare KeyError 'username'. The filter now raises an AnsibleFilterError that names the user and the missing fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 11:56:12 +01:00
kevinveenbirkenbach dbcc78e376 Merge remote-tracking branch 'origin/main' into fix/sys-svc-webserver-core/access-log-no-tokens/1192 2026-10-31 10:40:44 +01:00