Commit Graph
718 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 6e045bced8 fix(web-app-docs): encode line breaks in redirects, navigate by built URLs
- py/http-response-splitting (server.py:87): _redirect percent-encodes CR
  and LF in the Location header
- js/xss-through-dom (versions.js:82, languages.js:25): both switchers
  navigate to the URL they built instead of select.value

The new cases in test_server.py, versions.test.js and languages.test.js
fail without the fix and pass with it.

🎯 Fix: 92 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:54:27 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0af21e62ad test(web-app-pgadmin): read project files through the cached readers
test_no_uncached_file_reads and test_no_direct_yaml_calls failed test-lint in run 42056917806. env.j2, manager_ops.yml and the realign script now come from utils.cache; the before/after read of the tempdir SQLite file keeps a direct read with the cache-read opt-out, since a cached read would hide a write.

🎯 Fix: 97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:20:42 +00:00
kevinveenbirkenbachandClaude Opus 5.5 735308266e Merge two operator-approved sidetrees into feature/store-i18n
- fix/tests/store-i18n-lint-red (1205c4a2ae, 95/1): clears the raw-walk,
  project-root and env-key lint failures store-i18n would have pushed
- fix/sys-stk-front-proxy/body-filter-fold (4984f506c7, 92/2): the
  body_filter Lua is written in the vhost's single stack_host_contents
  call (about 184 s echoserver and 174 s galaxyserver per deploy,
  projected), and stack_host_contents compares bytes

🎯 Fix: 92-95 %
💥 Break: 1-2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90-94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:37:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4984f506c7 perf(sys-stk-front-proxy): write the body_filter Lua in the vhost's single contents call
Every front-proxy vhost wrote its body_filter Lua through a separate
stack_host_template task in sys-front-inj-all: two remote round trips when
unchanged (stat + file) and four to five when changed, 191 s over 54 vhosts
per echoserver deploy and 179 s over 48 per galaxyserver deploy. The Lua
now joins http.conf and the vhost in the Deploy task's stack_host_contents
call, written first as before, rendered through template_shallow with
application_id and domain pinned to the vhost.

- sys-front-inj-all/tasks/main.yml no longer writes the Lua; the new
  standalone.yml includes it and keeps the separate write for matomo's
  re-inject path, which has no vhost Deploy task
- stack_host_contents compares bytes instead of a universal-newline text
  read, so content with a CR stops reporting changed on every run and an
  existing non-UTF-8 file is replaced instead of crashing the module
- a new required diff_paths argument keeps --diff for the Lua only; vhost
  and http.conf content stay out of the diff

A Lua-only change still reports changed and fires reload-openresty once;
the served probe and the fallback redeploy keep their old triggers. Check
mode is unchanged.

Tests: the real stack_host_template action against the fold expression
across 99 roles and TLS/Tor/injection scenarios, byte-identical, with
trim and newline negative controls; owner, group and mode parity with the
copy module; byte compare with CR and non-UTF-8 files; the wiring of both
callers. The saving (about 184 s and 174 s per deploy, net) is projected
until the next deploys.

🎯 Fix: 92 %
💥 Break: 2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:31:05 +00:00
kevinveenbirkenbachandClaude Opus 5.5 1205c4a2ae fix(tests): clear the four lint failures store-i18n would push
- test_no_raw_project_walk: the Synapse single-writer test and the
  compose single-writer lint walked role task dirs with rglob; both now
  filter the cached iter_project_files walk (the same 1283 role task
  files), and the Synapse test asserts it found any
- test_project_root_import: copy_batch's traversal guard rejected '..'
  through a banned literal; it now compares posixpath.normpath("/" + name)
  with "/" + name, which under the preceding conditions holds exactly when
  no part is '..', and new cases cover an interior a/../b and a
  nested/../../ escape
- test_used_keys_declared: the resolve test embeds the old task text,
  whose 'INFINITO_DESIGN' is a JS marker, not an environment key; marked
  with a same-line nocheck on the operator's approval

🎯 Fix: 95 %
💥 Break: 1 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:24:20 +00:00
kevinveenbirkenbachandClaude Opus 5.5 6d4e5f7c59 Merge feature/ci/wordpress0-playwright-parallel into feature/store-i18n
- feature/ci/wordpress0-playwright-parallel (de40211272, 88/6, merged on
  the operator's word below the gate): roles that opt in through
  playwright.parallel (WordPress) run independent Playwright suites in
  parallel under a CPU-pressure gate, with dependency deferral, an onion
  cap and Tor lock, crash counting and per-job artifacts; roles without
  the opt-in run as before. The next push's WordPress#0 CI run is its
  live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:11:52 +00:00
kevinveenbirkenbachandClaude Opus 5.5 de40211272 perf(test-e2e-playwright): run independent Playwright suites in parallel under a pressure gate
WordPress#0 runs 18 Playwright suites one after another and exceeds its
CI time budget. A role that opts in through playwright.parallel in its
meta/tests.yml (WordPress: [3, 1, 1] per variant) now hands its suites to
an async scheduler, run_parallel.py, after staging:

- a suite starts only when its dependencies finished here (Keycloak first)
  and the host's CPU pressure, avg10 and avg60, stayed at or below 36 for
  30 s; the first container always starts and running ones are never
  killed
- onion suites share their own cap of 3 and a Tor lock with a
  recreate-pending marker; a probe that times out on the lock re-probes
  the current circuits and, failing that, leaves only that app to the
  serial path
- an app whose dependency was dropped is deferred, and every deferred app
  gets the serial first pass before the existing retry, in round order
- each job gets --oom-score-adj=1000 instead of a memory limit; crashed
  pages, closed browsers and killed workers are counted and named in the
  per-app timing line
- each job's result.json, stdout and stderr land in run-parallel/ and are
  archived with the reports
- the async ceiling is the serial worst case per app and reachable
  network, curl's per-attempt max time included (now
  TEST_E2E_PLAYWRIGHT_ONION_PROBE_MAX_TIME); it stays above the CI step
  timeout and never raises a budget

Roles without the opt-in run exactly as before; the onion probe script
with an empty lock behaves byte for byte like the previous one.

Tests: 126 role unit tests and 30 negative controls cover the gate
window, admission order, onion cap and lock, marker cleanup, the bounded
wait, dependency deferral, crash counting, artifact collection, the
ceiling for the real WordPress#0 app list, and serial equivalence. No CI
run has exercised it yet; the next push is its live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; committed and merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:10:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 882e9c5062 perf(web-app-pgadmin): skip the create_app boot when the administrator is in sync
The realign task booted pgAdmin's whole Flask application on every deploy,
about 97 s on galaxyserver at cpus 0.2, even when nothing had drifted. A
stdlib pre-check now reads pgAdmin's SQLite configuration database
read-only and skips the boot only when exactly one active, unlocked
internal account with the Administrator role carries the declared e-mail
and its stored pbkdf2_sha512 hash recomputes from the declared password the
way Flask-Security 5.8.3 hashes it (NFKD, HMAC-SHA512 with the salt from
the keys table, libpass's round and salt limits). Every other state, an
unreadable or locked database, an unknown hash scheme, or a configuration
database moved out of SQLite by environment, falls through to the
unchanged create_app path.

The administrator password no longer reaches the host's /bin/sh argv
through the task environment: it arrives base64-encoded on the first stdin
line and is passed into the container by name only, so any byte sequence
survives and no fragment can reach stderr.

Tests: the pre-check agrees with converge against passlib; hash, role, key,
lock and duplicate-account drift, an over-long salt and a configuration
database override fall through; the rendered task runs through /bin/sh
with a password holding a newline, quotes, $, a backslash and unicode;
negative controls for each guard. The saving is not measured by a timed
deploy yet.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 14:28:17 +00:00
kevinveenbirkenbachandClaude Opus 5.5 5556dabbb8 Merge perf/plugins/template-shallow-shared into feature/store-i18n
- perf/plugins/template-shallow-shared (f974ab3832, 90/3): template_shallow
  moves from web-opt-rdr-domains to plugins/lookup, and the nine template
  lookups of the design, dashboard, logout and matomo injectors render
  through it instead of deep-copying every task variable per call

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:32:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f974ab3832 perf(plugins): share template_shallow and render the front injectors through it
ansible-core's template lookup deep-copies every task variable on each
call. The role-local template_shallow of web-opt-rdr-domains, which hands
the builtin a top-level copy whose values stay shared, moves to
plugins/lookup so every role can use it. The nine template lookups of the
front injectors switch to it: six in sys-front-inj-design and one each in
sys-front-inj-dashboard, sys-front-inj-logout and sys-front-inj-matomo.
None of the switched templates or the lookups they call writes into the
task variables.

Tests:
- the plugin renders like the builtin, never deep-copies the caller's
  values and leaves the caller's variables unchanged
- for every role with design sources, clearnet and onion vhosts, with and
  without a logo, the design facts are identical to the builtin's, or it
  fails with the identical error
- the dashboard, logout and matomo code renders identically
- the redirect loop of web-opt-rdr-domains still renders through the
  shared plugin

The saving per deploy is not measured by a timed deploy yet.

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:31:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 75693c42e7 Merge two operator-approved sidetrees into feature/store-i18n
- perf/ci/wordpress0-arm64-pass2 (7b412ffa76, 90/4): the Matrix
  ansible-flavor Synapse vhost has one writer, so it no longer changes
  and restarts OpenResty in every swarm pass (WordPress#0)
- fix/sys-dns-cloudflare-records/none-want-group (d96fe45650, 93/3): a
  record set with an entry the converged check cannot judge stays
  pending instead of flipping

🎯 Fix: 90-93 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:55:24 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7b412ffa76 perf(web-app-matrix): write the Synapse vhost once instead of twice per deploy
In the ansible flavor the Synapse vhost had two writers: the shared
front-proxy include rendered proxy.conf.j2 for the Synapse domain, and a
second task rendered flavor/ansible/synapse.conf.j2 to the same file
right after it. The two contents differed, so the file changed on every
deploy and notified restart-openresty, which in swarm is a
service update --force of OpenResty in every pass (WordPress#0, fork run
41926377524).

proxy.conf.j2 now includes the ansible-flavor Synapse template for the
Synapse domain itself, and the second task is gone. The rendered vhost is
identical to the former final content across TLS, Element Call,
clearnet and onion, so the 8448 listener and the Matrix RTC locations
stay. A real change now goes through the front-proxy's reload, which
reaches OpenResty in swarm since 7e5da9422c.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:51:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 59b46235e8 Merge four operator-approved sidetrees into feature/store-i18n
- fix/svc-prx-openresty/reload-mode-force-leak (7e5da9422c, 90/4):
  OpenResty handlers notified from compose-forced roles (matrix, BBB)
  resolve OpenResty's real address in swarm, so their reloads no longer
  skip, and /logout inside their vhosts no longer points at 127.0.0.1
- perf/sys-dns-cloudflare-records/batch-apply: records the zone already
  holds as declared skip the apply loop (59889bb5af, 92/3), and record
  jobs are polled every second while collecting (f51ab74697, 92/3)
- perf/web-opt-rdr-domains/tls-resolve: redirect vhosts render through
  template_shallow without deep-copying the variables (4a6bec9cf0, 92/3)
- perf/sys-front-inj-design/resolve-setfacts: each role design template
  renders once per include (0d04336f93, 92/3)

The three tunes come from the galaxyserver and echoserver 2fd6c6d deploy
logs; all four passed independent skeptic review.

🎯 Fix: 90-92 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:41:16 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f51ab74697 perf(sys-dns-cloudflare-records): poll record jobs every second while collecting
The collect step polled each async record job every 5 s, so a job that
finished right after a poll idled up to 5 s before its result was read.
The interval is now cloudflare_collect_interval (1 s), and the poll cap
derives from it, so the attempt's collect deadline, the retry split and
ignore_errors behave exactly as before.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:21:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4a6bec9cf0 perf(web-opt-rdr-domains): render redirect vhosts without deep-copying the variables
ansible-core 2.20's template lookup deep-copies all task variables on
every call. The redirect loop renders two templates per source domain,
so on galaxyserver this copy dominated the resolve and render step.

A role-local lookup, template_shallow, hands the builtin template lookup
a top-level copy of the variables whose values stay shared with the
caller; nothing on the reachable template and lookup paths mutates them
in place. Rendered output is identical across 12 variants (TLS, Tor,
logout, self-signed, wildcard keep_path), per-item failures and
undefined-variable messages included. A real playbook with 65 items
went from 10.5 s to 5.8 s.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:11:56 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0d04336f93 perf(sys-front-inj-design): render each role design template once per include
The two Resolve tasks spent nearly all their time in the per-call
deep copy of ansible-core 2.20's template lookup, because every set_fact
value re-rendered the lazy task variables: the style include ran its
templates 6 times (10 with logo.css), the script include 6 times. On the
2fd6c6d deploys both together took 125 s on galaxyserver and 174 s on
echoserver.

The composed bodies now reach app.css.j2 and design.js.j2 through
template_vars, app_style_present tests the body directly (app.css.j2
always emits ":root {"), and the design check is inlined. Per include
the style lookups drop to 3 (5 with logo.css) and the script lookups to
4. The facts are byte-identical for 77 roles across clearnet, onion and
logo on or off, and for 36 edge cases, error messages included.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:08:50 +00:00
kevinveenbirkenbachandClaude Opus 5.5 009e4359b8 perf(sys-front-inj-design): skip repeat identical stylesheet writes per run
On a production deploy every vhost of an app re-includes the design and
CDN roles. That repeated two things:

- sys-svc-cdn checked and skipped its two-task run-once block on each of
  the ~50 repeat includes. The block is now one guarded include of
  tasks/00_core.yml, the convention the other roles use, so a repeat
  include skips one task instead of two.
- sys-front-inj-design wrote the same per-app style.css again for every
  vhost of a network that shares it (logout, stalwart, matrix, bluesky)
  and removed an already removed stylesheet once per vhost (the 11
  WordPress vhosts). A per-run record of path and content hash, keyed by
  application and the network's stylesheet file, now skips a write of
  identical content and a repeat removal. Different content still
  writes and the last write wins; each network keeps its own file; the
  record is set only after the write or removal succeeded.

Measured from galaxyserver and echoserver 2fd6c6d: about 24 s saved per
galaxyserver deploy and 23-33 s per echoserver deploy (estimate until the
next deploy re-measures it). Two independent reviews found no
correctness or coverage hole.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 08:40:44 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4596f81e01 test(svc-runner): read the role files through the ROLE_FILE constants
test_file_spot rejected the hand-built vars/meta paths in the new
compose project test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:39:08 +01:00
kevinveenbirkenbach 2fdf419d7a Merge branch 'fix/svc-runner/network-project-preflight' into feature/store-i18n 2026-11-04 21:21:21 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a57e9b944f fix(svc-runner): run the runner stack as its entity's compose project
svc-runner started its stack with 'compose --chdir /opt/github-runner up'
and no --project, so the wrapper named the project after the directory,
github-runner, while every other role uses its entity name. Pass 2 of the
swarm drill in fork run 41744154601 (svc-runner#0) then stopped in the
pre-flight: network 'runner' belongs to compose project 'github-runner'.
The deploy now passes --project from entity_name, and a unit test renders
the real task, runs the real wrapper and checks the pre-flight accepts the
network it creates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:17:13 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c0dd930d29 fix(merge): run the #1212 rehash before compose pull and every compose up
The main merge placed main's rehash handler behind store-i18n's compose pull
and build handlers, so pull ran against stale hashed config names, and the
store-i18n-only lone recreate path (compose-service-up) never rehashed at all.
The handler now sits right after the instance-dir guard, as on main, and
listens on compose-service-up too. The test covers every handler that runs
compose up instead of assuming there is only one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:01:08 +01:00
kevinveenbirkenbach fb004b88a0 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/web-app-mattermost/meta/services.yml
2026-11-04 19:43:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 248c9c5748 refactor(web-app-funkwhale): drop the same_origin prefix from the module sources
test_prefix_grouped_files flagged roles/web-app-funkwhale/files/python/same_origin/, whose three files all started with same_origin_; they were untracked while the lint ran in the objstore worktree. The sources are now settings.py, storage.py and urls.py, volumes.yml mounts them under the unchanged targets /app/same_origin_settings.py, /app/same_origin_storage.py and /app/same_origin_urls.py, so module names and DJANGO_SETTINGS_MODULE stay the same; the mirrored tests and the network_modes.md link follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:28:29 +01:00
kevinveenbirkenbachandClaude Opus 5.5 83923b27ad Merge branch 'fix/objstore/app-proxy-delivery' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:20:42 +01:00
kevinveenbirkenbachandClaude Opus 5.5 80fc8487a8 Merge branch 'fix/web-app-gitea/ldap-source-converge' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:10:26 +01:00
kevinveenbirkenbachandClaude Opus 5.5 6f491381be fix(web-app-gitea): converge duplicate LDAP auth sources and keep the bind password off the host argv
Setup ran add-ldap on every deploy under the name LDAP ({{ DOMAIN_PRIMARY }}); the name column compares case-sensitively, so a casing change created a second source (echoserver had LDAP (Infinito.Nexus) ID 1 with 331 users and LDAP (infinito.nexus) ID 3 with 12), and setup updated only the new one. Cleanup looked the source up with an awk regex in which the parentheses grouped, so it never matched. Setup and cleanup now find LDAP sources by their type (LDAP (via BindDN)), the source is named LDAP, and gitea_ldap_source_plan decides from the per-source user and external-login counts (files/sql/auth-source-users.sql through database_query, only when more than one source exists): keep the source users are linked to, else the lowest id, and delete the others; users on more than one source fail the deploy loudly and delete nothing. The bind password now travels on stdin, is read on the host, forwarded with container exec -e and expanded only by the shell inside the container, with no_log; it left GITEA_LDAP_AUTH_ARGS. On echoserver the 12 users of ID 3 were moved to ID 1 by hand first, so the next deploy keeps ID 1 and deletes ID 3.

Residual: the Gitea CLI takes the password only as --bind-password, so it sits in the argv of the in-container gitea process while add-ldap or update-ldap runs. The IAM line's 5f152d9f28 passes it through task environment: and keeps the name lookup; at the IAM merge take this side for the LDAP tasks and the GITEA_LDAP_* vars.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:10:09 +01:00
kevinveenbirkenbach f40964991a Merge branch 'fix/update/pacman-sig-404' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	roles/update/vars/main.yml
2026-11-04 17:46:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c62b3d2e53 fix(web-app-gitea): apply ROOT_URL on every start so the SSO callback matches Keycloak
On a long-lived instance the role's DOMAIN, ROOT_URL and SSH_DOMAIN variables never reach app.ini: the image's setup script writes them only when app.ini does not exist yet and applies only GITEA__section__KEY variables on later starts. echoserver's Gitea therefore still runs with ROOT_URL https://code.cymais.cloud/, sends that host as the OAuth2 callback, and Keycloak answers 'Invalid parameter: redirect_uri' (its log shows exactly https://code.cymais.cloud/user/oauth2/SSO%20Login%20%28INFINITO.NEXUS%29/callback). env.j2 now sets GITEA__server__DOMAIN, GITEA__server__ROOT_URL and GITEA__server__SSH_DOMAIN. The OAuth2 source is named 'SSO Login', because external_login_user.provider holds 25 characters and the 26-character domain-suffixed name would break every account link; setup and cleanup find the source by its type instead of by name, so the rename happens in place and no second button appears. OpenID 2.0 sign-in and sign-up are switched off. CI missed it because every CI run installs fresh and the Gitea specs never click the SSO button.

The IAM line's 144871ada1 carries the same rename and lookup plus a patched Gitea build for onion sign-in, but keeps the plain ROOT_URL variable; at the IAM merge keep these GITEA__server__ lines together with that patch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:23:36 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ac384eb872 fix(objstore): serve Decidim and Funkwhale object-store files from the app's own origin
Decidim#0 and Funkwhale#0 of fork run 41744154601 failed on the tor rows because both apps handed the browser presigned or public URLs on the object-store host, which publishes no onion vhost. Decidim now sets STORAGE_CDN_HOST to its own origin and switches Active Storage to rails_storage_proxy; an initializer routes direct uploads through the disk-service token route (Rails writes them to S3 server-side), points the redirect and legacy blob routes and the rich-text BlobRenderer at the proxy, safelists storage reads in Rack::Attack, and refuses to boot when the patched activestorage 7.2.2.2 / decidim 0.31.6 signatures change. Funkwhale runs a settings module with a storage whose public names (attachments, sized images, dynamic preferences) resolve to MEDIA_URL and whose exports get a signed one-hour token link; a /media view checks every name against clean_name and rejects backslashes and percent signs, signs per request method and answers with X-Accel-Redirect; the front nginx routes those paths to the API, sends relative protected covers back through it and no longer forwards X-Forwarded-Host, which broke the presigned signature. Both apps reach the store over its internal URL; the public-host CSP entry, extra_hosts entry and CA bundle are gone. The Playwright specs assert the direct upload goes to the app and accept no object-store origin.

Verified by compose deploys of both roles, clearnet update and tor-only reinstall, failed=0 with green Playwright and no object-store host in the rendered pages or APIs; swarm and Decidim variant 1 (disk storage) are not deployed yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 16:17:29 +01:00
kevinveenbirkenbachandClaude Opus 5.5 268b2a46f9 feat(web-app-wordpress): set admin_email, blogname and the FreedomTranslate options through the options map
WORDPRESS_OPTIONS now merges the role's own contributions (admin_email from the platform administrator, blogname from the design title while the design service is on, the freedomtranslate-wp config.options while that addon is on) under services.wordpress.options, so the inventory still wins and wordpress_options_checked guards the merged map. design.php no longer writes blogname, the addon task no longer writes its options, and WORDPRESS_OPTIONS_OWNED_ELSEWHERE drops the three. Every deploy now converges admin_email, and blogname while the design is on; a write WP-CLI reports as unchanged no longer counts as a change, and the FreedomTranslate readback runs after the options write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 12:53:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c3e2114e91 Merge branch 'fix/web-app-wordpress/permalink-options' into feature/store-i18n
Conflicts resolved: the inventory validator keeps store-i18n's custom-role
declaration skip and gains the open-subtree check; the suppression catalogue
keeps both new rows; WORDPRESS_PERMALINK_STRUCTURE gives way to the options
map. WORDPRESS_OPTIONS_OWNED_ELSEWHERE also lists the store-i18n writers the
merge brought along: the design logos (WORDPRESS_DESIGN_LOGOS keys) and the
freedomtranslate options; two custom-role validator tests pass open_paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:35:18 +01:00
kevinveenbirkenbachandClaude Opus 5.5 46e0bd1367 feat(web-app-wordpress): manage WordPress options from services.yml and pin the old permalink structure
Deploys since 197d74416d rewrote the blog's permalink structure to
/%postname%/, so links of the form /blog/YYYY/MM/DD/<slug>/ answer 404
(docs/security/user.md:24 fails the URL check).

services.wordpress.options maps WordPress option names to values; the role
reads each option, writes only the ones that drift and flushes the rewrite
rules after a write. permalink_structure is pinned to
/blog/%year%/%monthnum%/%day%/%postname%/, the value WordPress itself sets
for a subdirectory network. A guard refuses options the role writes
elsewhere, secret-named keys and values WordPress would not read back
unchanged.

The inventory validator accepts any child key under a services.yml key that
carries '# nocheck: inventory-open-subtree <reason>'; services.wordpress.options
carries it, everything else stays strict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:30:31 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c5a69ee951 test(web-app-discourse): build the vars path from ROLE_FILE_VARS_MAIN
test_file_spot flagged the hand-written vars/main.yml path in the chat
integration test (gate pass 16).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:20:58 +01:00
kevinveenbirkenbach 3d7c9da23e Merge branch 'fix/front-inj-design/css-per-network' into feature/store-i18n 2026-11-04 10:53:49 +01:00
kevinveenbirkenbachandClaude Opus 5.5 01e27bfddb fix(sys-front-inj-design): store and link the role stylesheet per network
The design inject renders a role's style.css once per vhost but copied every
render to one shared CDN path, so the last render won. xwiki's stylesheet
embeds its own origin; on a multi-network node the onion render landed last
and every clearnet page loaded onion URLs, failing network-leak.spec.js
whenever the pass picked clearnet.

The first network keeps style.css, every further network gets
<label>-style.css (onion-style.css), keyed by the same rule the tls lookup
uses for the render (serving_network), and each vhost links its own file.
Single-network roles keep the exact old path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 10:46:58 +01:00
kevinveenbirkenbachandClaude Opus 5.5 09e855ae1b fix(web-app-discourse): enable the chat integration and the bridged partners' providers
The chat-integration plugin ships with core and defaults to
chat_integration_enabled: false, while the addon task was empty, so the
spec's first assertion could never pass and topic notifications never left
Discourse for Matrix or Mattermost.

The addon now turns on chat_integration_enabled plus the provider of every
partner the round deploys, idempotently, reads the settings back and fails
when one does not stick. The spec asks the plugin's real admin route
(/admin/plugins/discourse-chat-integration/providers.json; the old path is a
404 on v2026.10.0) and requires an enabled partner provider, since the
catalogue now lists disabled providers too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 10:44:19 +01:00
kevinveenbirkenbachandClaude Opus 5.5 be4c1b2394 fix(update): stop retrying the 404 of pacman's optional database signature
pacman's DatabaseOptional fetches <repo>.db.sig after every database; Arch
publishes none, so every mirror answers 404. The XferCommand ran curl with
--retry-all-errors, which retried that permanent 404 three times, 2 s apart,
for about 18 s per refresh on every Arch host (deploy log 17.4-18.3 s).

pacman-xfer keeps retrying transient failures (transport errors, 5xx, every
4xx but 404/410), honours Retry-After on 429/503 up to 30 s and drops a failed
attempt's partial bytes like curl's own retry did; 404 and 410 fail at once.
The shell-download-retry lint accepts a download its script retries in a
bounded counter loop that leaves on success and sleeps between attempts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 09:06:45 +01:00
kevinveenbirkenbach de98d94fae Merge remote-tracking branch 'origin/main' into fix/sys-svc-compose/config-hash-lag/1212 2026-11-03 22:07:58 +01:00
kevinveenbirkenbachandClaude Opus 5.5 7ac795c0cb test(web-app-gitlab): build the runner-stdin environment with select_autoescape
ruff S701 flagged the jinja2 Environment without autoescape in the single-boot rails test; select_autoescape(default_for_string=False) keeps the rendered Ruby stdin unescaped, the repository's pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:56:06 +01:00
kevinveenbirkenbachandClaude Opus 5.5 81ef94e111 style(dev-python): fold the env filter onto one line after the rename
DEV_PYTHON_REQUIRES is shorter than the old name, so ruff format joins the
dict comprehension in test_install.py onto one line, and the CI format
check would report the drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:47:46 +01:00
kevinveenbirkenbach a70802eb1b Merge branch 'perf/web-app-gitlab/one-rails-boot' into feature/store-i18n 2026-11-03 21:43:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a6d15a5ed4 test(web-app-moodle): mark the tempdir walk of the auth_oidc sync test
test_no_raw_project_walk flagged tree.rglob in files_in; the walk scans the test's own temporary directory, the exception the lint documents, so it carries the project-walk nocheck instead of the cached project helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:43:54 +01:00
kevinveenbirkenbachandClaude Opus 5.5 6e2166d825 perf(web-app-gitlab): converge root, platform admin and appearance in one Rails boot
Three separate 'rails runner -' calls each paid a full Rails boot (47 s, 42 s and 39 s on the echoserver production deploy). platform_admin.yml now runs one runner whose stdin is a small step driver (run_steps.rb) followed by the unchanged root-password, platform-admin and appearance scripts, each wrapped in its own InfinitoSteps.run block: every step keeps its own [CHANGED]/[FAIL] verdict prefixed with its name, exit(0) ends only that step, and the run fails naming every failed step. Rails 7.2.3.1 evals runner stdin at TOPLEVEL_BINDING inside an executor that passes SystemExit through; puts writes to the swapped $stdout while the logger writes to files. The design render and slurp move to tasks/utils/design.yml ahead of it, keeping the numbered includes ascending, and the file layout leaves feature/iam-profile-rbac's hunks on platform_admin.yml and 00_core.yml mergeable without conflicts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:33:37 +01:00
kevinveenbirkenbachandClaude Opus 5.5 f7923b9f94 test(web-app-moodle): build the sync-body environment with select_autoescape
ruff S701 flagged the jinja2 Environment without autoescape in the auth_oidc sync test; select_autoescape(default_for_string=False) keeps the from_string shell body unescaped, as other tests do, and autoformat wrapped one long filecmp call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:31:24 +01:00
kevinveenbirkenbach 089ca505b6 Merge branch 'fix/web-app-moodle/oidc-plugin-missing' into feature/store-i18n 2026-11-03 21:16:59 +01:00
kevinveenbirkenbachandClaude Opus 5.5 9985fb6d08 style(dev-python): apply ruff format to the install test after the DEV_PYTHON_REQUIRES rename
The shorter variable name from 0bdb14cc1d (#1358) lets the dict comprehension fit on one line; make autoformat in gate pass 6 collapsed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:16:47 +01:00
kevinveenbirkenbachandClaude Opus 5.5 279a449566 fix(web-app-moodle): sync the auth_oidc plugin from the image into the code volume
The entrypoint refills the shared code volume only when the release changes, so a volume bootstrapped at 5.3.0 by an image without auth_oidc never received the plugin from a later same-release image; galaxyserver's production deploy failed at the presence check although the image ships it (probe: /opt/moodle-source/public/auth/oidc exists). 04_oidc.yml now asserts the image ships the plugin, naming the image when it does not, and syncs it into the volume as the runtime user through a staging directory, only when diff -rq finds a difference, before the existing presence check. The OIDC upgrade.php run no longer hides a failed plugin install behind failed_when: false, matching the MCP and objectfs plugin tasks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:42:17 +01:00
kevinveenbirkenbach 42315cb065 Merge commit '0bdb14cc1d' (fix/dev-python/python-requires-env-name, #1358) into feature/store-i18n 2026-11-03 20:25:21 +01:00
kevinveenbirkenbach 9642ff13e4 Merge remote-tracking branch 'origin/main' into feature/store-i18n 2026-11-03 20:22:44 +01:00
kevinveenbirkenbachandClaude Opus 5.5 0bdb14cc1d fix(dev-python): name the requires-python hand-over outside the .env key space
#1347 passed the supported Python range to the remote install.sh as
INFINITO_PYTHON_REQUIRES. Every INFINITO_* name is a key of the generated
.env, so test_every_referenced_infinito_key_is_registered fails on main
for this one, which no registry entry can carry: its value is derived
from pyproject.toml's requires-python, the single source, and a default
in default.env would be a second copy of the range. The task environment
and the script now call it DEV_PYTHON_REQUIRES; behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 19:55:27 +01:00