Commit Graph
1460 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 d4388a6a94 Merge branch 'fix/web-app-matrix/mdad-daemon-nofile' into feature/store-i18n
🎯 Fix: 92 %
💥 Break: 6 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 86
🚦 Gate: ⛔ break 6 % > 5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 07:50:30 +00:00
kevinveenbirkenbachandClaude Opus 5.5 e16681ac80 fix(web-app-matrix): bound the open-files limit of every MDAD nested container
containerd inside the matrix-mdad runner runs with LimitNOFILE=infinity, so every nested container without its own --ulimit inherits 2147483584 open files. uwsgi in matrix-synapse-usage-exporter sized its fd table from that and pinned 16 GiB in CI run 42239638356; ccaa075b54 capped that one container. The inner daemon.json now sets default-ulimits nofile 1048576 soft and hard, so any other nested service that sizes a table from RLIMIT_NOFILE stays bounded too. A container's own --ulimit still wins, so the exporter keeps its 65536.

Rendered daemon.json passes dockerd --validate (29.8.2); a string limit is rejected (negative control). The new unit test pins integer Soft/Hard within 65536..1048576 and fails on the base without the key. No live nested-dockerd run: the operator merges sidetrees straight into store-i18n, whose CI proves it; break 6 % accepted by the operator.

🎯 Fix: 92 %
💥 Break: 6 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 86
🚦 Gate: ⛔ break 6 % > 5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 07:49:13 +00:00
kevinveenbirkenbachandClaude Opus 5.5 86998924d9 fix(ci): name the gh failure when the retrigger finds no inputs
The retrigger guard aborted with 'its job log records no inputs' while _gh(check=False) dropped gh's stderr, so a transient log read failure and a moved Inputs block looked the same. _gh now takes an errors list instead of check: given, a failure appends gh's last stderr line with URL query strings cut (job logs redirect to signed blob URLs) and yields an empty string; absent, it aborts as before. inputs_from_jobs passes such a list through, and dispatched_priority appends how many log reads failed and the last error, or that none failed.

MEDUSA findings accepted as false positives (unchanged lines, no SQL or untrusted argv; independent skeptic review): MEDUSA-FTSCAN-231 runs.py:80 and runs.py:318, B603 gh.py:27 and gh.py:79, B105 runs.py:36.

🎯 Fix: 95 %
💥 Break: 2 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 93
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:12:50 +00:00
kevinveenbirkenbachandClaude Opus 5.5 ccaa075b54 fix(web-app-matrix): cap the usage exporter's open-files limit
MDAD's synapse-usage-exporter runs uwsgi with an HTTP router that writes an 8 byte slot per possible file descriptor, sized from RLIMIT_NOFILE. Inside the matrix-mdad runner the nested daemon hands its containers the kernel maximum of 2147483584, so the exporter pinned about 16 GiB of RAM and swap and the swarm watchdog aborted WordPress#0 and Nextcloud#7 in run 42239638356. The addon now declares config.nofile 65536 and the role passes it to MDAD as --ulimit=nofile=N:N for that container; a local compose deploy shows the exporter at nofile 65536 and 38 MiB.

🎯 Fix: 97 %
💥 Break: 2 %
💸 Cost: 1 %
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 100
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 21:00:16 +00:00
kevinveenbirkenbachandClaude Opus 5.5 2284d90078 refactor(sys-token-store): include the hand-over so the write block stays at three tasks
4c682fbdc1 grew the when-gated write block of write_many.yml to four children, which test_no_oversized_ansible_block and test_block_size reject, and its integration test read the plays' views uncached. The hand-over and the writer's own merge now live in tasks/utils/hand_over.yml, included from the block with no_log applied once; the order and the lockstep behaviour stay the same (re-proven on ansible-core 2.20.2 and 2.21.5). The integration test reads through utils.cache.files, and a unit test pins the masking of every task that carries the merged store.

🎯 Fix: 95 %
💥 Break: 2 %
💸 Cost: 104 %
⏱️ Time: 107 %
🛡️ Security: 100 %
🏆 Score: 84
🚦 Gate: ⚠️ cost 104 % > 100, time 107 % > 100: ask the operator before the commit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 13:29:30 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4c682fbdc1 fix(sys-token-store): hand a written store to every reader on its stack host
Swarm workers loaded sys_token_store_users before the manager wrote new tokens in pass 1, so they rendered msmtprc with an empty password (CI run 42128590251, WordPress#0). write_many.yml now notes once, before the block, whether the merge changed the store; delegates the merged store to every play host that shares the writer's STACK_HOST; and re-sets the writer's own merge afterwards, so a peer writer's hand-over cannot overwrite it. Verified with lockstep plays on ansible-core 2.20 and 2.21.

🎯 Fix: 93 %
💥 Break: 4 %
💸 Cost: 108 %
⏱️ Time: 116 %
🛡️ Security: 100 %
🏆 Score: 71
🚦 Gate: ⚠️ cost 108 % > 100, time 116 % > 100: ask the operator before the commit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 11:26:26 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0aac803507 perf(web-app-discourse): read the addon state back in the apply runner
Chat integration and Mermaid booted Rails twice each, once to apply and once to read back (10.9 s and 11.1 s for the read-backs on echoserver 94767d4). Each verify script now runs after its apply script in the same rails r and starts with SiteSetting.refresh!, so it reads what the database holds, as 2b11a7e9fe does for the site settings. The test stub keeps a dropped write in memory, so the read-back test fails without the refresh.

🎯 Fix: 90 %
💥 Break: 4 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 20:10:49 +00:00
kevinveenbirkenbachandClaude Opus 5.5 b5e299da1c test(install): fold the assertRaises into the patch context (SIM117)
🎯 Fix: 99 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 98
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:54:52 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7e48e7fe2c fix(install): reinstall node_modules when a declared devDependency is missing
install-lint only checked that @playwright/test existed, so node_modules installed before 1c5e5a4fe1 never received @axe-core/playwright and lint-playwright failed with Cannot find module. The guard now compares package.json's devDependencies with node_modules and runs npm ci when one is absent.

🎯 Fix: 93 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:47:07 +00:00
kevinveenbirkenbachandClaude Opus 5.5 c23f0f0d29 Merge five sidetrees into feature/store-i18n
- fix/web-app-docs/codeql-xss (6e045bced8, 92/3): redirect header splitting, DOM-text navigation
- fix/cli/codeql-file-modes (20fae00bdf, 95-97/1-3): owner-only lock and marker files, explicit legend strings, mirror shim import
- fix/roles/codeql-js-hygiene (787f8d9a6d, 92-97/2-3): crypto wrong password, log sanitizer shape, fillStyle save/restore
- fix/svc-ai-agent-broker/codeql-ssrf (eeb9b22ae4, 92/4): platform and relay path from server-side lists, named error paths
- fix/test-e2e-playwright/concurrency-test-load (eb6940a9ad, 95/1): concurrency proven from the trace (gate pass 2 failure)

CodeQL PR #625 alerts #1079 and #957 stay false positives for the operator to dismiss.

🎯 Fix: 92-99 %
💥 Break: 1-4 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 88-98
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:24:11 +00:00
kevinveenbirkenbachandClaude Opus 5.5 eb6940a9ad test(test-e2e-playwright): prove scheduler concurrency from the trace, not the wall clock
Gate pass 2 on store-i18n 9154a03fed failed test_jobs_run_concurrently_within_the_cap_and_keep_their_own_results at 2.7046 s against a 2.7 s bound: five interpreter starts under full host load. The peak of 2 read from the job trace already proves the overlap and the cap independent of load, and test_a_freed_slot_is_refilled_without_waiting_a_poll covers prompt refills, so the wall-clock bound goes.

🎯 Fix: 95 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:23:36 +00:00
kevinveenbirkenbachandClaude Opus 5.5 eeb9b22ae4 fix(svc-ai-agent-broker): select platform and relay path from server-side lists
py/partial-ssrf on PR #625 (agents.py:195, :250, server.py:123): the chat body's model now selects its entry from AGENT_PLATFORMS through agents.pick, so the agent host is built from the server-side key; a list or dict model answers 404 instead of a TypeError. The relay forwards only /v1/chat/completions and /v1/models with the broker's key; any other path answers 404 and logs relay_refused (the two paths are the ones the relay logs show in use).

py/empty-except (agents.py:201, :258): the health wait names its last error in the timeout, and the model-list fallback logs agent_model_fallback through the log function the server passes in.

Tests cover pick, six refused relay paths and both error paths; each fails against the old code.

🎯 Fix: 92 %
💥 Break: 4 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 88
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:18:22 +00:00
kevinveenbirkenbachandClaude Opus 5.5 dcfb220dca test(svc-ai-agent-broker): let the connect fake run Engine.__init__
py/missing-call-to-init on PR #625 (test_engine.py:110): _ConnectEngine never ran the base initializer; it now does, and a new test checks the base state.

🎯 Fix: 99 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 98
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:17:39 +00:00
kevinveenbirkenbachandClaude Opus 5.5 20fae00bdf refactor(cli): import ImageRef from its module instead of the mirror shim
py/unused-import on PR #625 (mirror/model.py:10): the module only re-imported ImageRef from utils.docker.image.discovery. providers.py and the four mirror tests import it from there, as wait/__main__.py already did; no other importer exists.

🎯 Fix: 97 %
💥 Break: 2 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 95
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:07:20 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0914910df5 fix(cli): make the architecture legend's split strings explicit
py/implicit-string-concatenation-in-list on PR #625 (architecture.py:147, :151): one table row and one paragraph were split over lines without parentheses; both are now parenthesized, likewise the f-string row ruff ISC004 flags. The new test pins four two-cell rows and one paragraph and fails with a comma inserted.

🎯 Fix: 97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:04:48 +00:00
kevinveenbirkenbachandClaude Opus 5.5 ae9907df81 fix(test-e2e-playwright): create the tor reader lock owner-only
py/overly-permissive-file on PR #625 (run_parallel.py:287): tor.lock was created 0o644; the scheduler and wait_onion_reachable.sh both run as root under become. The new test sets umask 0 and fails at 0o644.

🎯 Fix: 95 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 92
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:03:10 +00:00
kevinveenbirkenbachandClaude Opus 5.5 8c199a915d fix(cli): create role markers and the placement lock owner-only
py/overly-permissive-file on PR #625 (marker.py:60, placement.py:182): both files were created 0o644; only the owning user's pull, clean and list read them. The new tests set umask 0 and fail at 0o644.

🎯 Fix: 95 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 92
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:02:47 +00:00
kevinveenbirkenbachandClaude Opus 5.5 6e045bced8 fix(web-app-docs): encode line breaks in redirects, navigate by built URLs
- py/http-response-splitting (server.py:87): _redirect percent-encodes CR
  and LF in the Location header
- js/xss-through-dom (versions.js:82, languages.js:25): both switchers
  navigate to the URL they built instead of select.value

The new cases in test_server.py, versions.test.js and languages.test.js
fail without the fix and pass with it.

🎯 Fix: 92 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:54:27 +00:00
kevinveenbirkenbachandClaude Opus 5.5 2fc48cadbb Merge two operator-approved sidetrees into feature/store-i18n
- fix/sys-dns-cloudflare-records/api-base-url (d301a4d3c3, 95/1): test-external red in run 42056917806, the bare Cloudflare API base answered 404
- fix/web-app-pgadmin/test-cached-reads (0af21e62ad, 97/1): test-lint red in run 42056917806, uncached and direct YAML reads in the pgAdmin test

🎯 Fix: 95-97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 94-96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:25:07 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0af21e62ad test(web-app-pgadmin): read project files through the cached readers
test_no_uncached_file_reads and test_no_direct_yaml_calls failed test-lint in run 42056917806. env.j2, manager_ops.yml and the realign script now come from utils.cache; the before/after read of the tempdir SQLite file keeps a direct read with the cache-read opt-out, since a cached read would hide a write.

🎯 Fix: 97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:20:42 +00:00
kevinveenbirkenbachandClaude Opus 5.5 d301a4d3c3 fix(sys-dns-cloudflare-records): root the API constant at /zones
The bare https://api.cloudflare.com/client/v4 answers HTTP 404, so test_urls_reachable failed test-external in run 42056917806. Every request of the plugin goes below /zones; the constant now names that path, which answers 403 (alive for the URL test), and the requested URLs stay byte-identical.

🎯 Fix: 95 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:18:01 +00:00
kevinveenbirkenbachandClaude Opus 5.5 735308266e Merge two operator-approved sidetrees into feature/store-i18n
- fix/tests/store-i18n-lint-red (1205c4a2ae, 95/1): clears the raw-walk,
  project-root and env-key lint failures store-i18n would have pushed
- fix/sys-stk-front-proxy/body-filter-fold (4984f506c7, 92/2): the
  body_filter Lua is written in the vhost's single stack_host_contents
  call (about 184 s echoserver and 174 s galaxyserver per deploy,
  projected), and stack_host_contents compares bytes

🎯 Fix: 92-95 %
💥 Break: 1-2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90-94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:37:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4984f506c7 perf(sys-stk-front-proxy): write the body_filter Lua in the vhost's single contents call
Every front-proxy vhost wrote its body_filter Lua through a separate
stack_host_template task in sys-front-inj-all: two remote round trips when
unchanged (stat + file) and four to five when changed, 191 s over 54 vhosts
per echoserver deploy and 179 s over 48 per galaxyserver deploy. The Lua
now joins http.conf and the vhost in the Deploy task's stack_host_contents
call, written first as before, rendered through template_shallow with
application_id and domain pinned to the vhost.

- sys-front-inj-all/tasks/main.yml no longer writes the Lua; the new
  standalone.yml includes it and keeps the separate write for matomo's
  re-inject path, which has no vhost Deploy task
- stack_host_contents compares bytes instead of a universal-newline text
  read, so content with a CR stops reporting changed on every run and an
  existing non-UTF-8 file is replaced instead of crashing the module
- a new required diff_paths argument keeps --diff for the Lua only; vhost
  and http.conf content stay out of the diff

A Lua-only change still reports changed and fires reload-openresty once;
the served probe and the fallback redeploy keep their old triggers. Check
mode is unchanged.

Tests: the real stack_host_template action against the fold expression
across 99 roles and TLS/Tor/injection scenarios, byte-identical, with
trim and newline negative controls; owner, group and mode parity with the
copy module; byte compare with CR and non-UTF-8 files; the wiring of both
callers. The saving (about 184 s and 174 s per deploy, net) is projected
until the next deploys.

🎯 Fix: 92 %
💥 Break: 2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:31:05 +00:00
kevinveenbirkenbachandClaude Opus 5.5 1205c4a2ae fix(tests): clear the four lint failures store-i18n would push
- test_no_raw_project_walk: the Synapse single-writer test and the
  compose single-writer lint walked role task dirs with rglob; both now
  filter the cached iter_project_files walk (the same 1283 role task
  files), and the Synapse test asserts it found any
- test_project_root_import: copy_batch's traversal guard rejected '..'
  through a banned literal; it now compares posixpath.normpath("/" + name)
  with "/" + name, which under the preceding conditions holds exactly when
  no part is '..', and new cases cover an interior a/../b and a
  nested/../../ escape
- test_used_keys_declared: the resolve test embeds the old task text,
  whose 'INFINITO_DESIGN' is a JS marker, not an environment key; marked
  with a same-line nocheck on the operator's approval

🎯 Fix: 95 %
💥 Break: 1 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:24:20 +00:00
kevinveenbirkenbachandClaude Opus 5.5 6d4e5f7c59 Merge feature/ci/wordpress0-playwright-parallel into feature/store-i18n
- feature/ci/wordpress0-playwright-parallel (de40211272, 88/6, merged on
  the operator's word below the gate): roles that opt in through
  playwright.parallel (WordPress) run independent Playwright suites in
  parallel under a CPU-pressure gate, with dependency deferral, an onion
  cap and Tor lock, crash counting and per-job artifacts; roles without
  the opt-in run as before. The next push's WordPress#0 CI run is its
  live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:11:52 +00:00
kevinveenbirkenbachandClaude Opus 5.5 de40211272 perf(test-e2e-playwright): run independent Playwright suites in parallel under a pressure gate
WordPress#0 runs 18 Playwright suites one after another and exceeds its
CI time budget. A role that opts in through playwright.parallel in its
meta/tests.yml (WordPress: [3, 1, 1] per variant) now hands its suites to
an async scheduler, run_parallel.py, after staging:

- a suite starts only when its dependencies finished here (Keycloak first)
  and the host's CPU pressure, avg10 and avg60, stayed at or below 36 for
  30 s; the first container always starts and running ones are never
  killed
- onion suites share their own cap of 3 and a Tor lock with a
  recreate-pending marker; a probe that times out on the lock re-probes
  the current circuits and, failing that, leaves only that app to the
  serial path
- an app whose dependency was dropped is deferred, and every deferred app
  gets the serial first pass before the existing retry, in round order
- each job gets --oom-score-adj=1000 instead of a memory limit; crashed
  pages, closed browsers and killed workers are counted and named in the
  per-app timing line
- each job's result.json, stdout and stderr land in run-parallel/ and are
  archived with the reports
- the async ceiling is the serial worst case per app and reachable
  network, curl's per-attempt max time included (now
  TEST_E2E_PLAYWRIGHT_ONION_PROBE_MAX_TIME); it stays above the CI step
  timeout and never raises a budget

Roles without the opt-in run exactly as before; the onion probe script
with an empty lock behaves byte for byte like the previous one.

Tests: 126 role unit tests and 30 negative controls cover the gate
window, admission order, onion cap and lock, marker cleanup, the bounded
wait, dependency deferral, crash counting, artifact collection, the
ceiling for the real WordPress#0 app list, and serial equivalence. No CI
run has exercised it yet; the next push is its live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; committed and merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:10:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 882e9c5062 perf(web-app-pgadmin): skip the create_app boot when the administrator is in sync
The realign task booted pgAdmin's whole Flask application on every deploy,
about 97 s on galaxyserver at cpus 0.2, even when nothing had drifted. A
stdlib pre-check now reads pgAdmin's SQLite configuration database
read-only and skips the boot only when exactly one active, unlocked
internal account with the Administrator role carries the declared e-mail
and its stored pbkdf2_sha512 hash recomputes from the declared password the
way Flask-Security 5.8.3 hashes it (NFKD, HMAC-SHA512 with the salt from
the keys table, libpass's round and salt limits). Every other state, an
unreadable or locked database, an unknown hash scheme, or a configuration
database moved out of SQLite by environment, falls through to the
unchanged create_app path.

The administrator password no longer reaches the host's /bin/sh argv
through the task environment: it arrives base64-encoded on the first stdin
line and is passed into the container by name only, so any byte sequence
survives and no fragment can reach stderr.

Tests: the pre-check agrees with converge against passlib; hash, role, key,
lock and duplicate-account drift, an over-long salt and a configuration
database override fall through; the rendered task runs through /bin/sh
with a password holding a newline, quotes, $, a backslash and unicode;
negative controls for each guard. The saving is not measured by a timed
deploy yet.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 14:28:17 +00:00
kevinveenbirkenbachandClaude Opus 5.5 5556dabbb8 Merge perf/plugins/template-shallow-shared into feature/store-i18n
- perf/plugins/template-shallow-shared (f974ab3832, 90/3): template_shallow
  moves from web-opt-rdr-domains to plugins/lookup, and the nine template
  lookups of the design, dashboard, logout and matomo injectors render
  through it instead of deep-copying every task variable per call

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:32:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f974ab3832 perf(plugins): share template_shallow and render the front injectors through it
ansible-core's template lookup deep-copies every task variable on each
call. The role-local template_shallow of web-opt-rdr-domains, which hands
the builtin a top-level copy whose values stay shared, moves to
plugins/lookup so every role can use it. The nine template lookups of the
front injectors switch to it: six in sys-front-inj-design and one each in
sys-front-inj-dashboard, sys-front-inj-logout and sys-front-inj-matomo.
None of the switched templates or the lookups they call writes into the
task variables.

Tests:
- the plugin renders like the builtin, never deep-copies the caller's
  values and leaves the caller's variables unchanged
- for every role with design sources, clearnet and onion vhosts, with and
  without a logo, the design facts are identical to the builtin's, or it
  fails with the identical error
- the dashboard, logout and matomo code renders identically
- the redirect loop of web-opt-rdr-domains still renders through the
  shared plugin

The saving per deploy is not measured by a timed deploy yet.

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:31:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 75693c42e7 Merge two operator-approved sidetrees into feature/store-i18n
- perf/ci/wordpress0-arm64-pass2 (7b412ffa76, 90/4): the Matrix
  ansible-flavor Synapse vhost has one writer, so it no longer changes
  and restarts OpenResty in every swarm pass (WordPress#0)
- fix/sys-dns-cloudflare-records/none-want-group (d96fe45650, 93/3): a
  record set with an entry the converged check cannot judge stays
  pending instead of flipping

🎯 Fix: 90-93 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:55:24 +00:00
kevinveenbirkenbachandClaude Opus 5.5 d96fe45650 fix(sys-dns-cloudflare-records): keep a record set pending when one entry cannot be judged
cloudflare_records_converged skipped an entry its expected() rejects (a
TXT value given as an int, a ttl of null) when building the per-owner
declarations, yet cloudflare_dns still writes for such an entry. In
[TXT t 12345, TXT t "12345" ttl 300] the second entry was judged
converged and skipped while the first was applied with ttl 1, so the
record flipped on every run. A record set declared more than once now
stays pending as a whole when any of its entries cannot be judged, like
the hazardous solo and absent entries already do. No producer in the
repository builds such a list today.

🎯 Fix: 93 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:54:49 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7b412ffa76 perf(web-app-matrix): write the Synapse vhost once instead of twice per deploy
In the ansible flavor the Synapse vhost had two writers: the shared
front-proxy include rendered proxy.conf.j2 for the Synapse domain, and a
second task rendered flavor/ansible/synapse.conf.j2 to the same file
right after it. The two contents differed, so the file changed on every
deploy and notified restart-openresty, which in swarm is a
service update --force of OpenResty in every pass (WordPress#0, fork run
41926377524).

proxy.conf.j2 now includes the ansible-flavor Synapse template for the
Synapse domain itself, and the second task is gone. The rendered vhost is
identical to the former final content across TLS, Element Call,
clearnet and onion, so the 8448 listener and the Matrix RTC locations
stay. A real change now goes through the front-proxy's reload, which
reaches OpenResty in swarm since 7e5da9422c.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:51:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 59b46235e8 Merge four operator-approved sidetrees into feature/store-i18n
- fix/svc-prx-openresty/reload-mode-force-leak (7e5da9422c, 90/4):
  OpenResty handlers notified from compose-forced roles (matrix, BBB)
  resolve OpenResty's real address in swarm, so their reloads no longer
  skip, and /logout inside their vhosts no longer points at 127.0.0.1
- perf/sys-dns-cloudflare-records/batch-apply: records the zone already
  holds as declared skip the apply loop (59889bb5af, 92/3), and record
  jobs are polled every second while collecting (f51ab74697, 92/3)
- perf/web-opt-rdr-domains/tls-resolve: redirect vhosts render through
  template_shallow without deep-copying the variables (4a6bec9cf0, 92/3)
- perf/sys-front-inj-design/resolve-setfacts: each role design template
  renders once per include (0d04336f93, 92/3)

The three tunes come from the galaxyserver and echoserver 2fd6c6d deploy
logs; all four passed independent skeptic review.

🎯 Fix: 90-92 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:41:16 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7e5da9422c fix(svc-prx-openresty): resolve OpenResty's own address in handlers notified from forced roles
web-app-matrix (ansible flavor) and web-app-bigbluebutton set
compose_mode_force: compose in their role vars, and those vars stay in
scope when their front-proxy includes notify the OpenResty handlers. In
swarm the handlers' container_address lookup then returned the bare
compose name, the validate retried for about 35 s against a container
that does not exist and ended ok, and the reload was skipped: 3 of 52
reloads in the WordPress#0 job of fork run 41926377524 (both matrix
.onion vhosts), so changed vhosts stayed unserved.

The validate, reload and restart handler tasks now clear
compose_mode_force at task level (task vars outrank the inherited role
vars), as the engine, rdbms and objstore entry points already do, and so
do the svc-db-openldap handlers, which the new lint also covers. The
proxy_pass lookup applies a caller's override only when the term names
the scope's own application_id, because the /logout location rendered
inside matrix and BBB vhosts pointed at 127.0.0.1 in swarm.

A real ansible-playbook integration test proves both modes and fails
without the fix; the new lint keeps every handler lookup of a fixed role
clearing the override.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:40:41 +00:00
kevinveenbirkenbachandClaude Opus 5.5 5d5ba66046 Merge perf/stack-host-copy/changed-tree-bulk into feature/store-i18n
stack_host_copy writes a partly changed tree in one transfer instead of
one copy action per file (a643871997): the echoserver 2fd6c6d deploy paid
437 s for six changed design_assets copies, about 6 s per file. Shapes
the builtin copy treats differently make the new copy_batch module refuse
before it writes, and the regular copy runs instead. Operator-approved.
The independent review compared end state, changed, check and diff mode
against the builtin copy in a real playbook run and found them equal; a
write failing partway reports changed=true, where copy reports false.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 90 % | 2 % | 100 % | 15 % (estimated from round trips) | ~100 % | 100 | ⏳ score passes, gate.py not run |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:24:24 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f51ab74697 perf(sys-dns-cloudflare-records): poll record jobs every second while collecting
The collect step polled each async record job every 5 s, so a job that
finished right after a poll idled up to 5 s before its result was read.
The interval is now cloudflare_collect_interval (1 s), and the poll cap
derives from it, so the attempt's collect deadline, the retry split and
ignore_errors behave exactly as before.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:21:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 59889bb5af perf(sys-dns-cloudflare-records): skip records the zone already holds as declared
Every deploy sent each declared record through cloudflare_dns, one async
job per record, although nearly all of them already stood unchanged; the
galaxyserver 2fd6c6d deploy spent about two minutes collecting them.

A new action plugin, cloudflare_records_converged, reads each zone once
(all pages) and splits the declared records into pending and converged.
It judges A, AAAA, CNAME, MX, TXT and SRV with the same lookup key and
the same defaults the community.general 14.0.0 module applies; every
other type, every solo or absent entry, and every record set whose
entries would undo each other stay pending, as do all records of a zone
that cannot be read (with a warning). Only pending records reach the
unchanged apply and retry loop, so failures stay per record and loud.

The test pins the plugin's defaults against the module's argument_spec
and the reviewed collection version, and covers the hazardous groups
with a mutation check.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:18:57 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4a6bec9cf0 perf(web-opt-rdr-domains): render redirect vhosts without deep-copying the variables
ansible-core 2.20's template lookup deep-copies all task variables on
every call. The redirect loop renders two templates per source domain,
so on galaxyserver this copy dominated the resolve and render step.

A role-local lookup, template_shallow, hands the builtin template lookup
a top-level copy of the variables whose values stay shared with the
caller; nothing on the reachable template and lookup paths mutates them
in place. Rendered output is identical across 12 variants (TLS, Tor,
logout, self-signed, wildcard keep_path), per-item failures and
undefined-variable messages included. A real playbook with 65 items
went from 10.5 s to 5.8 s.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:11:56 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0d04336f93 perf(sys-front-inj-design): render each role design template once per include
The two Resolve tasks spent nearly all their time in the per-call
deep copy of ansible-core 2.20's template lookup, because every set_fact
value re-rendered the lazy task variables: the style include ran its
templates 6 times (10 with logo.css), the script include 6 times. On the
2fd6c6d deploys both together took 125 s on galaxyserver and 174 s on
echoserver.

The composed bodies now reach app.css.j2 and design.js.j2 through
template_vars, app_style_present tests the body directly (app.css.j2
always emits ":root {"), and the design check is inlined. Per include
the style lookups drop to 3 (5 with logo.css) and the script lookups to
4. The facts are byte-identical for 77 roles across clearnet, onion and
logo on or off, and for 36 edge cases, error messages included.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:08:50 +00:00
kevinveenbirkenbachandClaude Opus 5.5 a643871997 perf(stack-host-copy): write a changed tree in one transfer
When the tree check finds differing entries, pack exactly the differing files into one tar archive in source walk order, transfer it once into the remote tmpdir and let the new copy_batch module write each file through atomic_move with the copy module's directory, mode and unsafe_writes semantics. Mode-only drift and leaf directories settle in the same call; unchanged files are not sent.

Every shape the regular copy treats differently makes the module refuse before it writes: a symlink, directory or other non-regular entry at a destination, a missing path below a non-directory or a symlink loop, and two source paths reaching one remote entry. The regular copy then runs. Check mode, diff mode, unprivileged become, a dest holding $ and an archive above 16 MiB keep the regular copy. A write failure fails the task and names the file.

Executed with ansible-playbook -c local, pipelining on and off: the trees this action leaves equal the builtin copy's, including the symlink fallback; a 6-file changed tree took 2 module calls and 1 transfer instead of 14 and 4. Unit: a flat 15-file tree with every file changed drops from 31 module calls and 15 transfers to 2 and 1. Two dialectic rounds with three independent skeptics; every hole they found is fixed and tested. A third independent review compared end state, changed, check and diff mode against the builtin copy in a real playbook run and found them equal; one deliberate difference remains: a write failing partway reports changed=true with the files already written, where the builtin copy reports changed=false.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 90 % | 2 % | 100 % | 15 % (estimated from round trips) | ~100 % | 100 | ⏳ score passes, gate.py not run |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:06:49 +00:00
kevinveenbirkenbachandClaude Opus 5.5 009e4359b8 perf(sys-front-inj-design): skip repeat identical stylesheet writes per run
On a production deploy every vhost of an app re-includes the design and
CDN roles. That repeated two things:

- sys-svc-cdn checked and skipped its two-task run-once block on each of
  the ~50 repeat includes. The block is now one guarded include of
  tasks/00_core.yml, the convention the other roles use, so a repeat
  include skips one task instead of two.
- sys-front-inj-design wrote the same per-app style.css again for every
  vhost of a network that shares it (logout, stalwart, matrix, bluesky)
  and removed an already removed stylesheet once per vhost (the 11
  WordPress vhosts). A per-run record of path and content hash, keyed by
  application and the network's stylesheet file, now skips a write of
  identical content and a repeat removal. Different content still
  writes and the last write wins; each network keeps its own file; the
  record is set only after the write or removal succeeded.

Measured from galaxyserver and echoserver 2fd6c6d: about 24 s saved per
galaxyserver deploy and 23-33 s per echoserver deploy (estimate until the
next deploy re-measures it). Two independent reviews found no
correctness or coverage hole.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 08:40:44 +00:00
kevinveenbirkenbach 025ebe9754 Merge remote-tracking branch 'origin/main' into feature/store-i18n 2026-11-04 23:28:17 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a63cbcb1f7 fix(apparmor): leave a read-only /proc/sys alone in teardown and restore
Fork run 41911075483 (feature/store-i18n 2fd6c6d310) lost both workspace
jobs, debian:13 and ubuntu:26.04, in make security-apparmor-teardown:

  common.sh: line 75: /proc/sys/kernel/apparmor_restrict_unprivileged_unconfined: Read-only file system

Inside the workspace container /proc/sys is mounted read-only, and the
write that #1371 added ran under set -e. A sysctl that cannot be written
now leaves the restriction as it is with a log line instead of failing
the teardown, and only a successful write records the previous value.
Restore keeps its record when it cannot write the sysctl back.

The tests call the helpers under set -euo pipefail like teardown.sh and
use /proc/sys/kernel/ostype as a sysctl nobody can write; both new tests
fail against the previous common.sh.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 95 % | 3 % | – | – | ~100 % | 92 | ✅ 90/5, efficiency and speed <= 100, security 100 % |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 23:25:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 4596f81e01 test(svc-runner): read the role files through the ROLE_FILE constants
test_file_spot rejected the hand-built vars/meta paths in the new
compose project test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:39:08 +01:00
kevinveenbirkenbach 2fdf419d7a Merge branch 'fix/svc-runner/network-project-preflight' into feature/store-i18n 2026-11-04 21:21:21 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a57e9b944f fix(svc-runner): run the runner stack as its entity's compose project
svc-runner started its stack with 'compose --chdir /opt/github-runner up'
and no --project, so the wrapper named the project after the directory,
github-runner, while every other role uses its entity name. Pass 2 of the
swarm drill in fork run 41744154601 (svc-runner#0) then stopped in the
pre-flight: network 'runner' belongs to compose project 'github-runner'.
The deploy now passes --project from entity_name, and a unit test renders
the real task, runs the real wrapper and checks the pre-flight accepts the
network it creates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:17:13 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c0dd930d29 fix(merge): run the #1212 rehash before compose pull and every compose up
The main merge placed main's rehash handler behind store-i18n's compose pull
and build handlers, so pull ran against stale hashed config names, and the
store-i18n-only lone recreate path (compose-service-up) never rehashed at all.
The handler now sits right after the instance-dir guard, as on main, and
listens on compose-service-up too. The test covers every handler that runs
compose up instead of assuming there is only one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:01:08 +01:00
kevinveenbirkenbach fb004b88a0 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/web-app-mattermost/meta/services.yml
2026-11-04 19:43:38 +01:00
kevinveenbirkenbach 70f3716fbe Merge pull request #1370 from infinito-nexus/fix/update/follow-warn-on-unreadable-digest
fix(update): say when follow cannot read Docker Hub, and retry a 429
2026-11-04 19:16:52 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c068e44d1b fix(update): treat an incomplete Docker Hub response as unreadable
followed_tags caught OSError, ValueError, KeyError and RuntimeError, but
http.client.IncompleteRead, BadStatusLine and LineTooLong derive from
http.client.HTTPException only. One truncated digest response therefore
escaped to collect_updates (docker.py:344, no try) and aborted the whole
Docker update run instead of printing the follow warning for that image.
Every other fetch site of the updater already catches HTTPException
(docker.py, pip.py).

Reported by CodeRabbit on #1370.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:16:16 +01:00