Commit Graph
1445 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 0914910df5 fix(cli): make the architecture legend's split strings explicit
py/implicit-string-concatenation-in-list on PR #625 (architecture.py:147, :151): one table row and one paragraph were split over lines without parentheses; both are now parenthesized, likewise the f-string row ruff ISC004 flags. The new test pins four two-cell rows and one paragraph and fails with a comma inserted.

🎯 Fix: 97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:04:48 +00:00
kevinveenbirkenbachandClaude Opus 5.5 ae9907df81 fix(test-e2e-playwright): create the tor reader lock owner-only
py/overly-permissive-file on PR #625 (run_parallel.py:287): tor.lock was created 0o644; the scheduler and wait_onion_reachable.sh both run as root under become. The new test sets umask 0 and fails at 0o644.

🎯 Fix: 95 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 92
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:03:10 +00:00
kevinveenbirkenbachandClaude Opus 5.5 8c199a915d fix(cli): create role markers and the placement lock owner-only
py/overly-permissive-file on PR #625 (marker.py:60, placement.py:182): both files were created 0o644; only the owning user's pull, clean and list read them. The new tests set umask 0 and fail at 0o644.

🎯 Fix: 95 %
💥 Break: 3 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 92
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 17:02:47 +00:00
kevinveenbirkenbachandClaude Opus 5.5 2fc48cadbb Merge two operator-approved sidetrees into feature/store-i18n
- fix/sys-dns-cloudflare-records/api-base-url (d301a4d3c3, 95/1): test-external red in run 42056917806, the bare Cloudflare API base answered 404
- fix/web-app-pgadmin/test-cached-reads (0af21e62ad, 97/1): test-lint red in run 42056917806, uncached and direct YAML reads in the pgAdmin test

🎯 Fix: 95-97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 94-96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:25:07 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0af21e62ad test(web-app-pgadmin): read project files through the cached readers
test_no_uncached_file_reads and test_no_direct_yaml_calls failed test-lint in run 42056917806. env.j2, manager_ops.yml and the realign script now come from utils.cache; the before/after read of the tempdir SQLite file keeps a direct read with the cache-read opt-out, since a cached read would hide a write.

🎯 Fix: 97 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 96
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:20:42 +00:00
kevinveenbirkenbachandClaude Opus 5.5 d301a4d3c3 fix(sys-dns-cloudflare-records): root the API constant at /zones
The bare https://api.cloudflare.com/client/v4 answers HTTP 404, so test_urls_reachable failed test-external in run 42056917806. Every request of the plugin goes below /zones; the constant now names that path, which answers 403 (alive for the URL test), and the requested URLs stay byte-identical.

🎯 Fix: 95 %
💥 Break: 1 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:18:01 +00:00
kevinveenbirkenbachandClaude Opus 5.5 735308266e Merge two operator-approved sidetrees into feature/store-i18n
- fix/tests/store-i18n-lint-red (1205c4a2ae, 95/1): clears the raw-walk,
  project-root and env-key lint failures store-i18n would have pushed
- fix/sys-stk-front-proxy/body-filter-fold (4984f506c7, 92/2): the
  body_filter Lua is written in the vhost's single stack_host_contents
  call (about 184 s echoserver and 174 s galaxyserver per deploy,
  projected), and stack_host_contents compares bytes

🎯 Fix: 92-95 %
💥 Break: 1-2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90-94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:37:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4984f506c7 perf(sys-stk-front-proxy): write the body_filter Lua in the vhost's single contents call
Every front-proxy vhost wrote its body_filter Lua through a separate
stack_host_template task in sys-front-inj-all: two remote round trips when
unchanged (stat + file) and four to five when changed, 191 s over 54 vhosts
per echoserver deploy and 179 s over 48 per galaxyserver deploy. The Lua
now joins http.conf and the vhost in the Deploy task's stack_host_contents
call, written first as before, rendered through template_shallow with
application_id and domain pinned to the vhost.

- sys-front-inj-all/tasks/main.yml no longer writes the Lua; the new
  standalone.yml includes it and keeps the separate write for matomo's
  re-inject path, which has no vhost Deploy task
- stack_host_contents compares bytes instead of a universal-newline text
  read, so content with a CR stops reporting changed on every run and an
  existing non-UTF-8 file is replaced instead of crashing the module
- a new required diff_paths argument keeps --diff for the Lua only; vhost
  and http.conf content stay out of the diff

A Lua-only change still reports changed and fires reload-openresty once;
the served probe and the fallback redeploy keep their old triggers. Check
mode is unchanged.

Tests: the real stack_host_template action against the fold expression
across 99 roles and TLS/Tor/injection scenarios, byte-identical, with
trim and newline negative controls; owner, group and mode parity with the
copy module; byte compare with CR and non-UTF-8 files; the wiring of both
callers. The saving (about 184 s and 174 s per deploy, net) is projected
until the next deploys.

🎯 Fix: 92 %
💥 Break: 2 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:31:05 +00:00
kevinveenbirkenbachandClaude Opus 5.5 1205c4a2ae fix(tests): clear the four lint failures store-i18n would push
- test_no_raw_project_walk: the Synapse single-writer test and the
  compose single-writer lint walked role task dirs with rglob; both now
  filter the cached iter_project_files walk (the same 1283 role task
  files), and the Synapse test asserts it found any
- test_project_root_import: copy_batch's traversal guard rejected '..'
  through a banned literal; it now compares posixpath.normpath("/" + name)
  with "/" + name, which under the preceding conditions holds exactly when
  no part is '..', and new cases cover an interior a/../b and a
  nested/../../ escape
- test_used_keys_declared: the resolve test embeds the old task text,
  whose 'INFINITO_DESIGN' is a JS marker, not an environment key; marked
  with a same-line nocheck on the operator's approval

🎯 Fix: 95 %
💥 Break: 1 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:24:20 +00:00
kevinveenbirkenbachandClaude Opus 5.5 6d4e5f7c59 Merge feature/ci/wordpress0-playwright-parallel into feature/store-i18n
- feature/ci/wordpress0-playwright-parallel (de40211272, 88/6, merged on
  the operator's word below the gate): roles that opt in through
  playwright.parallel (WordPress) run independent Playwright suites in
  parallel under a CPU-pressure gate, with dependency deferral, an onion
  cap and Tor lock, crash counting and per-job artifacts; roles without
  the opt-in run as before. The next push's WordPress#0 CI run is its
  live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:11:52 +00:00
kevinveenbirkenbachandClaude Opus 5.5 de40211272 perf(test-e2e-playwright): run independent Playwright suites in parallel under a pressure gate
WordPress#0 runs 18 Playwright suites one after another and exceeds its
CI time budget. A role that opts in through playwright.parallel in its
meta/tests.yml (WordPress: [3, 1, 1] per variant) now hands its suites to
an async scheduler, run_parallel.py, after staging:

- a suite starts only when its dependencies finished here (Keycloak first)
  and the host's CPU pressure, avg10 and avg60, stayed at or below 36 for
  30 s; the first container always starts and running ones are never
  killed
- onion suites share their own cap of 3 and a Tor lock with a
  recreate-pending marker; a probe that times out on the lock re-probes
  the current circuits and, failing that, leaves only that app to the
  serial path
- an app whose dependency was dropped is deferred, and every deferred app
  gets the serial first pass before the existing retry, in round order
- each job gets --oom-score-adj=1000 instead of a memory limit; crashed
  pages, closed browsers and killed workers are counted and named in the
  per-app timing line
- each job's result.json, stdout and stderr land in run-parallel/ and are
  archived with the reports
- the async ceiling is the serial worst case per app and reachable
  network, curl's per-attempt max time included (now
  TEST_E2E_PLAYWRIGHT_ONION_PROBE_MAX_TIME); it stays above the CI step
  timeout and never raises a budget

Roles without the opt-in run exactly as before; the onion probe script
with an empty lock behaves byte for byte like the previous one.

Tests: 126 role unit tests and 30 negative controls cover the gate
window, admission order, onion cap and lock, marker cleanup, the bounded
wait, dependency deferral, crash counting, artifact collection, the
ceiling for the real WordPress#0 app list, and serial equivalence. No CI
run has exercised it yet; the next push is its live proof.

🎯 Fix: 88 %
💥 Break: 6 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 83
🚦 Gate: ⛔ fix 88 % < 90, break 6 % > 5; committed and merged on the operator's word, CI is the live proof

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:10:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 882e9c5062 perf(web-app-pgadmin): skip the create_app boot when the administrator is in sync
The realign task booted pgAdmin's whole Flask application on every deploy,
about 97 s on galaxyserver at cpus 0.2, even when nothing had drifted. A
stdlib pre-check now reads pgAdmin's SQLite configuration database
read-only and skips the boot only when exactly one active, unlocked
internal account with the Administrator role carries the declared e-mail
and its stored pbkdf2_sha512 hash recomputes from the declared password the
way Flask-Security 5.8.3 hashes it (NFKD, HMAC-SHA512 with the salt from
the keys table, libpass's round and salt limits). Every other state, an
unreadable or locked database, an unknown hash scheme, or a configuration
database moved out of SQLite by environment, falls through to the
unchanged create_app path.

The administrator password no longer reaches the host's /bin/sh argv
through the task environment: it arrives base64-encoded on the first stdin
line and is passed into the container by name only, so any byte sequence
survives and no fragment can reach stderr.

Tests: the pre-check agrees with converge against passlib; hash, role, key,
lock and duplicate-account drift, an over-long salt and a configuration
database override fall through; the rendered task runs through /bin/sh
with a password holding a newline, quotes, $, a backslash and unicode;
negative controls for each guard. The saving is not measured by a timed
deploy yet.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 14:28:17 +00:00
kevinveenbirkenbachandClaude Opus 5.5 5556dabbb8 Merge perf/plugins/template-shallow-shared into feature/store-i18n
- perf/plugins/template-shallow-shared (f974ab3832, 90/3): template_shallow
  moves from web-opt-rdr-domains to plugins/lookup, and the nine template
  lookups of the design, dashboard, logout and matomo injectors render
  through it instead of deep-copying every task variable per call

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:32:54 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f974ab3832 perf(plugins): share template_shallow and render the front injectors through it
ansible-core's template lookup deep-copies every task variable on each
call. The role-local template_shallow of web-opt-rdr-domains, which hands
the builtin a top-level copy whose values stay shared, moves to
plugins/lookup so every role can use it. The nine template lookups of the
front injectors switch to it: six in sys-front-inj-design and one each in
sys-front-inj-dashboard, sys-front-inj-logout and sys-front-inj-matomo.
None of the switched templates or the lookups they call writes into the
task variables.

Tests:
- the plugin renders like the builtin, never deep-copies the caller's
  values and leaves the caller's variables unchanged
- for every role with design sources, clearnet and onion vhosts, with and
  without a logo, the design facts are identical to the builtin's, or it
  fails with the identical error
- the dashboard, logout and matomo code renders identically
- the redirect loop of web-opt-rdr-domains still renders through the
  shared plugin

The saving per deploy is not measured by a timed deploy yet.

🎯 Fix: 90 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 87
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:31:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 75693c42e7 Merge two operator-approved sidetrees into feature/store-i18n
- perf/ci/wordpress0-arm64-pass2 (7b412ffa76, 90/4): the Matrix
  ansible-flavor Synapse vhost has one writer, so it no longer changes
  and restarts OpenResty in every swarm pass (WordPress#0)
- fix/sys-dns-cloudflare-records/none-want-group (d96fe45650, 93/3): a
  record set with an entry the converged check cannot judge stays
  pending instead of flipping

🎯 Fix: 90-93 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:55:24 +00:00
kevinveenbirkenbachandClaude Opus 5.5 d96fe45650 fix(sys-dns-cloudflare-records): keep a record set pending when one entry cannot be judged
cloudflare_records_converged skipped an entry its expected() rejects (a
TXT value given as an int, a ttl of null) when building the per-owner
declarations, yet cloudflare_dns still writes for such an entry. In
[TXT t 12345, TXT t "12345" ttl 300] the second entry was judged
converged and skipped while the first was applied with ttl 1, so the
record flipped on every run. A record set declared more than once now
stays pending as a whole when any of its entries cannot be judged, like
the hazardous solo and absent entries already do. No producer in the
repository builds such a list today.

🎯 Fix: 93 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 90
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:54:49 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7b412ffa76 perf(web-app-matrix): write the Synapse vhost once instead of twice per deploy
In the ansible flavor the Synapse vhost had two writers: the shared
front-proxy include rendered proxy.conf.j2 for the Synapse domain, and a
second task rendered flavor/ansible/synapse.conf.j2 to the same file
right after it. The two contents differed, so the file changed on every
deploy and notified restart-openresty, which in swarm is a
service update --force of OpenResty in every pass (WordPress#0, fork run
41926377524).

proxy.conf.j2 now includes the ansible-flavor Synapse template for the
Synapse domain itself, and the second task is gone. The rendered vhost is
identical to the former final content across TLS, Element Call,
clearnet and onion, so the 8448 listener and the Matrix RTC locations
stay. A real change now goes through the front-proxy's reload, which
reaches OpenResty in swarm since 7e5da9422c.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:51:23 +00:00
kevinveenbirkenbachandClaude Opus 5.5 59b46235e8 Merge four operator-approved sidetrees into feature/store-i18n
- fix/svc-prx-openresty/reload-mode-force-leak (7e5da9422c, 90/4):
  OpenResty handlers notified from compose-forced roles (matrix, BBB)
  resolve OpenResty's real address in swarm, so their reloads no longer
  skip, and /logout inside their vhosts no longer points at 127.0.0.1
- perf/sys-dns-cloudflare-records/batch-apply: records the zone already
  holds as declared skip the apply loop (59889bb5af, 92/3), and record
  jobs are polled every second while collecting (f51ab74697, 92/3)
- perf/web-opt-rdr-domains/tls-resolve: redirect vhosts render through
  template_shallow without deep-copying the variables (4a6bec9cf0, 92/3)
- perf/sys-front-inj-design/resolve-setfacts: each role design template
  renders once per include (0d04336f93, 92/3)

The three tunes come from the galaxyserver and echoserver 2fd6c6d deploy
logs; all four passed independent skeptic review.

🎯 Fix: 90-92 %
💥 Break: 3-4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86-89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:41:16 +00:00
kevinveenbirkenbachandClaude Opus 5.5 7e5da9422c fix(svc-prx-openresty): resolve OpenResty's own address in handlers notified from forced roles
web-app-matrix (ansible flavor) and web-app-bigbluebutton set
compose_mode_force: compose in their role vars, and those vars stay in
scope when their front-proxy includes notify the OpenResty handlers. In
swarm the handlers' container_address lookup then returned the bare
compose name, the validate retried for about 35 s against a container
that does not exist and ended ok, and the reload was skipped: 3 of 52
reloads in the WordPress#0 job of fork run 41926377524 (both matrix
.onion vhosts), so changed vhosts stayed unserved.

The validate, reload and restart handler tasks now clear
compose_mode_force at task level (task vars outrank the inherited role
vars), as the engine, rdbms and objstore entry points already do, and so
do the svc-db-openldap handlers, which the new lint also covers. The
proxy_pass lookup applies a caller's override only when the term names
the scope's own application_id, because the /logout location rendered
inside matrix and BBB vhosts pointed at 127.0.0.1 in swarm.

A real ansible-playbook integration test proves both modes and fails
without the fix; the new lint keeps every handler lookup of a fixed role
clearing the override.

🎯 Fix: 90 %
💥 Break: 4 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 86
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:40:41 +00:00
kevinveenbirkenbachandClaude Opus 5.5 5d5ba66046 Merge perf/stack-host-copy/changed-tree-bulk into feature/store-i18n
stack_host_copy writes a partly changed tree in one transfer instead of
one copy action per file (a643871997): the echoserver 2fd6c6d deploy paid
437 s for six changed design_assets copies, about 6 s per file. Shapes
the builtin copy treats differently make the new copy_batch module refuse
before it writes, and the regular copy runs instead. Operator-approved.
The independent review compared end state, changed, check and diff mode
against the builtin copy in a real playbook run and found them equal; a
write failing partway reports changed=true, where copy reports false.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 90 % | 2 % | 100 % | 15 % (estimated from round trips) | ~100 % | 100 | ⏳ score passes, gate.py not run |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:24:24 +00:00
kevinveenbirkenbachandClaude Opus 5.5 f51ab74697 perf(sys-dns-cloudflare-records): poll record jobs every second while collecting
The collect step polled each async record job every 5 s, so a job that
finished right after a poll idled up to 5 s before its result was read.
The interval is now cloudflare_collect_interval (1 s), and the poll cap
derives from it, so the attempt's collect deadline, the retry split and
ignore_errors behave exactly as before.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:21:37 +00:00
kevinveenbirkenbachandClaude Opus 5.5 59889bb5af perf(sys-dns-cloudflare-records): skip records the zone already holds as declared
Every deploy sent each declared record through cloudflare_dns, one async
job per record, although nearly all of them already stood unchanged; the
galaxyserver 2fd6c6d deploy spent about two minutes collecting them.

A new action plugin, cloudflare_records_converged, reads each zone once
(all pages) and splits the declared records into pending and converged.
It judges A, AAAA, CNAME, MX, TXT and SRV with the same lookup key and
the same defaults the community.general 14.0.0 module applies; every
other type, every solo or absent entry, and every record set whose
entries would undo each other stay pending, as do all records of a zone
that cannot be read (with a warning). Only pending records reach the
unchanged apply and retry loop, so failures stay per record and loud.

The test pins the plugin's defaults against the module's argument_spec
and the reviewed collection version, and covers the hazardous groups
with a mutation check.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:18:57 +00:00
kevinveenbirkenbachandClaude Opus 5.5 4a6bec9cf0 perf(web-opt-rdr-domains): render redirect vhosts without deep-copying the variables
ansible-core 2.20's template lookup deep-copies all task variables on
every call. The redirect loop renders two templates per source domain,
so on galaxyserver this copy dominated the resolve and render step.

A role-local lookup, template_shallow, hands the builtin template lookup
a top-level copy of the variables whose values stay shared with the
caller; nothing on the reachable template and lookup paths mutates them
in place. Rendered output is identical across 12 variants (TLS, Tor,
logout, self-signed, wildcard keep_path), per-item failures and
undefined-variable messages included. A real playbook with 65 items
went from 10.5 s to 5.8 s.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:11:56 +00:00
kevinveenbirkenbachandClaude Opus 5.5 0d04336f93 perf(sys-front-inj-design): render each role design template once per include
The two Resolve tasks spent nearly all their time in the per-call
deep copy of ansible-core 2.20's template lookup, because every set_fact
value re-rendered the lazy task variables: the style include ran its
templates 6 times (10 with logo.css), the script include 6 times. On the
2fd6c6d deploys both together took 125 s on galaxyserver and 174 s on
echoserver.

The composed bodies now reach app.css.j2 and design.js.j2 through
template_vars, app_style_present tests the body directly (app.css.j2
always emits ":root {"), and the design check is inlined. Per include
the style lookups drop to 3 (5 with logo.css) and the script lookups to
4. The facts are byte-identical for 77 roles across clearnet, onion and
logo on or off, and for 36 edge cases, error messages included.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:08:50 +00:00
kevinveenbirkenbachandClaude Opus 5.5 a643871997 perf(stack-host-copy): write a changed tree in one transfer
When the tree check finds differing entries, pack exactly the differing files into one tar archive in source walk order, transfer it once into the remote tmpdir and let the new copy_batch module write each file through atomic_move with the copy module's directory, mode and unsafe_writes semantics. Mode-only drift and leaf directories settle in the same call; unchanged files are not sent.

Every shape the regular copy treats differently makes the module refuse before it writes: a symlink, directory or other non-regular entry at a destination, a missing path below a non-directory or a symlink loop, and two source paths reaching one remote entry. The regular copy then runs. Check mode, diff mode, unprivileged become, a dest holding $ and an archive above 16 MiB keep the regular copy. A write failure fails the task and names the file.

Executed with ansible-playbook -c local, pipelining on and off: the trees this action leaves equal the builtin copy's, including the symlink fallback; a 6-file changed tree took 2 module calls and 1 transfer instead of 14 and 4. Unit: a flat 15-file tree with every file changed drops from 31 module calls and 15 transfers to 2 and 1. Two dialectic rounds with three independent skeptics; every hole they found is fixed and tested. A third independent review compared end state, changed, check and diff mode against the builtin copy in a real playbook run and found them equal; one deliberate difference remains: a write failing partway reports changed=true with the files already written, where the builtin copy reports changed=false.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 90 % | 2 % | 100 % | 15 % (estimated from round trips) | ~100 % | 100 | ⏳ score passes, gate.py not run |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 11:06:49 +00:00
kevinveenbirkenbachandClaude Opus 5.5 009e4359b8 perf(sys-front-inj-design): skip repeat identical stylesheet writes per run
On a production deploy every vhost of an app re-includes the design and
CDN roles. That repeated two things:

- sys-svc-cdn checked and skipped its two-task run-once block on each of
  the ~50 repeat includes. The block is now one guarded include of
  tasks/00_core.yml, the convention the other roles use, so a repeat
  include skips one task instead of two.
- sys-front-inj-design wrote the same per-app style.css again for every
  vhost of a network that shares it (logout, stalwart, matrix, bluesky)
  and removed an already removed stylesheet once per vhost (the 11
  WordPress vhosts). A per-run record of path and content hash, keyed by
  application and the network's stylesheet file, now skips a write of
  identical content and a repeat removal. Different content still
  writes and the last write wins; each network keeps its own file; the
  record is set only after the write or removal succeeded.

Measured from galaxyserver and echoserver 2fd6c6d: about 24 s saved per
galaxyserver deploy and 23-33 s per echoserver deploy (estimate until the
next deploy re-measures it). Two independent reviews found no
correctness or coverage hole.

🎯 Fix: 92 %
💥 Break: 3 %
🌱 Efficiency: –
⚡ Speed: –
🛡️ Security: ~100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, efficiency and speed <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 08:40:44 +00:00
kevinveenbirkenbach 025ebe9754 Merge remote-tracking branch 'origin/main' into feature/store-i18n 2026-11-04 23:28:17 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a63cbcb1f7 fix(apparmor): leave a read-only /proc/sys alone in teardown and restore
Fork run 41911075483 (feature/store-i18n 2fd6c6d310) lost both workspace
jobs, debian:13 and ubuntu:26.04, in make security-apparmor-teardown:

  common.sh: line 75: /proc/sys/kernel/apparmor_restrict_unprivileged_unconfined: Read-only file system

Inside the workspace container /proc/sys is mounted read-only, and the
write that #1371 added ran under set -e. A sysctl that cannot be written
now leaves the restriction as it is with a log line instead of failing
the teardown, and only a successful write records the previous value.
Restore keeps its record when it cannot write the sysctl back.

The tests call the helpers under set -euo pipefail like teardown.sh and
use /proc/sys/kernel/ostype as a sysctl nobody can write; both new tests
fail against the previous common.sh.

| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
| --- | --- | --- | --- | --- | --- | --- |
| 95 % | 3 % | – | – | ~100 % | 92 | ✅ 90/5, efficiency and speed <= 100, security 100 % |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 23:25:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 4596f81e01 test(svc-runner): read the role files through the ROLE_FILE constants
test_file_spot rejected the hand-built vars/meta paths in the new
compose project test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:39:08 +01:00
kevinveenbirkenbach 2fdf419d7a Merge branch 'fix/svc-runner/network-project-preflight' into feature/store-i18n 2026-11-04 21:21:21 +01:00
kevinveenbirkenbachandClaude Opus 5.5 a57e9b944f fix(svc-runner): run the runner stack as its entity's compose project
svc-runner started its stack with 'compose --chdir /opt/github-runner up'
and no --project, so the wrapper named the project after the directory,
github-runner, while every other role uses its entity name. Pass 2 of the
swarm drill in fork run 41744154601 (svc-runner#0) then stopped in the
pre-flight: network 'runner' belongs to compose project 'github-runner'.
The deploy now passes --project from entity_name, and a unit test renders
the real task, runs the real wrapper and checks the pre-flight accepts the
network it creates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:17:13 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c0dd930d29 fix(merge): run the #1212 rehash before compose pull and every compose up
The main merge placed main's rehash handler behind store-i18n's compose pull
and build handlers, so pull ran against stale hashed config names, and the
store-i18n-only lone recreate path (compose-service-up) never rehashed at all.
The handler now sits right after the instance-dir guard, as on main, and
listens on compose-service-up too. The test covers every handler that runs
compose up instead of assuming there is only one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:01:08 +01:00
kevinveenbirkenbach fb004b88a0 Merge remote-tracking branch 'origin/main' into feature/store-i18n
# Conflicts:
#	roles/web-app-mattermost/meta/services.yml
2026-11-04 19:43:38 +01:00
kevinveenbirkenbach 70f3716fbe Merge pull request #1370 from infinito-nexus/fix/update/follow-warn-on-unreadable-digest
fix(update): say when follow cannot read Docker Hub, and retry a 429
2026-11-04 19:16:52 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c068e44d1b fix(update): treat an incomplete Docker Hub response as unreadable
followed_tags caught OSError, ValueError, KeyError and RuntimeError, but
http.client.IncompleteRead, BadStatusLine and LineTooLong derive from
http.client.HTTPException only. One truncated digest response therefore
escaped to collect_updates (docker.py:344, no try) and aborted the whole
Docker update run instead of printing the follow warning for that image.
Every other fetch site of the updater already catches HTTPException
(docker.py, pip.py).

Reported by CodeRabbit on #1370.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:16:16 +01:00
kevinveenbirkenbach 4367272f8f Merge pull request #1348 from infinito-nexus/fix/sys-svc-compose/config-hash-lag/1212
fix(sys-svc-compose): recompute config and secret hashes before compose up
2026-11-04 19:08:59 +01:00
kevinveenbirkenbachandClaude Opus 5.5 248c9c5748 refactor(web-app-funkwhale): drop the same_origin prefix from the module sources
test_prefix_grouped_files flagged roles/web-app-funkwhale/files/python/same_origin/, whose three files all started with same_origin_; they were untracked while the lint ran in the objstore worktree. The sources are now settings.py, storage.py and urls.py, volumes.yml mounts them under the unchanged targets /app/same_origin_settings.py, /app/same_origin_storage.py and /app/same_origin_urls.py, so module names and DJANGO_SETTINGS_MODULE stay the same; the mirrored tests and the network_modes.md link follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:28:29 +01:00
kevinveenbirkenbachandClaude Opus 5.5 83923b27ad Merge branch 'fix/objstore/app-proxy-delivery' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:20:42 +01:00
kevinveenbirkenbachandClaude Opus 5.5 80fc8487a8 Merge branch 'fix/web-app-gitea/ldap-source-converge' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:10:26 +01:00
kevinveenbirkenbachandClaude Opus 5.5 6f491381be fix(web-app-gitea): converge duplicate LDAP auth sources and keep the bind password off the host argv
Setup ran add-ldap on every deploy under the name LDAP ({{ DOMAIN_PRIMARY }}); the name column compares case-sensitively, so a casing change created a second source (echoserver had LDAP (Infinito.Nexus) ID 1 with 331 users and LDAP (infinito.nexus) ID 3 with 12), and setup updated only the new one. Cleanup looked the source up with an awk regex in which the parentheses grouped, so it never matched. Setup and cleanup now find LDAP sources by their type (LDAP (via BindDN)), the source is named LDAP, and gitea_ldap_source_plan decides from the per-source user and external-login counts (files/sql/auth-source-users.sql through database_query, only when more than one source exists): keep the source users are linked to, else the lowest id, and delete the others; users on more than one source fail the deploy loudly and delete nothing. The bind password now travels on stdin, is read on the host, forwarded with container exec -e and expanded only by the shell inside the container, with no_log; it left GITEA_LDAP_AUTH_ARGS. On echoserver the 12 users of ID 3 were moved to ID 1 by hand first, so the next deploy keeps ID 1 and deletes ID 3.

Residual: the Gitea CLI takes the password only as --bind-password, so it sits in the argv of the in-container gitea process while add-ldap or update-ldap runs. The IAM line's 5f152d9f28 passes it through task environment: and keeps the name lookup; at the IAM merge take this side for the LDAP tasks and the GITEA_LDAP_* vars.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:10:09 +01:00
kevinveenbirkenbach f40964991a Merge branch 'fix/update/pacman-sig-404' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	roles/update/vars/main.yml
2026-11-04 17:46:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1c561a1e7d test(install): keep role-file paths out of the stage fixtures
tests/lint/ansible/roles/test_file_spot.py rejects a hard-coded
"roles/a/tasks/main.yml" in test_stage.py and asks for the
ROLE_FILE_TASKS_MAIN constant. The fixture only needs some nested file,
not a role, so it uses pkg/a/data.yml and pkg/a/.git instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:09 +01:00
kevinveenbirkenbachandClaude Opus 5.5 330b85f9e7 fix(apparmor): lift the unprivileged-unconfined restriction in teardown
On runner image 20261101 kernel.apparmor_restrict_unprivileged_unconfined
is on, so the kernel turns runc's change_onexec into the stack
docker-default//&unconfined. docker-default grants no signal to an
unconfined peer, and every in-container kill() is denied; run 41743807934
lost four package builds that way (journal: apparmor="DENIED"
operation="signal" comm="rsync" signal=usr2).

teardown.sh now writes 0 to the sysctl and records the value it found
under /run/infinito; restore.sh writes that value back and drops the
record. A missing sysctl or one already at 0 is left alone, and a second
teardown keeps the first record. CI runs teardown before every deploy,
workspace and dns job; dev hosts through make environment-bootstrap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:08 +01:00
kevinveenbirkenbachandClaude Opus 5.5 fd05f85d44 fix(install): stage the package build workspace with tar, not rsync
Run 41743807934 lost Matrix#2, Workspace (debian:13), Gitea#0 and Wazuh#0
to the full job budget right after "Preparing build workspace". On runner
image 20261101 the kernel stacks every container onto
docker-default//&unconfined, which refuses every in-container kill(); the
host journal logs apparmor="DENIED" operation="signal" comm="rsync"
signal=usr2. rsync's local copy waits for exactly that SIGUSR2 and sleeps
until the job is killed.

scripts/install/stage.sh copies the tree through a tar pipe, which sends
no signal; --format=posix keeps nanosecond mtimes. package.sh uses it on
the arch and debian paths and installs tar instead of rsync.

Copying this repository both ways gives identical content, modes, types,
link targets and mtimes on all 11306 entries. test_stage.py runs stage.sh
under a seccomp filter that fails every signal syscall with EACCES; the
old rsync copy hangs past its timeout under the same filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:35:08 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c62b3d2e53 fix(web-app-gitea): apply ROOT_URL on every start so the SSO callback matches Keycloak
On a long-lived instance the role's DOMAIN, ROOT_URL and SSH_DOMAIN variables never reach app.ini: the image's setup script writes them only when app.ini does not exist yet and applies only GITEA__section__KEY variables on later starts. echoserver's Gitea therefore still runs with ROOT_URL https://code.cymais.cloud/, sends that host as the OAuth2 callback, and Keycloak answers 'Invalid parameter: redirect_uri' (its log shows exactly https://code.cymais.cloud/user/oauth2/SSO%20Login%20%28INFINITO.NEXUS%29/callback). env.j2 now sets GITEA__server__DOMAIN, GITEA__server__ROOT_URL and GITEA__server__SSH_DOMAIN. The OAuth2 source is named 'SSO Login', because external_login_user.provider holds 25 characters and the 26-character domain-suffixed name would break every account link; setup and cleanup find the source by its type instead of by name, so the rename happens in place and no second button appears. OpenID 2.0 sign-in and sign-up are switched off. CI missed it because every CI run installs fresh and the Gitea specs never click the SSO button.

The IAM line's 144871ada1 carries the same rename and lookup plus a patched Gitea build for onion sign-in, but keeps the plain ROOT_URL variable; at the IAM merge keep these GITEA__server__ lines together with that patch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:23:36 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ac384eb872 fix(objstore): serve Decidim and Funkwhale object-store files from the app's own origin
Decidim#0 and Funkwhale#0 of fork run 41744154601 failed on the tor rows because both apps handed the browser presigned or public URLs on the object-store host, which publishes no onion vhost. Decidim now sets STORAGE_CDN_HOST to its own origin and switches Active Storage to rails_storage_proxy; an initializer routes direct uploads through the disk-service token route (Rails writes them to S3 server-side), points the redirect and legacy blob routes and the rich-text BlobRenderer at the proxy, safelists storage reads in Rack::Attack, and refuses to boot when the patched activestorage 7.2.2.2 / decidim 0.31.6 signatures change. Funkwhale runs a settings module with a storage whose public names (attachments, sized images, dynamic preferences) resolve to MEDIA_URL and whose exports get a signed one-hour token link; a /media view checks every name against clean_name and rejects backslashes and percent signs, signs per request method and answers with X-Accel-Redirect; the front nginx routes those paths to the API, sends relative protected covers back through it and no longer forwards X-Forwarded-Host, which broke the presigned signature. Both apps reach the store over its internal URL; the public-host CSP entry, extra_hosts entry and CA bundle are gone. The Playwright specs assert the direct upload goes to the app and accept no object-store origin.

Verified by compose deploys of both roles, clearnet update and tor-only reinstall, failed=0 with green Playwright and no object-store host in the rendered pages or APIs; swarm and Decidim variant 1 (disk storage) are not deployed yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 16:17:29 +01:00
kevinveenbirkenbach dedecefcbd Merge branch 'feature/web-app-wordpress/options-simplify' into feature/store-i18n
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	roles/web-app-wordpress/vars/main.yml
2026-11-04 12:58:32 +01:00
kevinveenbirkenbachandClaude Opus 5.5 268b2a46f9 feat(web-app-wordpress): set admin_email, blogname and the FreedomTranslate options through the options map
WORDPRESS_OPTIONS now merges the role's own contributions (admin_email from the platform administrator, blogname from the design title while the design service is on, the freedomtranslate-wp config.options while that addon is on) under services.wordpress.options, so the inventory still wins and wordpress_options_checked guards the merged map. design.php no longer writes blogname, the addon task no longer writes its options, and WORDPRESS_OPTIONS_OWNED_ELSEWHERE drops the three. Every deploy now converges admin_email, and blogname while the design is on; a write WP-CLI reports as unchanged no longer counts as a change, and the FreedomTranslate readback runs after the options write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 12:53:11 +01:00
kevinveenbirkenbachandClaude Opus 5.5 50ee46ed22 refactor(provision): group the services disabler modules in a services package
test_prefix_grouped_files flagged cli/administration/inventory/provision/services_disabler.py and services_disabled_conflicts.py, which share the 'services' prefix since the conflict check moved into its own module. Both now live in cli/administration/inventory/provision/services/ as disabler.py and disabled_conflicts.py; every importer, the purge script, the BLE001 per-file ignore and the requirement 037 link follow the new path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 12:23:07 +01:00
kevinveenbirkenbachandClaude Opus 5.5 c3e2114e91 Merge branch 'fix/web-app-wordpress/permalink-options' into feature/store-i18n
Conflicts resolved: the inventory validator keeps store-i18n's custom-role
declaration skip and gains the open-subtree check; the suppression catalogue
keeps both new rows; WORDPRESS_PERMALINK_STRUCTURE gives way to the options
map. WORDPRESS_OPTIONS_OWNED_ELSEWHERE also lists the store-i18n writers the
merge brought along: the design logos (WORDPRESS_DESIGN_LOGOS keys) and the
freedomtranslate options; two custom-role validator tests pass open_paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 11:35:18 +01:00