Commit Graph
8 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 493fedc6c6 refactor(tests): move the role unit mirrors under tests/unit/python
The role scripts moved into per-language folders but their unit mirrors did
not follow, so the index carried paths under tests/unit/roles/ that no longer
existed on disk while their counterparts under tests/unit/python/roles/ were
untracked.

That stale index broke the gate while naming the wrong culprit: the
repository lint reports "rsync --backup in 19 place(s)" because it counts
files it cannot open, and every entry was an [Errno 2] No such file or
directory rather than an actual rsync call.

Fifteen of the moves are recognised as renames; the rest are __init__ files
that collapsed into the new tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:33:48 +02:00
kevinveenbirkenbachandClaude Opus 5 54a6f3b533 feat(svc-ai-litellm): warm the chat model during the deploy
A backend pulls its preload model at deploy time but only loads it into
memory on the first inference. That cold load outlives a reverse proxy's
read timeout, so the first consumer request comes back as a 504 while every
later one succeeds.

Send one max_tokens=1 completion from inside the litellm container right
after the per-consumer keys exist, where the container id, master key and
port are already resolved. Five retries cover a gateway that is not
listening yet.

This pairs with the ollama keep_alive setting: the warm-up pays the load
once, keep_alive stops it decaying after five idle minutes.

Note this hardens the deploy: a gateway that cannot serve its configured
model now fails the role instead of surfacing later as a consumer error.
That makes the openrouter/auto fallback with its empty API key loud rather
than silent.

The script ships with a mirrored unit test rather than a nocheck waiver; it
patches urlopen and asserts the URL, method, authorization header, model,
max_tokens and that the timeout exceeds a proxy read timeout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:02:17 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 b387c80528 fix(container): reject a directory standing in for a CA bind source
must_exist accepted any existing path, so a directory that dockerd auto-created where a bind source was missing passed the guard. The wrapper was then mounted over the container path and set as --entrypoint, and every image that has an ENTRYPOINT died with rc 126 'is a directory: permission denied' — seven identical retries, because the directory persists. Requiring a regular file routes that state into the existing 'CA injection disabled' fallback, which the same suite already exercises green when the wrapper is simply absent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:02 +02:00
kevinveenbirkenbach d391b4cbb8 test(unit): cover the postmarks, mediawiki and baserow shims
Bookwyrm has tests for the same trusted-header SSO bridge and settings
patcher; postmarks and baserow ship theirs untested, and the composer
require-dev stripper is a pure transformation nobody exercised.

Each test was checked by breaking its source: the admin gate failing
open, the group matcher loosened to a prefix, require-dev left in place
and the SSO gate forced open all turn the new tests red.
2026-08-22 22:31:37 +02:00
kevinveenbirkenbachandClaude Opus 5 b192415a5b fix(decidim): take the ruby fragment path from its one source
patch.py located files/ruby/ relative to its own __file__. That holds in
the repo, where the script sits next to the fragments, but the image
copies the script to /tmp/patch.py and the fragments to /tmp/ruby/, so
parent.parent resolved to / and the build died with
FileNotFoundError: /ruby/omniauth_provider.rb on every retry.

DECIDIM_PATCH_RUBY_DOCKER already decides where the COPY puts the
fragments. Reading it in the script makes that variable the only place
the location is stated, so the layout can change without a second
place silently disagreeing. Missing variable raises instead of
guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 13:03:48 +02:00
kevinveenbirkenbachandClaude Opus 5 8f4d842375 test(lint): require a mirrored unit test for every shipped script
`roles/<role>/files/<lang>/<rel>` is tested by
`tests/unit/<lang>/roles/<role>/files/<rel>`, renamed to that language's
convention. The language appears once, in `tests/unit/`, because the suite
already says what the file is. Without a fixed mapping a shipped script is
tested wherever its author happened to look, and nobody can tell an untested
file from one whose test lives somewhere unexpected.

Most of what a role ships is a script against a live console -- it boots a
Laravel kernel, imports frappe, or writes through an ActiveRecord model -- so
the escape is a `nocheck: mirrored-unit-test` comment in the file, with a
rationale of its own. A file carrying both a test and an exemption fails: the
exemption outlived its reason and reads as "untested" to everyone after you.

Five scripts turned out to have real units behind the console wrapper, so they
get tests rather than markers (62 cases):

  * svc-net-tor/onion_self_reach.py -- pins that an HTTP error status, a closed
    connection and a refused connection all count as reached, because each
    proves the loopback fast path resolved; only a timeout or an unresolved
    name is a failure.
  * sys-lock/sys-lock.py -- `parse_time_to_seconds` unit precedence, and that a
    service still activating holds the lock (otherwise the gate races it).
  * svc-opt-keyboard-color/script.py -- colour interpolation and the
    zero-length transition window that would otherwise divide by zero.
  * sys-svc-compose/swarm/secret_rotate.py -- including the binary fallback a
    TLS key needs, and that a missing source leaves the name alone rather than
    pinning the hash of a file that is not there.
  * sys-front-inj-logout/logout.js -- pins that `href="/logout"` alone does not
    match and a non-interactive tag is never patched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:10 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00