Commit Graph
19 Commits
Author SHA1 Message Date
Alejandro Roman accc9e4c7c feat(UI): fix migration, errors happening in local deploy and swarm integraton issues, tor merged and implement stalwart with tor 2026-09-01 12:34:57 +04:00
Alejandro Roman 333079cc2a feat(UI): fix migration, errors happening in local deploy and swarm integraton + merged main 2026-08-30 21:39:52 +04:00
kevinveenbirkenbachandClaude Opus 5 ae71d8ea2c fix(test): give Composer an archive extractor
Without a zip extension or an unzip/7z binary Composer silently drops to
--prefer-source and mirror-clones every dependency, which exceeds its
300 s process timeout on phpunit.git. That took the Arch workspace job
from a 2m16s test-unit to a 16m08s failure. Debian never saw it because
apt pulls php-zip and unzip in as Composer dependencies; pacman does not.

Arch ships no zip extension and no php-zip package, so the extractor has
to be the unzip binary rather than the extension. Its mapping already
covers all five package managers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 02:40:06 +02:00
kevinveenbirkenbachandClaude Opus 5 3781c30235 fix(env): write the cache-stack declaration from a handler, not the registry
48845db97 made three capability decisions declarable and named them INFINITO_*, which put them under setdefault: the live environment wins and the value is written back. Makefile exports BASH_ENV, so .env is re-exported into every make recipe, and scripts/tests/code/run.sh reaches dotenv through 'make setup' on every test run. A declaration made for one run therefore became a machine fact on first regeneration and stayed one - that is what made the workspace failure intermittent rather than reproducible, and why fedora died on 11 assertions where debian passed the same code.

A dedicated handler writing through eb.set recomputes the value every time and discards whatever the environment held, so the key stays registered in default.env and visible to the lints while remaining structurally unpinnable. running_on/github.py already worked this way; the distinction now has a name in utils/env/README.md. The exporting shell still steers its own run, because load.sh preserves non-empty caller values.

The other two declarations are deleted rather than moved. image_mirror_enabled and docker_root_ephemeral both derive from runs_on_github(), and the orchestrator does not unset INFINITO_RUNNING_ON_GITHUB, so inside the workspace they were already true and the declarations restated a value that held anyway. Measured across all four runtime contexts: only the workspace row changes, and only for the cache.

Verified by reversing the probe: 'INFINITO_CACHE_STACK=false make dotenv' writes .env=true, where the registry path wrote the exported value back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 23:24:15 +02:00
kevinveenbirkenbachandClaude Opus 5 48845db979 refactor(profile): split runtime capabilities from runtime identity
One boolean answered five unrelated questions: whether the pull-through cache stack runs, whether image pulls take the GHCR mirror, whether the docker data root may be wiped, plus runtime naming and inventory resolution. The workspace is the environment where those answers diverge — it simulates a developer machine on a real runner — so getting one of them meant misreporting where the suite runs and silently paying for the other four.

is_ci() and runs_on_github() stay facts about the machine. cache_stack_enabled, image_mirror_enabled and docker_root_ephemeral become decisions that default off those facts and can be declared explicitly, so an environment states what it wants. Every default reproduces the previous condition exactly, including the distinction the old mirror and wipe gates drew between a real runner and act.

73ae40585 got the cache stack by flipping the identity flag and lost the docker-root wipe with it; the workspace now declares all three capabilities instead. Pairs with 4c068d1cf, which makes the resulting stray bind sources harmless rather than fatal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:03 +02:00
kevinveenbirkenbachandClaude Opus 5 b387c80528 fix(container): reject a directory standing in for a CA bind source
must_exist accepted any existing path, so a directory that dockerd auto-created where a bind source was missing passed the guard. The wrapper was then mounted over the container path and set as --entrypoint, and every image that has an ENTRYPOINT died with rc 126 'is a directory: permission denied' — seven identical retries, because the directory persists. Requiring a regular file routes that state into the existing 'CA injection disabled' fallback, which the same suite already exercises green when the wrapper is simply absent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:02 +02:00
kevinveenbirkenbachandClaude Opus 5 8230c187a2 fix(swarm-test): ask the swarm for the DB service
03_wait_converge built the DB service name from the engine type:
"${DB_DEP}_${DB_DEP}" -> mariadb_mariadb. That name is only correct when
the database is central. force_shared_db deliberately leaves a
manager-pinned role carrying a literal `shared: false` alone, so
web-app-magento keeps its embedded engine, and sys-svc-rdbms deploys it as
magento_database inside magento's own stack.

The name came from the engine, which is global, and was looked up with
`docker service ls` across the whole swarm instead of the role's stack, so
the `shared` flag never entered the check at any step. Both polarities were
wrong:

  variant #1 - no consumer pulls the central provider in, so
  mariadb_mariadb does not exist. Run 32630684501 job 97175522113 polled it
  ninety times and failed the drill, while the rescue artifact shows
  magento_database at 1/1 Running and the play at failed=0.

  variant #0 - matomo and mailu declare a Jinja `shared`, get force-shared
  and drag svc-db-mariadb in. Job 95662862942 then reported
  `db(mariadb): 1/1 Running 2 hours ago` beside `magento: Running 21
  minutes ago` - green on a database magento never touches.

resolve_db_service asks the deployed swarm which service carries this
role's database: <entity>_database first, else <dep>_<dep>. The two cannot
collide, because sys-svc-rdbms renders the _database sidecar only under
`lookup('database', application_id, 'local')`. Exact-line matching, not
docker's substring name filter, per 246558a13.

Only web-app-magento changes behaviour: it is the one role that is both
pinned by force_shared_db and seen by the DB_DEP grep. matrix and
bigbluebutton never reach the DB half - their entity key carries no image,
so skip_if_no_swarm_service exits first - and discourse resolves
postgres_postgres exactly as before, in every variant.

A role that declares a DB dep no service serves now says so and gates the
app half alone, instead of polling a name nobody deployed.

diagnostics.sh carried the same guess in its container filter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 16:38:53 +02:00
kevinveenbirkenbachandClaude Opus 5 e22ba686f5 fix(lint): prune gitignored vendor trees from the project walk
The composer vendor tree lands in the repo root during the workspace suite, and the cached project walk descends it, so third-party files turn repo lints red. 8ff6b490b patched two consumers; the filename convention lint was the third.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 10:12:12 +02:00
kevinveenbirkenbachandClaude Opus 5 3136110ccf fix(test): install the PHP extensions PHPUnit requires
ensure_php_toolchain installed the php and composer binaries and stopped
there. A distro ships the interpreter and its extensions as separate
packages, and ensure_command_present returns early once shutil.which
finds php - which it always does since the image bakes the interpreter -
so the extensions were never provisioned on any path.

composer then refused to resolve the vendor tree with
"phpunit/phpunit requires ext-dom * -> it is missing from your system"
and test-unit-php failed in every workspace job. The loaded ini list in
those logs shows 20-mbstring.ini present and no 20-dom.ini, 20-xml.ini
or 20-xmlwriter.ini, i.e. php-xml missing.

Extensions are now probed against php -m and installed from a per-manager
mapping, then re-probed so a package that does not deliver the extension
raises instead of leaving composer to fail later. The loop sits in
ensure_php_toolchain because both entry points pass through it: the image
build via utils.install.interpreters and the test run via
scripts/install/php.sh, so a prebuilt image is repaired at test time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 04:31:56 +02:00
kevinveenbirkenbach c9b45f5804 style(test): wrap the assertEqual ruff keeps reformatting
The committed line was 89 characters, one over the default limit, so
every autoformat run rewrote it and it kept reappearing as an unrelated
worktree change.
2026-08-22 22:35:17 +02:00
kevinveenbirkenbach d391b4cbb8 test(unit): cover the postmarks, mediawiki and baserow shims
Bookwyrm has tests for the same trusted-header SSO bridge and settings
patcher; postmarks and baserow ship theirs untested, and the composer
require-dev stripper is a pure transformation nobody exercised.

Each test was checked by breaking its source: the admin gate failing
open, the group matcher loosened to a prefix, require-dev left in place
and the SSO gate forced open all turn the new tests red.
2026-08-22 22:31:37 +02:00
kevinveenbirkenbach 2e3f257134 feat(image): bake the php and ruby interpreters into the image
The unit suites had to install their toolchain on every run. The image
now carries the interpreters, so the make prerequisite finds them present.

The Composer vendor tree deliberately stays out of the image build: the
package cache is reached through the compose extra_hosts mapping, which
only exists for a running container, so a build-time composer install
would bypass the cache and talk to packagist directly. Fetching it from
the test target keeps it on the cached path.
2026-08-22 20:56:55 +02:00
kevinveenbirkenbach 3222904232 fix(test): install php and ruby instead of skipping their suites
test-unit-php and test-unit-ruby exited 0 whenever the interpreter or the
vendor tree was missing, so both suites were silently absent from every
gate. They now provision their toolchain through install-php/install-ruby
and fail loudly when that is impossible.

The three ensure_*_present helpers collapse into ensure_command_present
in system_pkg, and the ruby runner rejects an empty suite because
minitest reports success when nothing was required.
2026-08-22 20:30:40 +02:00
kevinveenbirkenbach 9b78933153 refactor(env): group the CA handlers into their own subpackage
The two ca_* handlers sat flat in utils/env/handlers/infinito, which the
prefix-grouping lint rejects, and the bundle-candidates comment exceeded
the .env comment length limit.
2026-08-22 20:24:53 +02:00
kevinveenbirkenbachandClaude Opus 5 d14354e892 feat(diagnostics): sample what the resolver answers, not just its socket
Run 32118850138 lost the :53 listener, so the sampler counts listeners.
Run 32538854067 lost DNS with the listener still bound: sockets-udp.txt
showed dnsmasq holding 0.0.0.0:53 while every forwarded query timed out,
and the sampler reported that hour as healthy.

Each tick now also records what a glibc caller gets back — for the zone
this resolver owns, derived from its own address= line, and for one name
it must forward — with the latency that caller paid. The rescue collector
probes the in-zone name too: probing external names alone cannot separate
"forwarding is gone" from "dnsmasq is gone".

The sampler now also runs on the swarm lab DNS nodes; it only ever ran in
the compose path, which is why the Magento artifact carried no sampler log
at all. Its location comes from group_vars FILE_DNS53_SAMPLER_LOG and is
required at every call site, so a run cannot quietly collect nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 17:55:05 +02:00
kevinveenbirkenbachandClaude Opus 5 b192415a5b fix(decidim): take the ruby fragment path from its one source
patch.py located files/ruby/ relative to its own __file__. That holds in
the repo, where the script sits next to the fragments, but the image
copies the script to /tmp/patch.py and the fragments to /tmp/ruby/, so
parent.parent resolved to / and the build died with
FileNotFoundError: /ruby/omniauth_provider.rb on every retry.

DECIDIM_PATCH_RUBY_DOCKER already decides where the COPY puts the
fragments. Reading it in the script makes that variable the only place
the location is stated, so the layout can change without a second
place silently disagreeing. Missing variable raises instead of
guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 13:03:48 +02:00
kevinveenbirkenbachandClaude Opus 5 ce244132a6 fix(test): install node instead of dying on its absence
test-unit-javascript ran `node --test` without declaring where node
comes from, while its python sibling declares `install`. On a bare
distro image the target died with `exec: node: not found` (127), which
took down the pre-commit hook and all five test-workspace jobs.

The installer stack already knows how to pull a command from the system
package manager; node was simply missing from the table. Wiring the
target to install-node provisions it the same way php, ruby and npm are
provisioned, and fails loudly when the runtime cannot be installed
rather than skipping the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 13:01:36 +02:00
kevinveenbirkenbachandClaude Opus 5 8f4d842375 test(lint): require a mirrored unit test for every shipped script
`roles/<role>/files/<lang>/<rel>` is tested by
`tests/unit/<lang>/roles/<role>/files/<rel>`, renamed to that language's
convention. The language appears once, in `tests/unit/`, because the suite
already says what the file is. Without a fixed mapping a shipped script is
tested wherever its author happened to look, and nobody can tell an untested
file from one whose test lives somewhere unexpected.

Most of what a role ships is a script against a live console -- it boots a
Laravel kernel, imports frappe, or writes through an ActiveRecord model -- so
the escape is a `nocheck: mirrored-unit-test` comment in the file, with a
rationale of its own. A file carrying both a test and an exemption fails: the
exemption outlived its reason and reads as "untested" to everyone after you.

Five scripts turned out to have real units behind the console wrapper, so they
get tests rather than markers (62 cases):

  * svc-net-tor/onion_self_reach.py -- pins that an HTTP error status, a closed
    connection and a refused connection all count as reached, because each
    proves the loopback fast path resolved; only a timeout or an unresolved
    name is a failure.
  * sys-lock/sys-lock.py -- `parse_time_to_seconds` unit precedence, and that a
    service still activating holds the lock (otherwise the gate races it).
  * svc-opt-keyboard-color/script.py -- colour interpolation and the
    zero-length transition window that would otherwise divide by zero.
  * sys-svc-compose/swarm/secret_rotate.py -- including the binary fallback a
    TLS key needs, and that a missing source leaves the name alone rather than
    pinning the hash of a file that is not there.
  * sys-front-inj-logout/logout.js -- pins that `href="/logout"` alone does not
    match and a non-interactive tag is never patched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:10 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00