Commit Graph
65 Commits
Author SHA1 Message Date
evangelostsak b777b65e84 test(mesh): hold the planning guarantees at any mesh count
The guarantees were asserted against the declared swarm and data pair,
so they proved two meshes work rather than that the planner works. A
deployment declares its own, and the review asks for any number of them with
the nodes arranged however the client wants.

The same invariants now run over generated topologies: one to eight meshes,
three arrangements each, hosts dealt out so some belong to several meshes and
some to none. 2247 subtests, four seconds, and the planner needed no change to
pass them.
2026-10-13 19:36:43 +02:00
evangelostsak 75fd3e9433 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-13 01:08:20 +02:00
kevinveenbirkenbachandClaude Opus 5.5 f7987679dd fix(ci): read every page of the job listing before reporting failed roles
report_failures asked `gh api --paginate` for the run's jobs with the jq
filter `[.jobs[] | {name, conclusion}]`. gh applies that filter to each page
and prints one array per page, so a run with more jobs than one page holds
produced two JSON documents, and json.loads stopped with "Extra data: line 2
column 1". Main run 38218402254 (113 jobs) failed its "Report main role
failures" job that way, and no issue was opened for its failed roles.

The listing now emits one job object per line and parses the lines one by
one, the same way logs/download/github.py lists jobs.

Verified: the old parse of the real output of run 38218402254 raises the
same "Extra data: line 2 column 1 (char 8491)" as CI; list_jobs on that run
returns all 113 jobs, 16 of them failed, grouped into 12 failed roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:46:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e31b9557d2 fix(env): keep the host .env when the runner container runs make setup
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.

compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.

test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.

Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
evangelostsak 34b49036d7 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-12 17:52:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ba796d7f1 fix(ci): pull the images the runner uses itself from the GHCR mirror
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.

iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.

Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:41 +02:00
evangelostsak ff1db88753 revert(swarm-test): remove the SSH-on-underlay measurement arm
The arm existed to answer one question: how much of the meshed arm's
cost is the connection plugin and how much is the tunnel. Runs 37592061960
(docker), 37753506661 (SSH on the underlay) and 37621556682 (SSH over
WireGuard) answered it on web-app-gitea#0, debian/ext4:

  pass 1  5964s / 8511s / 8832s
  pass 2  5319s / 7472s / 7705s

Leaving ansible_connection: docker costs +4700s, the tunnel +554s - 89.5%
against 10.5%. The answer is recorded; the scaffolding does not need to ship.

Reverts d792519725, 1b28f87b37, 89b8a8c8d1, 0470491e1a, 529c5ace43,
dd3d10a9a2, c20dd0e5f5, 458d8956e1, 63ade1e03b, d281555945 and 4adfb6e3d2.
95cfc8b7b7 is already superseded by bbfd82f89, which sets the profiling limit
upstream.
2026-10-09 12:12:03 +02:00
evangelostsak 4adfb6e3d8 test(mesh-cli): cover the addresses the transport dials
One test asserts ansible_host lands on the given address, the other
that a missing member raises. Either would have caught the UNREACHABLE run
before it was dispatched.
2026-10-08 10:58:23 +02:00
evangelostsak 25c7fdd1a5 test(mesh): pin the switched host to a tty-free transport
The carriage returns a pty adds are invisible until a role parses its own
output, so the absence of the tty is asserted where the transport is
written rather than left to the next role that reads stdout.
2026-10-03 23:20:26 +02:00
evangelostsak e456b1eee6 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.

Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
2026-10-03 02:29:45 +02:00
evangelostsak 152588b511 perf(mesh): retune the multiplexer for the hosts the mesh switches
The pass that runs over the mesh reaches every host by SSH where the
pass before it used the container connection, and it pays that difference
once per task. Measured in a container: an exec through a warm multiplexer
costs 7ms, a fresh connection 85ms, and 165ms once sudo asks for the
password every task demands. The pass runs thousands of tasks.

Two settings threw the multiplexer away. A persist window of 20s closes the
master whenever a host waits longer than that for its next task, which in a
linear play is most of them; and the named ControlPath points at a directory
nothing creates, so a master that cannot open its socket reconnects per task
without reporting it.

Neither is retuned for the repository. The tool that moves a host onto the
mesh now writes that host's own ssh_args, derived from the repository's
value so every other option is kept and only the window and the socket
differ. A host the mesh never touches keeps exactly what it had.

Confirmed against the built command: a switched host gets ControlPersist=600s
and a ControlPath under the directory Ansible creates, while a host beside it
keeps the repository's own.
2026-10-02 16:35:29 +02:00
evangelostsak dc14e8a3a8 fix(mesh): keep every member on the address it already holds
The offset came from the spoke's position in a sorted list, so a host that
sorts earlier renumbered every member behind it. `wg syncconf` carries keys
and peers onto a live interface and never its address, so a renumbered member
keeps answering on the old address while the peers and the inventory have
moved to the new one -- and the address it gave up can be handed to a new
member while the old owner still holds it.

The planner now reuses the addresses already on disk and allocates only an
unseen member, from the lowest free offset. A value outside the subnet, a
duplicate, and the hub's own offset are refused, so a mirrored host_vars
cannot hand a spoke the hub's address. Rotation keeps addresses: it replaces
identities, not addresses.

The routing assertions re-planned the mesh and compared the result with
itself, so a writer that stored the mesh subnet in place of the routed range
would have passed; they now read the entry off disk, which is what the role
reads.
2026-10-02 00:54:20 +02:00
evangelostsak 94f2401fab test(mesh): pin one key per host across a rotation
Asserts the hub's two mesh entries carry the same key and that its spoke holds
it. A first attempt matched only the first entry and passed against the bug.
2026-09-29 08:59:23 +02:00
evangelostsak 7634c1ed32 test(mesh): pin the controller's key across a rerun
Without the fix its public half changes on every write, which is what made
the hub re-render and re-key a tunnel nothing had asked to rotate.
2026-09-28 15:44:30 +02:00
evangelostsak ba4b8cd6f5 test(credentials): cover the kept subtree
The destination keeps its own block and still takes the rotated values around
it; a destination without that block takes the source's.
2026-09-28 11:46:01 +02:00
evangelostsak b16a2fb511 revert(provision): restore the user password generator tests
Follows the generator back to its previous behaviour.
2026-09-27 23:55:21 +02:00
evangelostsak fbf17949d7 test(mesh): cover the endpoint argument
A stated address replaces the name peers dial, an unstated host keeps its
name, and a malformed pair aborts rather than being ignored.
2026-09-27 02:05:00 +02:00
evangelostsak 7a996cd0f5 test(provision): cover the declared password reference
Three cases: the value is pinned from host_vars, its vault node survives, and
a reference to something absent still falls back to a fresh password.
2026-09-27 01:30:59 +02:00
evangelostsak f4c336b5a1 test(provision): pin the declared-password behaviour
Covers both halves: the policy reaches the caller with no value, and a stated
password is carried through and draws no random pin.
2026-09-26 23:30:07 +02:00
evangelostsak a868045d69 test(mesh): cover the controller in the foreign-entry prune
Mirrors the hub's host_vars onto the controller the way the credential reset
does, then asserts the rerun leaves only the mesh the controller belongs to.
Without the fix the file keeps the hub's data entry, is_hub included.
2026-09-26 19:47:10 +02:00
evangelostsak 3325a357c7 test(ci): carry the mesh state in the plan fixture 2026-09-26 09:48:20 +02:00
evangelostsak 5ae72d3b88 test(mesh): cover the controller spoke and the transport switch
Pins the two failure modes the design turns on: a host with no mesh address is
left reachable rather than stranded, and an address is only read from the host
that owns it, because the swarm reset mirrors one node's host_vars over every
other.
2026-09-26 01:27:07 +02:00
evangelostsak 0a6700e378 Merge origin/main into feature/svc-net-wireguard
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.

Seven conflicts:

- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
  in e706170a0 were modified upstream. The deletion stands, which is the whole
  premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
  legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
  and this branch's _write_mesh call after it.

The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
2026-09-24 21:14:25 +02:00
Alejandro Roman 9df3287adf Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-22 12:16:44 +04:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
evangelostsak 74bdfbae9c test(inventory): pin pruning, and assert mesh membership structurally
Adds the mirror case pruning has to survive and its inverse, that an owned mesh
is left untouched.

The existing assertions searched the file for 'swarm:' and 'data:', which the
credential keys mesh_private_key_swarm and mesh_private_key_data also match, so
they passed against a file whose mesh block was empty. They now read the mesh
names out of the parsed document.
2026-09-19 12:07:43 +02:00
evangelostsak 82c5d00c71 test(inventory): pin that a mirrored host_vars is reminted, not adopted
The failure this covers is silent in the inventory and total on the cluster:
every node adopts the hub's key and address, every interface claims 10.100.0.1,
and nothing handshakes. Asserts the mirrored hosts read as unkeyed and that a
rewrite gives the three swarm members three distinct addresses again.
2026-09-19 11:16:59 +02:00
kevinveenbirkenbachandClaude Opus 5 673de05a60 fix(inventory): write only the service toggles a role's contract declares
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:

    Missing default for web-svc-cdn: services.dashboard.shared
    Missing default for web-svc-mirror: services.dashboard.shared
    [ERROR] Inventory validation failed. Aborting deployment.

_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.

Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.

Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.

Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.

Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 06:00:56 +02:00
evangelostsak 160b0eaff7 fix(inventory): route the whole pool through the hub from a spoke
A spoke's AllowedIPs for the hub was its own mesh subnet, so a worker on
10.100.0.0/24 had no route to the NFS server on 10.100.1.11 and could not
reach the plane it is deliberately not a member of -- which is the entire
reason the hub sits in both.

The symptom would have been misleading: every tunnel reports a healthy
handshake, the gate passes because it only pings peers the host is configured
for, and the failure surfaces later as an NFS mount that hangs. It points at
storage rather than at routing.

Spokes now route the declared pool through the hub while the hub keeps each
spoke pinned to a single address; routing the pool in both directions would
blackhole every member behind whichever spoke came first.

Found by running the writer against the prefixed five-host topology before the
deploy rather than during it.
2026-09-18 16:37:18 +02:00
evangelostsak 0b8f8ac46f test(inventory): exercise the mesh CLI as the swarm routine invokes it
Covers the entry point rather than the library: one invocation writes every
member across both inventory files, a missing host_vars directory fails instead
of silently writing nothing, and a rerun is a no-op end to end.
2026-09-18 09:51:17 +02:00
evangelostsak 33dbd1ed8c test(inventory): pin what the mesh writer may put on disk
Asserted against real files in a tempdir, because the guarantees are about
files: which host_vars a secret may appear in, and whether a second run of an
unchanged mesh leaves them alone.

Two cases carry the security argument. A private key must reach exactly one
host_vars file, which is what keeps one host's inventory from impersonating
another, and it must never appear there in plaintext. A third covers the
inverse of the idempotence shortcut: a public key whose credential has gone
missing is worse than no key, because peers accept it while its owner cannot
use it, so that member is reported unkeyed and the pair is reminted together.

A worker is asserted NOT to receive the data mesh. Carrying it would peer the
worker directly with the NFS server and bypass the routing through the hub that
this topology depends on.
2026-09-18 09:29:35 +02:00
evangelostsak 10f0b94877 test(inventory): pin mesh planning, which fails silently when wrong
Every failure mode here is invisible at deploy time. A spoke missing from the
hub's peer list configures cleanly and never handshakes. A public key that does
not answer for its private half looks identical until traffic is attempted. A
partial rotation leaves the hub holding an old key while the spoke presents a
new one, and the handshake is refused with nothing logged on either side.

The load-bearing case is the hub filter: the manager is a member of
svc-swarm-node as well as the hub group, so left unfiltered it peers with
itself, which WireGuard accepts and which then routes the hub's own traffic
into its own tunnel.

The subnet guards are asserted against the shipped declaration too, so the
configuration proves its own collision checks rather than only the synthetic
ones.
2026-09-18 09:29:16 +02:00
evangelostsak c629fce2b7 test(inventory): add the mesh test package root
Resolves PROJECT_ROOT for the mesh suites, which read the shipped mesh
declaration rather than restating it.
2026-09-18 09:27:31 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 b356954d2e style(inventory): wrap the keycloak users fixture the way autoformat does
make quality-high pass 75 rewrote it: a concatenation whose right operand is a
multi-line literal gets parenthesised, with the operator leading its own line.

The string itself is unchanged, so
test_narrowing_the_credentials_still_rotates_every_user keeps the declaration
that separates web-app-keycloak from web-app-a and still fails when the user
half of the rotation is narrowed along with the credentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:50:27 +02:00
kevinveenbirkenbachandClaude Opus 5 46110f4311 perf(swarm-test): rotate the round's credentials, not every block in the file
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.

matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.

The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.

The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.

The CLI aborts when --include matches no block instead of rotating nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:32:02 +02:00
kevinveenbirkenbachandClaude Opus 5 da09ee575b fix(coredns): answer every record type for the test zone instead of forwarding it
The CI zone served A and AAAA for INFINITO_DOMAIN and let every other type fall through to the public resolvers, which return NXDOMAIN for the reserved .test TLD. Per RFC 2308 that denial covers the whole name, not the queried type, so any caching resolver in front of CoreDNS applies it to the A record the deployment depends on.

Without such a cache nobody noticed: Mastodon's EmailMxValidator asks MX, takes the NXDOMAIN, then asks A anyway and gets the address. On a Tor node svc-net-tor puts dnsmasq in front, the denial is cached, and tootctl accounts create ends on 'Failure/Error: email unreachable'. It cost Fediwall#0 and Fediwall#2 in run 34653542922 and Fediwall#2 in run 34653754175, all three compose plus tor; the same two jobs passed in run 34614940508 as compose without tor, same command, same administrator@infinito.test, rc 0.

A template with no answer returns NOERROR with an empty answer section and the SOA in authority, so the zone denies a type without denying the name. It sits after the A and AAAA templates, which would otherwise have their names answered NODATA, and before forward, so no type escapes.

Measured against 172.30.0.53 in the running stack: infinito.test MX went from NXDOMAIN with an authority record to NOERROR with zero answers and the aa flag, infinito.test A and auth.infinito.test A still return 172.30.0.10, and deb.debian.org still resolves through the forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 09:35:58 +02:00
kevinveenbirkenbachandClaude Opus 5 c32bf320ff feat(ci): add a fourth deploy chunk so a short priority chunk strands no budget
A sweep may deploy 210 rows (available) in chunks of 80. Three declared
blocks hold exactly that when there is no priority line, but priority
rows get a chunk of their own that is never topped up with regular rows.
Nine priority rows therefore occupied a whole block, left the regular
rows 2 x 80 = 160, and 42 rows of budget went unused.

Adding a block to the YAML alone would not have changed that:
matrix.chunks_of handed the split slots.chunk_count(), which is
min(ceil(available / chunk size), INFINITO_CI_MAX_CHUNKS) and ignores
the priority seam, so the plan still stopped at three blocks.

call-orchestrator.yml now declares test-deploy-chunk-3, gated on chunks
0 to 2 like every chunk gates on all before it, and report-main-failures
and done need it. INFINITO_CI_MAX_CHUNKS is 4, one above what the budget
fills, and slots.chunk_blocks() returns it. chunks_of hands the split
every declared block; available still caps the rows, and a chunk whose
slice comes back empty skips its deploy. chunk_count() stays the "chunks
filled" figure of the budget report.

Tests: the split spends 9 + 80 + 80 + 41 = 210 rows with a spare block;
chunks_of plans three chunks from chunk_blocks=3 while chunk_count=2
(the previous wiring yields [1, 10]); an integration guard requires the
declared chunk jobs to equal INFINITO_CI_MAX_CHUNKS and to exceed
ceil(available / chunk size) by one, and fails on the stale value 3.

A shell that inherited INFINITO_CI_MAX_CHUNKS=3 keeps failing that guard
until it is restarted; make dotenv-force rewrites .env.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:46:48 +02:00
kevinveenbirkenbachandClaude Opus 5 c7d162fa88 fix(coredns): derive the domain regex from INFINITO_DOMAIN when rendering
INFINITO_DOMAIN_RE carried an already escaped regex through the env
chain: EnvBuilder.setdefault prefers a value already in the process env,
quote_for_dotenv escapes every backslash, and load.sh unescapes it again.
One inherited value with a doubled backslash was enough to write
"infinito\\\\.test" into .env, the rendered Corefile then matched a
literal backslash, and getent hosts test.infinito.test failed with rc 2.

The renderer now substitutes DOMAIN_RE = re.escape(INFINITO_DOMAIN) at
render time, so no backslash travels through the env or .env at all. The
template placeholder is renamed because it is a derived render value, not
an env key, and INFINITO_DOMAIN_RE is removed from default.env and the
passthrough keys, leaving INFINITO_DOMAIN as the single source.

A renderer test renders the real template and requires a single-escaped
match with no doubled backslash and no leftover placeholder. Verified
live: the Corefile renders ^infinito\.test\.$, test.infinito.test
resolves to 172.30.0.10 and test.infinitoXtest does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 20:53:19 +02:00
kevinveenbirkenbachandClaude Opus 5 8434a31611 fix(ci): answer a role-only selection by existence, not by variant None
Rows are indexed by `(name, variant)` and a variant is an int. A token that
names only a role carries no variant, so the lookup ran with `None` and could
never hit, and every such token was reported as matching no discovered row.

That is the form `collapse_to_roles` produces, so it is the form a retriggered
priority line arrives in. Run 34348821577 shows the result: all twenty-one
priority entries carried

    priority: '<role>' matches no discovered row
              (no row in this run's mode and lifecycle envelope)

while baserow, checkmk, decidim and the rest went on to deploy fine. The
warning was not wrong about one entry and right about twenty; it was blind. A
role that genuinely has no row in the envelope looked exactly like a role with a
full set, which is how priority work goes missing without anyone being able to
see which entry did.

A token that narrows nothing beyond the role name is now answered by existence:
it asks for the role in whatever axes the rotation picks, so any discovered row
satisfies it. `Pin.pinned` already draws that line - "whether the token narrows
anything at all beyond the role name" - and pinned tokens keep the old path,
`check_pins` included.

Three tests come with it. There was already one for a bare role that matches
nothing, and none for a bare role that matches something, which is why the
lookup could stay broken. One of the new ones uses variant 4 so a future
implementation cannot pass by quietly guessing variant 0. Checked against HEAD:
the same input returns the spurious warning there, so the tests fail without
this change.

This restores the signal, not the lost jobs. The other silent path, `priority`
being truncated to the job budget in cli/meta/ci/chunks.py, is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:44:59 +02:00
kevinveenbirkenbachandClaude Opus 5 3d6b3900e0 refactor(cli): gather the documentation generators under cli/build/docs
Two of them lived inside docs/ next to the pages they render, which is why both had grown a __file__ upward walk to find the repository root, a sys.path insert to import from utils, and a nocheck for each. A generator parked among documents reads as documentation and drops out of the checks that apply to code.

Moving them surfaced six real violations that the old location had hidden: yaml.safe_load instead of the shared cache, two uncached read_text calls, five hard-coded role paths instead of ROLE_FILE_META_SERVICES and ROLE_FILE_META_MCP, two make targets without an alias, and a requirement reference in code. All six are fixed at the root, none with a suppression, and both generators lose their project-root exemptions because a package supplies PROJECT_ROOT.

cli/build/readme joins them so the three generators that write into docs/ sit together. A new lint keeps docs/ to .md, .html, .rst and .nocheck; it enumerates through git ls-files so untracked __pycache__ is not its business. Checked against HEAD, it names exactly the three .py files that moved.

026-unified-addon-syntax stated that the matrix generator stays under docs/requirements/. The lint makes that impossible, so the requirement now records where the generator actually lives.

Verified: every one of the twelve modules under the new tree imports, no old module path survives anywhere in the repository, 482 lint tests and 1246 cli unit tests pass, and both generators run without drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 23:17:55 +02:00
kevinveenbirkenbachandClaude Opus 5 e41be38cb7 perf(vault): encrypt in-process instead of spawning ansible-vault per value
VaultHandler.encrypt_string started one ansible-vault process for every single credential. The swarm matrix' rotation gate between PASS 1 and PASS 2 drives 113 application subprocesses over 855 credential keys, so the step cost 1078s in run 33923190220 while the green baseline never ran it at all. Measured per value: 226ms via the CLI, 3.3ms via VaultLib, so 855 keys drop from 193s to 2.8s.

The snippet layout is a contract: encrypt_leaves splits the returned lines and strips the body indent, and existing inventories were written by the CLI. Verified byte-compatible (same 'name: !vault |' header, same 10-space body indent, same $ANSIBLE_VAULT;1.1;AES256 marker), and ansible-vault decrypt reads back what the new path writes.

Reading the password file delegates to ansible's own FileVaultSecret rather than approximating it: ansible strips, then decrypts a vaulted password file, then rejects an empty secret. A hand-rolled .strip() covered only the first of those. _vault stays lazy because callers construct VaultHandler with a password file that need not exist yet.

The tests replace two subprocess.run mocks that asserted against invented output. They now pin the layout encrypt_leaves depends on, and pin the guarantee from 052cea02e2 (values starting with a dash), which landed without a test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 21:11:19 +02:00
kevinveenbirkenbachandClaude Opus 5 f1e86eef58 fix(ci-trigger): retire a green suite on any input
The retrigger turned a suite off only when the source run had been
dispatched with that suite explicitly on. b1ca39e9dd introduced that
guard together with the feature itself, and it contradicts the same
commit's message and docstring, both of which state that the job record
is the only evidence.

The guard made the feature a no-op in its most common case. An input
left on its default renders no title segment and resolves to empty, so
config.get('workspace') was never 'true' for a run that reached the
suite on the 'auto' default. call-orchestrator.yml reads 'auto' as
global scope off the whitelist alone, and a --failed retrigger puts its
selection on the priority line while leaving the whitelist empty, so
every retrigger in the chain redeployed a suite that had already gone
green. Measured on run 33785198875: six workspace jobs green, and the
dispatch still went out carrying workspace=auto.

Drop the guard. suite_passed already refuses to retire anything that
failed, was cancelled, timed out, was skipped or never ran at all, so
the job record decides on its own.

Three tests cover the paths 8600f1d9aa left open, all of which pinned
the explicit 'true': green on 'auto', green with no recorded input, and
red on 'auto' staying on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 04:06:04 +02:00
Alejandro Roman f9d3a1345f fix(stalwart): fix TOr and testing properly, local pass 2026-09-04 02:43:39 +04:00
kevinveenbirkenbach 8600f1d9aa test(ci-trigger): pin the workspace suite rule
The rule lived only in suite_passed's body, and it inverts easily on a
casual read: ask for the suite again when it failed. Under that reading
the retrigger aliases would drop the suite in their most common case,
because they force-cancel the source run before reading it, so a shard
that was still running comes back cancelled rather than red.

Only an all-green suite is retired. Failed, cancelled, skipped, timed
out, still running and never dispatched all keep it on the retrigger,
and each of those now has a test. The two docstrings enumerate them, and
so does the i8ciall description, which is where an operator reads it.
2026-09-03 17:37:33 +02:00
kevinveenbirkenbachandClaude Opus 5 a703e3fc80 fix(tests): one import form, two missed modules
65c72fc374 settled the convention: import unittest, and reach the submodule as import unittest.mock as mock, so a call site has one spelling of the name. test_onion.py and test_tor_socks_proxy.py still carried from unittest import mock; CodeQL reports them as py/import-and-import-from, the last two in the repository.

Same one-line change per file the sweep applied; mock.<attr> call sites untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 17:19:48 +02:00
kevinveenbirkenbach 1e78a02c34 fix(ci-trigger): resume where the last run left off
The regular line restarted at the head on every retrigger. resume_offset
compared the source run's deploy tokens against a freshly computed
ranking, but every axis rotates with the sweep number and a retrigger is
a new run with a new one: measured against the current 277-row ranking,
2 of 277 tokens matched. The walk stopped on its first row every time,
and the 66 rows past the job budget never deployed at all.

Membership now compares role#variant, the identity the rotation cannot
move, and a row counts as proven only when every job it had was green.
A priority row deploys its whole cross-product, so one role#variant can
carry several verdicts, and a green compose job must not carry a red
swarm one past the line. The offset also inherits the source run's own
offset as its floor rather than falling back to the head.

The priority line drops --include-unrun with it: only the hard failures
and the priority entries the run never deployed belong there. The rows
it never reached are the regular line's turn now that the offset walks.
2026-09-03 13:38:07 +02:00
kevinveenbirkenbachandClaude Fable 5.1 eab8f5fc90 fix(inventory): give pinned user passwords the mark the realm policy demands
The shell-safe default drew from letters and digits only, and every user without declared accounts is an identity account, so kcadm set-password rejected hermes and openclaw with invalidPasswordMinSpecialCharsMessage as soon as keycloak entered the GitLab#1 round without LDAP (4164f37981) and the extras file stopped replacing the users map (03e42bae6d; run 33689701895). The generator now draws from letters, digits and -_. and guarantees upper case, lower case, a digit and one mark, never in the leading position.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 02:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5.1 1a4ee72249 feat(ci): let a retrigger override chunk_gate, with i8ciallon as the non-stop i8ciall
A retrigger copies every dispatch input from its source run, chunk_gate
included, so a sweep that stopped at its first red chunk came back stopping
there again, and nothing short of a manual dispatch could ask the chain to
deploy every chunk and report all of it at once.

`infinito administration deploy ci trigger --chunk-gate {true,false}` now
overrides the carried value after the source run's configuration is read; it
is the one deliberate departure from "reproduce the source run exactly", and
the module docstring says so. Omitted, the carried value or the workflow
default applies as before.

`i8ciallon` is `i8ciall` with `--chunk-gate false`: force-cancel the run, put
everything without a verdict back on the priority line, and keep deploying
past a failed chunk. The alias lint caps a name at two adjacent consonants,
which is why it is not `i8ciallnonstop`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 00:12:56 +02:00