Commit Graph
828 Commits
Author SHA1 Message Date
evangelostsak 56cfdca4ff feat(credentials): let the mirror keep per-host material
The mirror copies one host's file over every other so a rotation reaches all
of them, which is right for a shared secret and wrong for anything a
generator writes per host: the copy hands every node the source's identity.

--mirror-keep names the subtrees that belong to the destination and are put
back after the copy. Absent, the mirror behaves exactly as before.
2026-09-28 11:45:34 +02:00
evangelostsak 68ae39a4c9 revert(provision): restore the user password generator
Out of scope for 037: the declared-reference pin was not what the swarm
transport needed, and the credential path should not change for it.
2026-09-27 23:55:10 +02:00
evangelostsak e98af2abc5 feat(mesh): take the underlay addresses on the command line
--endpoint HOST=ADDRESS, repeatable. A topology whose inventory names do not
resolve on every member states the address once, where the mesh is written,
rather than every member having to be taught the names.
2026-09-27 02:03:29 +02:00
evangelostsak fccd4d8382 feat(mesh): carry the endpoint into every peer entry
The rendered interface needs it; the member's own block does not, since a
host never dials itself.
2026-09-27 02:03:17 +02:00
evangelostsak b0697c08d6 feat(mesh): resolve each member's endpoint when planning
Defaults to the inventory name, which is what peers already dial wherever
those names resolve for every member.
2026-09-27 02:03:05 +02:00
evangelostsak 35572b8c06 feat(mesh): give a member the underlay address peers dial
The requirement's topology pins one per host and calls it the underlay
address; the mesh had no field for it and used the inventory name instead.
2026-09-27 02:02:54 +02:00
evangelostsak e154ad0b6f fix(provision): pin the value a declared password points at
meta/users.yml states 'password: {{ ansible_become_password }}' to say the
account shares the become secret. Provisioning pinned a fresh random over it,
so the account was created with one secret while become sent another; the
lookup returns the expression as a literal, so skipping the pin would hand
the account the text instead. Resolve the reference against host_vars and pin
that value, vault node included.

Any deploy that reaches a host as the administrator over ssh hits this, not
just the mesh: the container connection escalates to root without a password,
which is why nothing caught it.
2026-09-27 01:30:41 +02:00
evangelostsak 880de69687 fix(provision): keep the password a role states outright
A declared password is a dict when the role asks for a policy and a string
when it names the value. Only the dict case needs a generated secret, but
both took one, so user-administrator's '{{ ansible_become_password }}' was
overwritten by a random pin: the account was created with one secret while
become sent another.

Nothing caught it because the container connection escalates to root without
a password. The first transport that logs in as the administrator and sudos
for real gets 'Incorrect sudo password' on every host.
2026-09-26 23:29:54 +02:00
evangelostsak 81c32584e0 fix(mesh): prune the controller's foreign mesh entries too
The prune walks the hosts the inventory groups resolve to, but the controller
joins by name rather than through a group, so it was never visited. Its
mirrored host_vars kept the manager's data-mesh entry -- address 10.100.1.1,
is_hub true -- and the controller brought that interface up and enabled hub
forwarding for a mesh it is not a member of.
2026-09-26 19:28:13 +02:00
evangelostsak 189c91122c fix(ci): feed the mesh axis to the retrigger
entries_of requires it, so a retrigger would have aborted the same way the
plan did.
2026-09-26 09:48:20 +02:00
evangelostsak c839dd07ff fix(ci): accept the mesh axis in the plan renderer
The discovery step builds its table with plan.py, not matrix.py, so passing
--vpn aborted the chunk before a runner was spent on it. The plan now takes
the flag and shows the state as a column, the way it already shows the onion.
2026-09-26 09:48:09 +02:00
evangelostsak f0bd7b1bb0 feat(ci): accept --vpn when building the matrix 2026-09-26 08:31:11 +02:00
evangelostsak a170ccb4a3 feat(mesh): expose the controller through the writer CLI 2026-09-26 01:27:19 +02:00
evangelostsak 435d839ac5 feat(mesh): admit the Ansible controller as a spoke
Named rather than resolved from a group: the controller is not a swarm member
and has no role group, and inventing one would put a non-role name into
group_names where every service and placement lookup would special-case it.
2026-09-26 01:27:07 +02:00
evangelostsak c471940798 feat(mesh): move the Ansible transport onto the mesh
The first pass of a deploy reaches the hosts however it can, because no tunnel
exists yet and that pass is what creates one. Every pass after it connects
over the mesh, which is what makes the mesh load-bearing rather than merely
present: a broken tunnel then fails the deploy at the connection, and no role
can quietly fall back to the underlay it was supposed to stop using.

A separate step rather than a fact set during the play, because a play cannot
change the transport it is already running over. A host holding no address on
the mesh is left alone -- rewriting it would strand a host that was
reachable.
2026-09-26 01:26:53 +02:00
evangelostsak 978c0eeb28 refactor(mesh): one key per host, not one per mesh
A host presents the same identity on every plane it belongs to, so the hub no
longer carries two identities and the writer has one credential to keep in
step. A member already keyed by an earlier mesh keeps that key instead of
minting a second one.
2026-09-25 22:35:22 +02:00
evangelostsak e337e2f51f feat(mesh): follow the pool reference into its SPOT
The pool is declared once under NETWORK_* and referenced from the mesh file.
Ansible resolves that at play time; this writer runs before the play, so it
follows the one hop rather than duplicating the value.
2026-09-25 22:35:17 +02:00
evangelostsak 6628bc8eb0 fix(ci): let the failure reporter find a meshed run's artifacts
The slug it rebuilds gained the mesh state, so without it the reporter looked
for an artifact name no job uploads.
2026-09-25 22:34:16 +02:00
evangelostsak 0a6700e378 Merge origin/main into feature/svc-net-wireguard
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.

Seven conflicts:

- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
  in e706170a0 were modified upstream. The deletion stands, which is the whole
  premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
  legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
  and this branch's _write_mesh call after it.

The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
2026-09-24 21:14:25 +02:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
evangelostsak ae26d38247 fix(inventory): prune foreign meshes before planning
Runs first so a mirrored entry cannot be mistaken for this host's own, and
reports what it dropped, because a silently removed mesh is exactly the kind of
change that should be visible in a deploy log.
2026-09-19 12:07:33 +02:00
evangelostsak c4fd05ba82 fix(inventory): drop mesh entries a host does not own
Writing a member's own entry never removes another's. After the swarm reset
mirrors one node's host_vars onto every other, the NFS server keeps the hub's
swarm entry and brings up a second interface claiming 10.100.0.1. Its own data
mesh stays healthy, so nothing looks wrong on it -- but it now answers for the
hub's address and swallows the return path, and every worker's route to the
export dies there.

Observed as a worker with a live tunnel, a correct route and 100% packet loss
to the NFS server, while the hub reached the same address fine.
2026-09-19 12:07:24 +02:00
evangelostsak df23a74ffc fix(inventory): record which host a mesh entry belongs to
The swarm rotation gate mirrors one node's host_vars over every other, which is
correct for credentials that are identical per host and hands every node the
hub's identity here. Observed on a live cluster: a worker's swarm.conf was
byte-identical to the manager's, same private key, Address 10.100.0.1, a
ListenPort it should not have, and both spokes as peers. Nothing handshaked and
every NFS mount over the mesh failed.

Re-running the writer alone does not fix it, because a mirrored file presents
the hub's public key and credential and would be read as already keyed. The
entry now names its owner, so one that names a different host counts as unkeyed
and the pair is reminted.
2026-09-19 11:16:28 +02:00
evangelostsak c9d856b645 fix(inventory): make the mesh entry point executable
It carries the same shebang as every sibling __main__.py and none of their
executable bit, which ruff rejects as EXE001.

Not reproducible from here: the sandbox masks the mode, so ruff reports the
file as executable whatever it actually is, and a local run passes with the bit
cleared and a cold cache. CI is the only place this rule can fail.
2026-09-19 09:08:57 +02:00
kevinveenbirkenbachandClaude Opus 5 673de05a60 fix(inventory): write only the service toggles a role's contract declares
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:

    Missing default for web-svc-cdn: services.dashboard.shared
    Missing default for web-svc-mirror: services.dashboard.shared
    [ERROR] Inventory validation failed. Aborting deployment.

_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.

Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.

Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.

Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.

Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 06:00:56 +02:00
evangelostsak 160b0eaff7 fix(inventory): route the whole pool through the hub from a spoke
A spoke's AllowedIPs for the hub was its own mesh subnet, so a worker on
10.100.0.0/24 had no route to the NFS server on 10.100.1.11 and could not
reach the plane it is deliberately not a member of -- which is the entire
reason the hub sits in both.

The symptom would have been misleading: every tunnel reports a healthy
handshake, the gate passes because it only pings peers the host is configured
for, and the failure surfaces later as an NFS mount that hangs. It points at
storage rather than at routing.

Spokes now route the declared pool through the hub while the hub keeps each
spoke pinned to a single address; routing the pool in both directions would
blackhole every member behind whichever spoke came first.

Found by running the writer against the prefixed five-host topology before the
deploy rather than during it.
2026-09-18 16:37:18 +02:00
kevinveenbirkenbachandClaude Opus 5 2567c5ece1 feat(ci): let a manual run set the chunk size the formula would derive
chunk_size() multiplied the runner slots a chunk may assume by the waves the
queue window affords, and nothing could say otherwise. The derivation is the
right default -- too few rows leaves runners idle, too many push a chunk's tail
past the queue cancel -- but a run that wants smaller chunks had to edit
default.env to get them.

INFINITO_CI_CHUNK_SIZE now takes precedence when it is greater than zero, and
zero keeps the derivation. The input travels the same path offset already
travels: entry-manual-steer.yml offers the field and maps empty to 0,
call-orchestrator.yml hands it to all four chunk blocks, call-test-deploy.yml
exports it as the env var. It is a string rather than a number for the reason
resume_from_chunk already documents: a workflow_dispatch number input reaches
the caller as a string, and a number-typed workflow_call input then makes
GitHub refuse the job before it exists.

Two places in call-test-deploy.yml set it, not one. The plan summary resolves
the chunk size itself, so exporting it only for the app resolution would print
a plan that disagrees with the matrix the run then deploys.

Measured against the live helper: 0 gives 80 rows over 3 chunks, 25 gives 25
over 4, and returning to 0 restores 80 and 3.

The gate found three faults in the first attempt and all three are in this
commit: the key sorted after INFINITO_CI_CONCURRENCY rather than before it, the
dispatch description ran to 81 characters against a limit of 80, and the
retrigger fixture did not carry the new input. That last one matters beyond the
test: it is the check that would otherwise let a retrigger drop the chunk size
silently. carried_inputs() itself needed nothing, since it derives from the
declared inputs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:25:12 +02:00
evangelostsak dae11dd57e feat(inventory): add cross-host credential generation
Every credential generator in this package is single-host scoped.
credentials_generator shells out once per (app_id, host_vars_file) pair, and
that subprocess sees one host's inventory with no knowledge of any other. A
mesh credential breaks the assumption: the value one host receives is derived
from a secret minted for a different host, because a peer holds the public half
of a key whose private half lives elsewhere.

This package is the missing writer that sees the whole host set at once, so a
mesh is written complete or not at all. It is specified as cross-host
credential generation rather than as a WireGuard utility -- the application id
is a parameter and a mesh is a named hub-and-spoke grouping of inventory hosts,
so a later consumer needing paired secrets reuses it unmodified.

Planning is a pure function of the inventory, the spec and the public keys that
already exist, so mesh completeness, key pairing and rotation are provable
without a deploy. Keys are minted in-process with Curve25519 in the encoding
WireGuard expects, so the controller needs no wg binary to plan a mesh it never
joins, and a public key is always derived from its private half rather than
stored, because a stored pair can drift into a mesh that looks configured and
never handshakes.

Group membership is read across every inventory file it is spread over: the
swarm lab keeps its backup host in a sibling backup.yml, so a mesh spanning
both planes is only complete when both files have been read.

Vault encryption is salted, so a member that already holds a key is left
untouched rather than re-encrypted; otherwise every run would rewrite the file
and no deploy could be a no-op.
2026-09-18 09:06:52 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 f0ecf3e337 style(update): apply the formatting the gate produced
The nocheck markers were written as long single lines; ruff splits the
call across three lines and leaves the marker behind the closing paren.
Committing the formatter's own output keeps the tree and the history in
step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:51:25 +02:00
kevinveenbirkenbachandClaude Opus 5 94baeb7502 fix(lint): meet the repository rules the collection work broke
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.

File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.

restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 623ecc0248 feat(update): retune the preferred collection source from measured latency
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.

A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.

History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:59:45 +02:00
kevinveenbirkenbachandClaude Opus 5 e992175fcc feat(update): keep the pinned Ansible collections from ageing unnoticed
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.

Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.

The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:58:35 +02:00
kevinveenbirkenbachandClaude Opus 5 05fcec52e7 fix(planner): give every variant-carrying app the round's index, not only the include
round_variants was built by walking round_include. When a primary's variant switches its couplings off the planner resolves no prerequisites, round_include collapses to the primary, and no dependency gets an index at all. derive_includes then evaluates them at base config, where an unrendered Jinja coupling counts as enabled, so the closure inflates to everything reachable instead of what the variant declares.

Measured: svc-ai-litellm round 1 deploys 10 apps instead of 32, web-app-bigbluebutton round 1 and web-app-peertube round 1 drop from 34 to 10 and 11. Every other round of every role measured is unchanged, including all fourteen nextcloud rounds. The apps that leave are not dependencies of the round: svc-ai-litellm declares litellm, lmstudio, ollama and postgres and no sso at all, while bigbluebutton and peertube declare sso but switch it off in variant 1 together with dashboard, matomo, css, logout and prometheus.

complexity/cli.py:300 already documents the rule this makes true: the planner hands each pulled-in dependency round_index if round_index is below its variant count else 0, and pulls_at_zero implements exactly that. The coverage model was describing behaviour the planner only delivered for apps inside the include.

init.py filters the printed variant map to the apps actually shown, because round_variants now carries 266 entries and round 1 would otherwise print 112 of them on one line while ten apps deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 13:29:43 +02:00
kevinveenbirkenbachandClaude Opus 5 e74a585402 fix(guide): ask the reader for the clearnet resolver a Tor node needs
svc-net-tor aborts at roles/svc-net-tor/tasks/router.yml:2 when networks.internet.dns is empty, and rightly so: dnsmasq runs with no-resolv, so an empty upstream leaves the node answering .onion and nothing else. That value is set in exactly two places, both under inventories/development/, and the documented production path reaches neither - the provisioner writes no networks key at all. A reader following the role README verbatim therefore cannot deploy a Tor node.

CI has been proving this since 2026-08-31 without anyone seeing it: the README replay rides on the role's smallest variant and is skipped when the deploy is red, so the only row that ever reached it is one where svc-net-tor itself went green. That happened first on Ubuntu, where the local-service defect abb6f37297 fixes never fired.

The template now emits the DNS assignment, the -e passthrough and the networks entry in --vars for a role whose tasks name networks.internet.dns, which today is svc-net-tor alone; the generator derives that rather than taking a declaration, so a second role asking for the value documents it by doing so. scripts/github/guide/compose_deploy.sh substitutes the placeholder from INFINITO_DNS_IP, the same SPOT inventories/development/default.yml reads, next to the existing domain substitution.

This follows 972596a758, which added the administrator key to the same block for the same reason. No fallback inside the role: defaulting to NETWORK_PUBLIC_DNS_RESOLVERS would route a Tor node's clearnet queries to Cloudflare without the operator choosing it, which is what the assert exists to prevent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 02:17:26 +02:00
kevinveenbirkenbachandClaude Opus 5 46110f4311 perf(swarm-test): rotate the round's credentials, not every block in the file
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.

matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.

The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.

The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.

The CLI aborts when --include matches no block instead of rotating nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:32:02 +02:00
kevinveenbirkenbachandClaude Opus 5 c32bf320ff feat(ci): add a fourth deploy chunk so a short priority chunk strands no budget
A sweep may deploy 210 rows (available) in chunks of 80. Three declared
blocks hold exactly that when there is no priority line, but priority
rows get a chunk of their own that is never topped up with regular rows.
Nine priority rows therefore occupied a whole block, left the regular
rows 2 x 80 = 160, and 42 rows of budget went unused.

Adding a block to the YAML alone would not have changed that:
matrix.chunks_of handed the split slots.chunk_count(), which is
min(ceil(available / chunk size), INFINITO_CI_MAX_CHUNKS) and ignores
the priority seam, so the plan still stopped at three blocks.

call-orchestrator.yml now declares test-deploy-chunk-3, gated on chunks
0 to 2 like every chunk gates on all before it, and report-main-failures
and done need it. INFINITO_CI_MAX_CHUNKS is 4, one above what the budget
fills, and slots.chunk_blocks() returns it. chunks_of hands the split
every declared block; available still caps the rows, and a chunk whose
slice comes back empty skips its deploy. chunk_count() stays the "chunks
filled" figure of the budget report.

Tests: the split spends 9 + 80 + 80 + 41 = 210 rows with a spare block;
chunks_of plans three chunks from chunk_blocks=3 while chunk_count=2
(the previous wiring yields [1, 10]); an integration guard requires the
declared chunk jobs to equal INFINITO_CI_MAX_CHUNKS and to exceed
ceil(available / chunk size) by one, and fails on the stale value 3.

A shell that inherited INFINITO_CI_MAX_CHUNKS=3 keeps failing that guard
until it is restarted; make dotenv-force rewrites .env.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:46:48 +02:00
kevinveenbirkenbachandClaude Opus 5 c7d162fa88 fix(coredns): derive the domain regex from INFINITO_DOMAIN when rendering
INFINITO_DOMAIN_RE carried an already escaped regex through the env
chain: EnvBuilder.setdefault prefers a value already in the process env,
quote_for_dotenv escapes every backslash, and load.sh unescapes it again.
One inherited value with a doubled backslash was enough to write
"infinito\\\\.test" into .env, the rendered Corefile then matched a
literal backslash, and getent hosts test.infinito.test failed with rc 2.

The renderer now substitutes DOMAIN_RE = re.escape(INFINITO_DOMAIN) at
render time, so no backslash travels through the env or .env at all. The
template placeholder is renamed because it is a derived render value, not
an env key, and INFINITO_DOMAIN_RE is removed from default.env and the
passthrough keys, leaving INFINITO_DOMAIN as the single source.

A renderer test renders the real template and requires a single-escaped
match with no doubled backslash and no leftover placeholder. Verified
live: the Corefile renders ^infinito\.test\.$, test.infinito.test
resolves to 172.30.0.10 and test.infinitoXtest does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 20:53:19 +02:00
kevinveenbirkenbachandClaude Opus 5 8434a31611 fix(ci): answer a role-only selection by existence, not by variant None
Rows are indexed by `(name, variant)` and a variant is an int. A token that
names only a role carries no variant, so the lookup ran with `None` and could
never hit, and every such token was reported as matching no discovered row.

That is the form `collapse_to_roles` produces, so it is the form a retriggered
priority line arrives in. Run 34348821577 shows the result: all twenty-one
priority entries carried

    priority: '<role>' matches no discovered row
              (no row in this run's mode and lifecycle envelope)

while baserow, checkmk, decidim and the rest went on to deploy fine. The
warning was not wrong about one entry and right about twenty; it was blind. A
role that genuinely has no row in the envelope looked exactly like a role with a
full set, which is how priority work goes missing without anyone being able to
see which entry did.

A token that narrows nothing beyond the role name is now answered by existence:
it asks for the role in whatever axes the rotation picks, so any discovered row
satisfies it. `Pin.pinned` already draws that line - "whether the token narrows
anything at all beyond the role name" - and pinned tokens keep the old path,
`check_pins` included.

Three tests come with it. There was already one for a bare role that matches
nothing, and none for a bare role that matches something, which is why the
lookup could stay broken. One of the new ones uses variant 4 so a future
implementation cannot pass by quietly guessing variant 0. Checked against HEAD:
the same input returns the spurious warning there, so the tests fail without
this change.

This restores the signal, not the lost jobs. The other silent path, `priority`
being truncated to the job budget in cli/meta/ci/chunks.py, is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:44:59 +02:00
kevinveenbirkenbachandClaude Opus 5 5e6884b16d feat(update): keep pinned pip requirements from ageing unnoticed
The version cron updates docker image tags and repository refs, and tests/external/update covers docker, addons and repository. Neither sees a version inside an Ansible task variable, and dependabot reads pyproject.toml rather than roles/. A pin set once therefore froze and nothing thawed it: backup-docker-to-local has not moved since it was written, and the automtu pin one commit back would have gone the same way.

utils/update/pip mirrors the docker module: collect the pins, ask PyPI, rewrite in place. Only exact == pins are collected, because a range or a VCS reference states an intent an updater must not overrule. The external check warns rather than fails, like its docker sibling, so a slow PyPI cannot redden a run and a release the repository has deliberately not taken is not a regression.

The cron job is the docker job with one script swapped, so a changed pin arrives as its own PR under update/pip-pins.

Executed: the collector finds both pins in the repository; a synthetic file rewrites from 1.0.0 to 1.1.1 with the indent, quoting and neighbouring line untouched; automtu==1.0.0 would be reported while a bare name and a git+ reference are passed over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 01:44:51 +02:00
kevinveenbirkenbachandClaude Opus 5 3d6b3900e0 refactor(cli): gather the documentation generators under cli/build/docs
Two of them lived inside docs/ next to the pages they render, which is why both had grown a __file__ upward walk to find the repository root, a sys.path insert to import from utils, and a nocheck for each. A generator parked among documents reads as documentation and drops out of the checks that apply to code.

Moving them surfaced six real violations that the old location had hidden: yaml.safe_load instead of the shared cache, two uncached read_text calls, five hard-coded role paths instead of ROLE_FILE_META_SERVICES and ROLE_FILE_META_MCP, two make targets without an alias, and a requirement reference in code. All six are fixed at the root, none with a suppression, and both generators lose their project-root exemptions because a package supplies PROJECT_ROOT.

cli/build/readme joins them so the three generators that write into docs/ sit together. A new lint keeps docs/ to .md, .html, .rst and .nocheck; it enumerates through git ls-files so untracked __pycache__ is not its business. Checked against HEAD, it names exactly the three .py files that moved.

026-unified-addon-syntax stated that the matrix generator stays under docs/requirements/. The lint makes that impossible, so the requirement now records where the generator actually lives.

Verified: every one of the twelve modules under the new tree imports, no old module path survives anywhere in the repository, 482 lint tests and 1246 cli unit tests pass, and both generators run without drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 23:17:55 +02:00
kevinveenbirkenbachandClaude Opus 5 f1e86eef58 fix(ci-trigger): retire a green suite on any input
The retrigger turned a suite off only when the source run had been
dispatched with that suite explicitly on. b1ca39e9dd introduced that
guard together with the feature itself, and it contradicts the same
commit's message and docstring, both of which state that the job record
is the only evidence.

The guard made the feature a no-op in its most common case. An input
left on its default renders no title segment and resolves to empty, so
config.get('workspace') was never 'true' for a run that reached the
suite on the 'auto' default. call-orchestrator.yml reads 'auto' as
global scope off the whitelist alone, and a --failed retrigger puts its
selection on the priority line while leaving the whitelist empty, so
every retrigger in the chain redeployed a suite that had already gone
green. Measured on run 33785198875: six workspace jobs green, and the
dispatch still went out carrying workspace=auto.

Drop the guard. suite_passed already refuses to retire anything that
failed, was cancelled, timed out, was skipped or never ran at all, so
the job record decides on its own.

Three tests cover the paths 8600f1d9aa left open, all of which pinned
the explicit 'true': green on 'auto', green with no recorded input, and
red on 'auto' staying on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 04:06:04 +02:00
kevinveenbirkenbach 8600f1d9aa test(ci-trigger): pin the workspace suite rule
The rule lived only in suite_passed's body, and it inverts easily on a
casual read: ask for the suite again when it failed. Under that reading
the retrigger aliases would drop the suite in their most common case,
because they force-cancel the source run before reading it, so a shard
that was still running comes back cancelled rather than red.

Only an all-green suite is retired. Failed, cancelled, skipped, timed
out, still running and never dispatched all keep it on the retrigger,
and each of those now has a test. The two docstrings enumerate them, and
so does the i8ciall description, which is where an operator reads it.
2026-09-03 17:37:33 +02:00
kevinveenbirkenbachandClaude Opus 5 27ba2820f4 refactor: delete five private names nothing reads
Four module-level constants were defined and never read: _OPERATORS in the complexity filter, superseded by the op group of _TOKEN_RE two lines below; _VALID_SWARM_CONDITIONS, which swarm_restart_condition never checks because it maps to one of three literals by construction; _MISSING in services/config.py, where resolution already goes through get(strict=False, default={}); and _DEFAULT_FLAG_VALUES_THAT_ARE_NOT_NAMES in the exec-resolver lint, referenced by no rule in it. The nfs_fstype lookup rebound its variables parameter and returned without reading it.

Every name was private to its module and a repo-wide search over every file type finds no reader. CodeQL flagged them as unused; ruff cannot, because pyflakes has no rule for module globals and treats a rebound parameter as used.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 16:45:19 +02:00
kevinveenbirkenbachandClaude Opus 5 fe7a9d53ea fix(cli,utils): settle three CodeQL error findings
CodeQL reports cmd and sort_spec as possibly uninitialised after the except branch that calls parser.error, because it does not model the argparse error method as NoReturn. It is: usage is printed and the process exits with status 2. An explicit return 2 on that branch states the exit status the call already produces, so the analyser and a reader see the same path.

The retry loop in the registry client counted two attempts in a variable the body never read; the underscore prefix was the convention for that, but CodeQL only honours the bare name.

No path changes behaviour; the three alerts stop pointing at correct code so the scanner stays readable for the ones that are not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 16:40:13 +02:00
kevinveenbirkenbach 1e78a02c34 fix(ci-trigger): resume where the last run left off
The regular line restarted at the head on every retrigger. resume_offset
compared the source run's deploy tokens against a freshly computed
ranking, but every axis rotates with the sweep number and a retrigger is
a new run with a new one: measured against the current 277-row ranking,
2 of 277 tokens matched. The walk stopped on its first row every time,
and the 66 rows past the job budget never deployed at all.

Membership now compares role#variant, the identity the rotation cannot
move, and a row counts as proven only when every job it had was green.
A priority row deploys its whole cross-product, so one role#variant can
carry several verdicts, and a green compose job must not carry a red
swarm one past the line. The offset also inherits the source run's own
offset as its floor rather than falling back to the head.

The priority line drops --include-unrun with it: only the hard failures
and the priority entries the run never deployed belong there. The rows
it never reached are the regular line's turn now that the offset walks.
2026-09-03 13:38:07 +02:00
kevinveenbirkenbachandClaude Fable 5.1 eab8f5fc90 fix(inventory): give pinned user passwords the mark the realm policy demands
The shell-safe default drew from letters and digits only, and every user without declared accounts is an identity account, so kcadm set-password rejected hermes and openclaw with invalidPasswordMinSpecialCharsMessage as soon as keycloak entered the GitLab#1 round without LDAP (4164f37981) and the extras file stopped replacing the users map (03e42bae6d; run 33689701895). The generator now draws from letters, digits and -_. and guarantees upper case, lower case, a digit and one mark, never in the leading position.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 02:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5.1 1a4ee72249 feat(ci): let a retrigger override chunk_gate, with i8ciallon as the non-stop i8ciall
A retrigger copies every dispatch input from its source run, chunk_gate
included, so a sweep that stopped at its first red chunk came back stopping
there again, and nothing short of a manual dispatch could ask the chain to
deploy every chunk and report all of it at once.

`infinito administration deploy ci trigger --chunk-gate {true,false}` now
overrides the carried value after the source run's configuration is read; it
is the one deliberate departure from "reproduce the source run exactly", and
the module docstring says so. Omitted, the carried value or the workflow
default applies as before.

`i8ciallon` is `i8ciall` with `--chunk-gate false`: force-cancel the run, put
everything without a verdict back on the priority line, and keep deploying
past a failed chunk. The alias lint caps a name at two adjacent consonants,
which is why it is not `i8ciallnonstop`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 00:12:56 +02:00
kevinveenbirkenbachandClaude Opus 5 f19d224dd0 fix(ci-trigger): never resume at a row the priority line takes
A retrigger by role hands the priority line a bare role name so the rotation
can pick fresh axes, and the matrix then drops that role from the regular
query entirely. The resume offset was computed against a ranking discovered
without the priority line, so it could name one of exactly those rows, and
the run it was handed to had already guaranteed the row would not be there.

Chunk 0 of sweep 77 died on it before deploying anything:

    offset 'web-app-nextcloud#11' names no row of the regular line

and with it every other chunk of the run. The priority line of that run
promoted 27 roles whole, nextcloud among them; its regular line held no
nextcloud row at all.

Give resume_offset the priority pins and let it skip what they claim, by the
same two rules the matrix applies: a variant-less pin takes its whole role, a
pinned one takes those variants. The walk continues past a claimed row rather
than stopping, so a promoted role in the middle of the green window does not
truncate the resume point, and a window that is claimed end to end yields no
offset and starts at the head.

The ranking itself still discovers without the priority line, so an
unresolvable token there still cannot abort the helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 02:07:41 +02:00
kevinveenbirkenbachandClaude Opus 5 0b13fb73d0 style: apply make autoformat
Two leftovers of my own. The sweep that removed the reverse client entries
appended a newline where it cut a block, which at end of file leaves a blank
line after the last key in four services.yml. And the two cache-read nocheck
comments in the bond editor were written past the line limit, so the
formatter breaks their read_text calls across three lines and carries the
comment to the closing paren.

The wrap does not cost the suppression: test_no_uncached_file_reads matches
the statement, not the line the directive sits on, and passes on this form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:24:47 +02:00