Commit Graph
149 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 e31b9557d2 fix(env): keep the host .env when the runner container runs make setup
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.

compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.

test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.

Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
kevinveenbirkenbachandClaude Opus 5 9204f053dd fix(ci): keep the star on a priority row the sweep budget missed
cells() awarded the priority glyph only in the branch that has a chunk, so a
priority row beyond the budget rendered as a bare disabled mark,
indistinguishable from an ordinary row the sweep did not reach. The two need
different answers: --offset indexes the regular line only, so it never reaches a
missed priority row, which needs a shorter priority line or a run of its own.
The status now carries both glyphs and the module docstring says what to do
about it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-12 18:04:19 +02:00
evangelostsak 34b49036d7 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-12 17:52:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 25a1f08ab7 feat(design): leave the design queue by switching the design service off
The design queue skipped a hand-kept UI_LESS_ROLES list in
cli/meta/roles/design. The decision now lives in each role: a web-app- or
web-svc- role without a UI to design sets design.enabled: false in its
meta/services.yml, and design_disabled() reads that flag. web-svc-design,
the provider, stays out by its entity name.

A new lint, test_design_off_reason.py, requires a design entry in every
web-app- and web-svc- role and, where the entry is switched off, a
"# nocheck: design-off  Reason: <text>" on the design: line or the comment
lines directly above it. The rule is in the suppression catalog, and
requirement 037 decision 13 describes it.

Twelve roles switch design off with a reason checked against their code:
asset and simpleicons were already off; file, onlyoffice, bridgy-fed and
mig were on and no longer receive the shared CSS and design script; cdn,
collabora, coturn, legal, mirror and xmpp had no entry. The four that were
on drop their bond and their design variant overrides, which the
disabled-bond and variants-static-override lints reject on a literal false.
The queue lists 77 candidates; HEAD's list also counted bridgy-fed, mig,
collabora and onlyoffice. Cosmos diagrams and the integration matrix are
regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 17:09:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0327114c44 fix(inventory): drop disable= toggles once their key leaves the list
Provisioning merges into host_vars and never deletes, so the false
enabled/shared toggles that apply_services_disabled writes for a key stayed
in the round inventory after a later run dropped that key from disable=.
That covered the provider's own entry too: with seaweedfs once disabled,
web-app-seaweedfs kept shared: false, the service registry offered no
seaweedfs provider, container_networks attached no consumer, and BookWyrm
ran USE_S3=true against http://seaweedfs:8333 without a route.

reconcile_services_disabled_from_env records the keys of each disable run
in the host_vars key _INFINITO_SERVICES_DISABLED. On the next run it deletes,
in every application block, the enabled/shared toggles that are still
exactly false for a recorded key the current disable no longer lists, drops
a block left empty, and rewrites or removes the record. It runs right after
ensure_host_vars_file, before credentials and the variant bake, so a variant
that wants a toggle off writes it again. With an unchanged or empty disable
it does nothing. An inventory written before this change has no record, so
its older toggles are only reverted after one run with the old list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 11:07:21 +02:00
kevinveenbirkenbach 1fb98b5b76 Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-10 10:59:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ba796d7f1 fix(ci): pull the images the runner uses itself from the GHCR mirror
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.

iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.

Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:41 +02:00
kevinveenbirkenbachandClaude Opus 5 d3610af31f fix(disable): turn off a service key that only a variant bakes
`disable=lam` aborted the local deploy of web-app-openbao with "`disable`
conflicts with the current inventory state", and web-app-moodle carries the same
construction in two of its variants.

The writer and the conflict check disagreed on scope. apply_services_disabled
walked `find_roles_with_service`, the roles whose meta/services.yml declares the
key, so neither role was in that set for `lam` and nothing was written.
find_services_disabled_conflicts walks every deployed app's host_vars instead,
found the `lam` block the variant had baked, and refused the run. Dropping
`lam` from the list is not an answer either: the round then deploys web-app-lam
although ldap is off.

The writer now also flips a key it finds in host_vars under an app whose role
does not declare it, and only the flags that are already in that block. It never
creates a key or a flag, which is what `_declared_toggles` exists to prevent:
compare_application_keys rejects a host_vars key the role contract does not
carry. Overwriting a value the variant already wrote introduces nothing, so that
rule is untouched; test_skips_app_whose_role_does_not_define_service still
passes and pins it.

The alternative the TODO proposed, declaring `lam` in meta/services.yml with the
`'web-app-lam' in group_names` flags, is the worse trade: a consumer entry is
resolved through the service registry, so it would bind web-app-lam and its
whole closure into every round that carries openbao or moodle, for a key both
roles read only to gate a Playwright spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-09 22:48:06 +02:00
evangelostsak ff1db88753 revert(swarm-test): remove the SSH-on-underlay measurement arm
The arm existed to answer one question: how much of the meshed arm's
cost is the connection plugin and how much is the tunnel. Runs 37592061960
(docker), 37753506661 (SSH on the underlay) and 37621556682 (SSH over
WireGuard) answered it on web-app-gitea#0, debian/ext4:

  pass 1  5964s / 8511s / 8832s
  pass 2  5319s / 7472s / 7705s

Leaving ansible_connection: docker costs +4700s, the tunnel +554s - 89.5%
against 10.5%. The answer is recorded; the scaffolding does not need to ship.

Reverts d792519725, 1b28f87b37, 89b8a8c8d1, 0470491e1a, 529c5ace43,
dd3d10a9a2, c20dd0e5f5, 458d8956e1, 63ade1e03b, d281555945 and 4adfb6e3d2.
95cfc8b7b7 is already superseded by bbfd82f89, which sets the profiling limit
upstream.
2026-10-09 12:12:03 +02:00
evangelostsak 4adfb6e3d8 test(mesh-cli): cover the addresses the transport dials
One test asserts ansible_host lands on the given address, the other
that a missing member raises. Either would have caught the UNREACHABLE run
before it was dispatched.
2026-10-08 10:58:23 +02:00
kevinveenbirkenbach ebad39422c Merge branch 'feature/designer' into feature/store-i18n 2026-10-06 00:56:17 +02:00
kevinveenbirkenbachandClaude Opus 5.5 86cefeff26 feat(web-svc-design): derive one corporate design from a single base color
web-svc-css becomes web-svc-design and owns the configuration that lived in group_vars/all/13_design.yml. Consumers switch their css service key to design, and sys-front-inj-css becomes sys-front-inj-design. javascript stays an independent key; a role's optional design.js.j2 is injected together with the design and CSP-hashed like the functional script.

An in-repo OKLCH lookup replaces colorscheme_generator. From design.colors.base alone it derives surfaces, text, borders, the primary and status fills with their on-colors, hover and active states and a dedicated dark palette; a unit test proves WCAG AA, and AAA for body text, across six base colors in both modes. Semantic --design-* tokens replace --color-NN-SS in every role stylesheet. The shared element styles sit in the cascade layer infinito-design, so an app's own design system always wins and takes the palette through a variable mapping in its role stylesheet or through its own theming mechanism.

design.logo and design.title resolve globally from web-svc-design or per role from its design entry, and false or 0 disables the replacement. The design_assets action renders the logo lockup per slot as SVG, PNG and ICO, and the legacy title keys move into the design entries.

Three roles carry the design end to end. The dashboard takes logo, favicon and title from the generator and replaces the hard-coded port-ui colors. Gitea maps its theme variables onto the tokens and mounts the generated logo and a branded start page. Nextcloud receives primary color, logo, name and slogan through occ theming and ships no stylesheet.

make design-sync re-renders the static design files of a deployed app, make design-gallery captures before and after screenshots in light, dark, desktop and mobile, and make design-queue orders the roles that are due for a design pass. The i8-designer skill and docs/agents/action/design.md describe that loop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 06:04:14 +02:00
kevinveenbirkenbachandClaude Opus 5 e2948332fd fix(merge): repair what the main merge left broken
The merge brought in main's new roles and carried this branch's cross-cutting
renames and lints past them, so files that never conflicted were left
inconsistent. make quality-high is green again: 4 test targets, 15 lint targets.

a05071fe78 renamed get_entity_name to entity_name. Three files main added still
used the old name and merged cleanly because this branch never touched them:
plugins/lookup/email.py raised ImportError on import, which also took down
sys-ctl-hlth-msmtp's mail_recipient lookup; web-app-stalwart/tasks/00_core.yml
applied `| get_entity_name`, a filter that does not exist (the registration in
plugins/filter/get/entity_name.py is `entity_name`), so a compose deploy of
stalwart would have failed while rendering sys-stk-front-proxy's vars; and
test_database.py patched a name the module no longer binds.

web-app-stalwart and web-app-mailu spell DEPLOYMENT_MODE out in 15 places. The
named booleans in group_vars/all/18_swarm.yml are the SPOT and the bare form is
the idiom: 177 existing uses, and web-app-listmonk/tasks/utils/mcp.yml:42,49
drives both as mutually exclusive `when:` conditions, which a truthy string
would break.

svc-net-tor-smtp pinned aiosmtpd and PySocks inside its Dockerfile, where
Dependabot cannot see them. The pins move to files/requirements.txt; since
sys-svc-compose stages only the Dockerfile, 00_core.yml now stages the
requirements file into the build context too.

stalwart's webui_version had no upstream declared. Its three inline
`environment:` blocks stay, with a nocheck each: the role sets no
docker_compose_env, so one templates/env.j2 would also be the env_file of the
sibling containers, and clamav would receive the recovery admin credential and
the OIDC client secret.

The 13 bindings in stalwart's playwright env.js defaulted to "" and were
consumed unconditionally. Every key is emitted unconditionally by
playwright.env.j2 from a lookup that fails rather than yields empty, and
OIDC_ISSUER_URL always concatenates the realm path, so requireDotenvValue is
the correct binding. gitlab's adminUsername gets the toBeTruthy guard its two
siblings already had.

web-svc-api and svc-cache-package collided with stalwart on local.http 8094 and
8095 and move to 8098 and 8099, which exhausts the 8001-8099 band.
svc-ai-agent-broker collided with stalwart on 192.168.223.0/24 and moves to
192.168.81.0/24; the suggester only subdivides blocks already in use at the
requested prefix, so the move was validated with --block.

web-app-weblate calls lookup('email') without emitting
lookup('container_extra_hosts'), so on an onion deploy its SMTP client drops
every message without an error.

test_main.py read the primary domain from a literal while main moved
INFINITO_DOMAIN to main.infinito.test. It now takes the value from
default_domain_primary(), in the shape e2beab9f1c uses on
feature/web-app-wazuh/add-role, so that branch merges here without a conflict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 03:21:09 +02:00
kevinveenbirkenbachandClaude Opus 5 f2fedfc677 Merge branch 'main' into feature/store-i18n
Fourteen real conflicts plus the 66 generated READMEs.

utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.

plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.

roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.

group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.

docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.

inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.

roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.

The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 01:57:17 +02:00
evangelostsak 25c7fdd1a5 test(mesh): pin the switched host to a tty-free transport
The carriage returns a pty adds are invisible until a role parses its own
output, so the absence of the tty is asserted where the transport is
written rather than left to the next role that reads stdout.
2026-10-03 23:20:26 +02:00
kevinveenbirkenbachandClaude Opus 5 0d2fadce0c feat(cache): serve every checkout's own cache declarations
A checkout that declared a new cache host rendered the artefacts and reached nothing: the Nexus proxy repository, the frontend's leaf certificate and the nginx upstream map all live in the running stack, which only the owning checkout set up. cache-apply installs all three and reloads the frontend, and compose up runs it for a checkout that merely joins the network, so the manual call is left for a declaration changed between two bring-ups.

The map source is no longer selectable. INFINITO_CACHE_CONF_SOURCE existed to let a worktree read the primary checkout's map instead of its own, but the frontend mounts one map and the install replaces it, so the checkout that applied last wins either way: the knob chose which file to point at, never who prevailed. Dropping it also retires the git subprocess that resolved the owning checkout on every env build.

Two checkouts sharing a stack therefore overwrite each other's map, which the documentation now states instead of implying a selector could prevent it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-03 21:40:34 +02:00
evangelostsak e456b1eee6 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.

Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
2026-10-03 02:29:45 +02:00
evangelostsak 152588b511 perf(mesh): retune the multiplexer for the hosts the mesh switches
The pass that runs over the mesh reaches every host by SSH where the
pass before it used the container connection, and it pays that difference
once per task. Measured in a container: an exec through a warm multiplexer
costs 7ms, a fresh connection 85ms, and 165ms once sudo asks for the
password every task demands. The pass runs thousands of tasks.

Two settings threw the multiplexer away. A persist window of 20s closes the
master whenever a host waits longer than that for its next task, which in a
linear play is most of them; and the named ControlPath points at a directory
nothing creates, so a master that cannot open its socket reconnects per task
without reporting it.

Neither is retuned for the repository. The tool that moves a host onto the
mesh now writes that host's own ssh_args, derived from the repository's
value so every other option is kept and only the window and the socket
differ. A host the mesh never touches keeps exactly what it had.

Confirmed against the built command: a switched host gets ControlPersist=600s
and a ControlPath under the directory Ansible creates, while a host beside it
keeps the repository's own.
2026-10-02 16:35:29 +02:00
evangelostsak dc14e8a3a8 fix(mesh): keep every member on the address it already holds
The offset came from the spoke's position in a sorted list, so a host that
sorts earlier renumbered every member behind it. `wg syncconf` carries keys
and peers onto a live interface and never its address, so a renumbered member
keeps answering on the old address while the peers and the inventory have
moved to the new one -- and the address it gave up can be handed to a new
member while the old owner still holds it.

The planner now reuses the addresses already on disk and allocates only an
unseen member, from the lowest free offset. A value outside the subnet, a
duplicate, and the hub's own offset are refused, so a mirrored host_vars
cannot hand a spoke the hub's address. Rotation keeps addresses: it replaces
identities, not addresses.

The routing assertions re-planned the mesh and compared the result with
itself, so a writer that stored the mesh subnet in place of the routed range
would have passed; they now read the entry off disk, which is what the role
reads.
2026-10-02 00:54:20 +02:00
kevinveenbirkenbachandClaude Opus 5 d88fe1d83a test(lint): let the legacy-path guard see the trees that still named the old layout
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.

So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.

`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.

Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.

`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.

Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.

The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 21:27:40 +02:00
kevinveenbirkenbachandClaude Opus 5 058e76e58c fix(test): read the two planner call sites through the shared cache
`test_no_uncached_file_reads` requires every read under tests/, utils/, cli/,
plugins/ and library/ to go through `utils.cache.files.read_text`, which keeps
one decoded read per file for the whole pytest session. The test added in
3b85e4e1e1 called `Path.read_text` directly and so read init.py and
deploy/cli.py past that cache.

The rule allows `# nocheck: cache-read` only for a read outside the project
tree, which is how bfdf3dee79 handled a tempdir. These are real source files,
so the reader is the answer and the marker would have been the convenient
wrong one; e5430ad4ca is the matching precedent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 02:34:39 +02:00
kevinveenbirkenbachandClaude Opus 5 3b85e4e1e1 fix(deploy): let init pin the guide host the way deploy already does
adb53392bd taught the deploy planner to pin the instructions host to its
smallest variant and to leave a pinned app out of the round total. It taught
only the deploy side. `init` writes the inventory and `deploy` reads it back,
and the directory carries a round suffix only when the plan holds more than
one round, so the two sides named different directories:

    >>> Inventory initialized at /home/runner/inventories/github-0
    === matrix-deploy: round 1/1 inv=/home/runner/inventories/github
    [ERROR] Some application_ids are invalid for this inventory
    [DEBUG] inventory file does not exist

Nine jobs of run 36596159856 died there, before ansible-playbook started. The
trigger is a subject with a single variant: init counted the host's two
variants and suffixed, deploy pinned the host out and did not. svc-ai-s1
declares two variants of its own and was the only instructions row to survive,
which is the shape of the bug read backwards.

`variant_select` opens by stating the invariant this broke - "keeps init and
deploy walking the exact same selection (SPOT)" - so the fix is to honour it
rather than to special-case the suffix.

The accompanying test asserts both entry points forward `pinned_variants`.
The five tests that shipped with adb53392bd all exercised the pin helper on
its own, which is why nothing caught a caller that never asked it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 01:40:55 +02:00
kevinveenbirkenbachandClaude Opus 5 adb53392bd fix(ci): deploy the instructions host in the shape its README documents
A row marked for the instructions replay gets `web-app-docs` appended to its
apps, and the planner hands every primary app the round's index. The host
therefore took the subject's index, which for a passenger is an arbitrary
shape. Measured in run 36558500327: the same appended role deployed
dashboard, matomo, prometheus and css in 141 tasks on a round-0 row and in
none at all on a round-1 row, purely because of whose round it was riding.

The README's Production block describes the bare role, so the host now takes
its smallest variant in every round. The index comes from `smallest_variant`
rather than a literal, so it follows `meta/variants.yml` instead of drifting
from it.

The row names the host in an environment variable. `--apps app#variant` would
have read better at the call site but needs a new matrix column in both
`axes.py` and `cli/meta/ci/plan.py`, while the deploy CLI already reads
`disable` from the environment and the four shell layers between the workflow
and the CLI pass it through untouched.

A pinned app no longer counts toward the round total: a passenger should not
lengthen the subject's sweep. That removes an unreachable round rather than
coverage, because `deployed_rounds` plans one app at a time and a role with
one variant therefore only ever yields `variant: 0` rows, which the workflow
pins on every deploy.

The host is left alone when it is also the subject. `web-app-docs` carries the
replay on its own rows too, where the apps list reads `web-app-docs,
web-app-docs` and the round index is the entire point of the sweep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 18:00:45 +02:00
kevinveenbirkenbachandClaude Opus 5 16c853545a fix(deploy): stop the dedicated deploy from git-cleaning its own checkout
`MODE_CLEANUP` says what to tidy on the deployed system: prune unused Docker
resources, clear Mastodon caches, switch an LDAP user list from `present` to
`exact`. Eight roles read it that way. The dedicated runner read the same flag
and ran `make clean` in the control repository instead, which is
`git clean -fdX` over every ignored path there.

`inventories/production/` is one of those paths. The published host install
guide provisions into it and then deploys against `$INVENTORY/.password`, so
the deploy deleted the vault password it was about to use:

    Removing inventories/production/
    [ERROR] The vault password file .../inventories/production/.password was
    not found

Reproduced in CI on two unrelated rows, arm64/zfs/debian and
amd64/btrfs/centos, so it is neither architecture nor filesystem specific.

The coupling was a name collision, not a decision. 7e5990aa16 generated the
MODE_* flags from `01_modes.yml` and removed the legacy skip flags, and the
repository clean - which had had a flag of its own - was attached to the
generated `MODE_CLEANUP` because one now existed by that name. No other deploy
path does this; `container`, `development` and `swarm` never call `make` at
all.

`make clean` keeps its own entry points, `make setup-clean`, `make clean-sudo`
and `infinito --git-clean`, where a caller asks for it deliberately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 17:24:05 +02:00
kevinveenbirkenbachandClaude Opus 5 62a8f1ce2a feat(ci): retrigger a failed role whole and an unfinished one by variant
--roles-only collapsed the entire failed set, so a job that never reached a verdict came back as a bare role name and the rotation re-picked axes the run had already proved green, dropping the one combination it never got to. Nothing distinguished a verdict from an interruption.

collapse_failures() takes the failed set and the hard-failure subset: a role that failed outright collapses to its name, everything else keeps the exact role#variant@mode+tor it was given. A role with a hard failure absorbs its own unfinished rows, since the collapsed name already covers every variant of it.

With --strict the two arguments are the same list, so i8ciall, i8ciallon and i8cifo are byte-identical to before; a test asserts that agreement rather than leaving it to inspection. The new combination is exposed as i8civar, which is i8ciallon without --strict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 13:33:41 +02:00
evangelostsak 94f2401fab test(mesh): pin one key per host across a rotation
Asserts the hub's two mesh entries carry the same key and that its spoke holds
it. A first attempt matched only the first entry and passed against the bug.
2026-09-29 08:59:23 +02:00
kevinveenbirkenbachandClaude Opus 5 c782208f99 feat(tools): rename the i18n lane to tools and give the cache paths one spot
The lane is no longer LibreTranslate-only, so `i18n` is the wrong name for
the runner, its compose override, its inventory and its env keys. Renames
to `tools` throughout and points the lane at
`svc-ai-libretranslate-engine`, which is the role that actually owns the
`libretranslate-engine` service key.

The six cache host paths were each written out in full. They now derive
from `DIR_CACHE`, one plain string in `group_vars/all/05_paths.yml`, via a
`cache_paths` handler; the generated values are byte-identical. The handler
is registered directly after `cache_stack` rather than late, because
`registry_cache_max_size` reads `INFINITO_CACHE_REGISTRY_HOST_PATH` and
returns early when it is empty, which silently dropped
`INFINITO_CACHE_REGISTRY_MAX_SIZE` from the generated `.env` and broke
every `docker compose` invocation that layers `cache.override.yml`.

`INFINITO_TOOLS_MODELS_HOST_PATH` derives from the same base, so the model
bind the lane mounts follows the cache layout instead of repeating it.

The inventory pins `DOCKER_BIND_HOST=0.0.0.0`: the group_vars default keys
off `DOCKER_IN_CONTAINER`, which evaluates false for the inner deploy and
would leave the lane's services on the runner's loopback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 23:50:26 +02:00
evangelostsak 7634c1ed32 test(mesh): pin the controller's key across a rerun
Without the fix its public half changes on every write, which is what made
the hub re-render and re-key a tunnel nothing had asked to rotate.
2026-09-28 15:44:30 +02:00
evangelostsak ba4b8cd6f5 test(credentials): cover the kept subtree
The destination keeps its own block and still takes the rotated values around
it; a destination without that block takes the source's.
2026-09-28 11:46:01 +02:00
kevinveenbirkenbachandClaude Opus 5 a37039f0cb fix(web-app-docs): name the replayed role instead of taking the first one
The instructions test brings up a machine per role, so it can only ever
replay one guide. Which one it picked was a side effect of ordering: the
lookup returned the first invokable peer the round listed, and on a CI row
that is the role under test only because the row's apps happen to read
<role>,web-app-docs.

It now reads APP_ID, which the deploy workflow already sets to the row's app
and which the swarm routine already documents as the app under test. A round
that carries no APP_ID, or one whose app publishes no Production block, draws
an invokable peer at random instead, so a local round covers a different
guide each time rather than always the first one the inventory lists.

Ansible runs inside the infinito container rather than on the runner, and
that exec passes an explicit allowlist, so the name arrives only where it is
forwarded. Two seams carry it: _FORWARDED_ENV_KEYS, which covers CI and the
local initialize and reinstall verbs, and the update verb's own docker exec,
which assembles its arguments without that CLI.

The pin is tolerant rather than strict. APP_ID is set on every deploy row and
not only on documented ones, so refusing an app that publishes no guide would
let a generic variable abort a deploy.

Measured: the draw is observed over 200 rounds rather than assumed, and the
forwarding is negative-controlled, removing the key and watching the new test
fail on that key alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 09:16:29 +02:00
evangelostsak b16a2fb511 revert(provision): restore the user password generator tests
Follows the generator back to its previous behaviour.
2026-09-27 23:55:21 +02:00
kevinveenbirkenbachandClaude Opus 5 666312dfbd refactor(i18n): translate against the dedicated LibreTranslate alone
server() preferred the deployed service and fell back to a throwaway
container when it did not answer. That container ran the upstream image
directly, with LT_UPDATE_MODELS on and neither the package-cache CA nor
the argos-net.com hijack, so it refetched gigabytes from upstream on
every run while the deployed service already carries those models in its
image, installed through the cache at build time. The fallback is gone.

It was reached by accident rather than by failure. deploy() ended in
await_service(), which polled for 300 seconds; the first start of the
service installs its models and takes far longer, so the wait expired,
deploy() reported nothing running, and the caller spawned the throwaway
beside the service it had just deployed successfully. await_service and
SERVICE_TIMEOUT_SECONDS are removed, leaving the single readiness wait
the client already performs with the budget those models need.

Step 0 of the deploy also stopped forcing a recreate. It rebuilt a
healthy lane on every invocation and took whatever model install was in
flight with it; compose recreates on a changed configuration by itself.

The wait no longer looks like a hang: it reports how many of the wanted
languages the server offers, which ones are still missing, how long it
has waited against its budget, and what the lane has pulled. i18n-tune
waits the same way and gets the same report.

service_url() and lane_container() replace the two places that each
rebuilt one of those strings, and deploy() reads the environment once.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 10:56:42 +02:00
kevinveenbirkenbachandClaude Opus 5 44473f7de1 refactor(i18n): move the dev files into their conventional homes
The top-level i18n/ directory held a compose override and a development
inventory and nothing else, so neither sat with its siblings. The override
joins compose/, whose entries are already named <name>.override.yml, and
the inventory joins inventories/development/, which is what it is. The
directory is gone.

Both moves are pure renames; only the four sites naming the old paths
change, and the -f argument in compose_file_args is the one that decides
which override every docker compose call in the i18n lane layers on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 10:37:18 +02:00
evangelostsak fbf17949d7 test(mesh): cover the endpoint argument
A stated address replaces the name peers dial, an unstated host keeps its
name, and a malformed pair aborts rather than being ignored.
2026-09-27 02:05:00 +02:00
evangelostsak 7a996cd0f5 test(provision): cover the declared password reference
Three cases: the value is pinned from host_vars, its vault node survives, and
a reference to something absent still falls back to a fresh password.
2026-09-27 01:30:59 +02:00
evangelostsak f4c336b5a1 test(provision): pin the declared-password behaviour
Covers both halves: the policy reaches the caller with no value, and a stated
password is carried through and draws no random pin.
2026-09-26 23:30:07 +02:00
kevinveenbirkenbachandClaude Opus 5 50091194d5 fix(deploy): keep a non-invokable prerequisite out of the round's app list
Every desktop job of run 36246124624 aborted before ansible started:

    [ERROR] Some application_ids are invalid for this inventory:
      - user-workstation: not present in inventory

The include list the planner returns becomes two things at once: the
inventory provisioner's --include, and the deploy ids validate_application_ids
checks against the resulting inventory. The inventory is built from
get_all_invokable_apps, and utils/roles/stage.py loops a stage only over
invokable category paths, so a non-invokable id in that list is one no group
can hold and no stage can dispatch. `user-workstation` landed there because
the desktop-base service edge pulls it into the closure.

Measured over all 179 invokable roles, old resolution against new:

    rounds losing a member: 33
    distinct dropped roles: ['dev-java', 'user-workstation']

`user-workstation` in all 33 dsk rounds, `dev-java` in dsk-bluray-player,
which carries it as a meta dependency and would have aborted the same way
on main the day a round picked that role.

Neither stops running. A meta dependency resolves at role-include time,
independent of the inventory. `user-workstation` reaches its host through
the service loader, whose preload closure is computed from the filtered
whitelist and still contains it, and through dsk-base's own include_role:

    dsk-firefox: include=['dsk-base', 'dsk-firefox']
                 user-workstation in preload closure=True
                 workstation service required=True

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 23:06:58 +02:00
evangelostsak a868045d69 test(mesh): cover the controller in the foreign-entry prune
Mirrors the hub's host_vars onto the controller the way the credential reset
does, then asserts the rerun leaves only the mesh the controller belongs to.
Without the fix the file keeps the hub's data entry, is_hub included.
2026-09-26 19:47:10 +02:00
evangelostsak 3325a357c7 test(ci): carry the mesh state in the plan fixture 2026-09-26 09:48:20 +02:00
kevinveenbirkenbachandClaude Opus 5 c39f3ec9ee feat(web-app-docs): replay the published instructions in its CLI test
The replay of a role's "### Production" block moves out of the CI guide
scripts and into the documentation role itself. The block is read off the
rendered page over HTTP, so what runs is what a reader copies, and the role
that publishes the page is also the role that proves it.

scripts/github/guide/{one,compose_deploy,host_deploy}.sh and
cli/build/docs/readme/production.py are gone with the workflow step that
called them. The instructions marker now only adds web-app-docs to the row's
apps; test-e2e-cli discovers the role through its new templates/test.env.j2
and runs files/test/test.sh.

The block runs in a throwaway machine, never on the stack host. compose.yml
starts one service built from files/test/Dockerfile, whose base is
INFINITO_PARENT_IMAGE out of the mounted checkout's .env, so the machine is
the stack host's own distribution and the replay exercises the installation
instead of a prebuilt image. files/test/init.sh adds an init when the base
carries none and keeps the full -Syu that ef0f03ea98 established for arch.
scripts/tests/workspace/base/01_install.sh then prepares the machine before
the block runs.

Nothing is skipped any more. The previous host-install branch exited 0 and
passed silently over 65 of the 178 invokable roles; a block without a
"docker run" is now replayed against systemd in that same machine, with the
git clone dropped in favour of the mounted tree under test.

The target role comes from the new guide_role lookup, which reads
list_invokable_app_ids() rather than matching role-name prefixes, and falls
back to the asking role when the round deploys nothing else. deployment.py
grows running_apps() so both lookups share one definition of what the round
deploys. required_by_coverage.role_is_invokable drops its own walk over
categories.yml and delegates to the same SPOT; measured equal across all 280
roles.

Carried in the same index, because the refactor pass ran over this change
set: library.py splits into sites.py and build_queue.py mixins, builder.py
hands translated_languages to catalogs.py and folds three repeated log-tail
and state writes, and placeholders.py becomes a facade over spans.py,
repairs.py and damage.py. No test changed for those splits, and the two
ref-resolution dicts builder.py carried are replaced by _wanted_ref, which
also stops hashing the deployed tree on every tag build.

Verified: 7132 unit and lint tests pass over the whole tree, the guide_role
lookup renders through a real Templar, the base image resolves from .env,
and compose config validates the service. The replay itself has never been
executed; that needs a deploy of web-app-docs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 04:39:48 +02:00
evangelostsak 5ae72d3b88 test(mesh): cover the controller spoke and the transport switch
Pins the two failure modes the design turns on: a host with no mesh address is
left reachable rather than stranded, and an address is only read from the host
that owns it, because the swarm reset mirrors one node's host_vars over every
other.
2026-09-26 01:27:07 +02:00
kevinveenbirkenbachandClaude Opus 5 aceb5295ef feat(docs): let the documentation build generate what make docs generated
Cosmos, Quick Setup and Credits follow from meta/services.yml, categories.yml
and galaxy_info.author, yet 279 role READMEs carried a copy that went stale
whenever the metadata moved and nobody re-ran make docs. The documentation
build now runs the generators itself, against its own scratch checkout, and
the committed READMEs keep only the prose a person writes.

utils.cache.PROJECT_ROOT derives from the module's location, so putting the
checkout first on PYTHONPATH makes the generator operate on the scratch tree.
i18n.py and builder.py add it; the one remaining coupling, a relative_to over
PROJECT_ROOT in a log line, is gone.

CI extracted the "### Production" block from the committed README with awk.
Stripping the section made guide_deployable() return "" for every role, which
selects no deploy row and turns nothing red: a fail-open regression. It now
asks _is_role_invokable, which matches the old condition exactly (178 roles
either way, both difference sets empty), and cli/build/docs/readme/production.py
renders the block for the two guide scripts. All 178 renders are byte-identical
to what HEAD served, and both scripts' extraction pipelines produce identical
output for a web app, a service and a desktop role.

test_readme.py inverts: the three sections are now a hard failure in a
committed README. readme_md.md documents what each is derived from.

Removed: the docs, cosmos, readme-generate, readme-index and readme-check
targets, the five aliases pointing at them, and docs from make quality.

Verified by running generate_commands() in series against a copy of the
checkout: every command exits 0, no README is left without Cosmos or Credits,
and exactly the 178 invokable roles get a Quick Setup. The ansible_roles step
needs pandoc, which only the docs image carries, so that one link of the chain
is unexercised here; it is unchanged by this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 01:09:22 +02:00
kevinveenbirkenbach f2cee9bb34 Merge branch 'feature/svc-ai-litellm/remote-models-config' into feature/store-i18n
# Conflicts:
#	cli/build/docs/readme/generate.py
#	docs/requirements/035-mcp-proxy-expansion.md
#	roles/svc-ai-lmstudio/meta/services.yml
2026-09-25 16:45:16 +02:00
kevinveenbirkenbachandClaude Opus 5 ac54e2c487 test(compose): pin the GPU count the file list now depends on
Both modules pin every environment variable compose_file_args() reads. Adding Profile.gpu_reservable() in 4d09286d8b added INFINITO_GPU_COUNT as a reader without pinning it, so the expectation silently followed the ambient environment: green on a shell that has no such variable, red in the test container where BASH_ENV injects .env and this host reads 'all'. That was eight of the unit failures in the gate.

Pin it to 0 in the three affected fixtures and add the two cases the gate never had: no GPU means no override, a non-zero count adds it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-25 14:53:50 +02:00
kevinveenbirkenbachandClaude Opus 5 53c120aaac refactor(dsk): rename the desktop tier and give its agents a category
Every desk- role becomes dsk-, and the three coding agents move to dsk-gnt- under a category of their own. The category carries a title, hanzi and icon but no invokable flag: an invokable child under an invokable parent is rejected outright, which is also why web itself is not invokable.

Three things a search and replace would have broken without a sound. get_entity_name now strips dsk-gnt-, so the primary entity keys agnt-claude, agnt-cursor and agnt-pi no longer resolved and all three roles dropped out of the CI plan. Eighteen run_once guards carried the old name against a convention its own test enforces. DESKTOP_ROLE_PREFIX in the addon spec lint is a literal, so the desktop exemption it grants would have matched nothing and the lint would have moved to the wrong rule in silence.

docs/requirements/035 is not a historical document but a register a lint holds to every role with an application_id, so its list carries the new ids. In 026 only the links move; the prose stays as a statement about the tree at the time it was written.

Measured against the plan with the rename normalised, the deployed role#variant set is unchanged. A stale id in an inventory is caught before any deploy starts: the pre-flight validator reports Invalid group 'desk-firefox' (not in application_ids).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 22:15:41 +02:00
evangelostsak 0a6700e378 Merge origin/main into feature/svc-net-wireguard
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.

Seven conflicts:

- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
  in e706170a0 were modified upstream. The deletion stands, which is the whole
  premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
  legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
  and this branch's _write_mesh call after it.

The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
2026-09-24 21:14:25 +02:00
kevinveenbirkenbachandClaude Opus 5 d6c4646b1e fix(deploy-verify): let required_by waive a role the runtime gates off
tasks/stages/03_destructor.yml includes test-e2e-cli only `when: RUNTIME in [dev, act, github]`, so a production host skips it by design. The coverage verifier demanded it anyway and failed every partial deploy there with "required role(s) did not execute". It stayed hidden because a full deploy passes no --id, which leaves allowed_applications empty and the guard unarmed.

required_by takes a `runtimes` list beside the existing compose/swarm blocks, read before the mode narrowing replaces rb. An unknown or empty runtime waives nothing, so the guard keeps its teeth wherever the runtime cannot be established; only a known runtime the list omits drops the requirement. This is 48753de136 one axis further: a role that a deploy correctly skips must not be demanded of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 18:21:49 +02:00
kevinveenbirkenbachandClaude Opus 5 25dff2a780 fix(i18n): stop the translator from losing entries the server only refused
Four defects in the client surfaced in one run. LibreTranslate serves Chinese as zh-Hans and zh-Hant, never as bare zh, so the readiness wait blocked on a target that was never coming and the run stood still for an hour without printing a line. A blanket except turned every transient refusal into a permanently discarded entry, and the fallback path answered a failed batch by firing twenty single-text requests at the same overloaded server; the same catalogs that lost 130 to 295 entries came back with 0 discarded once the batch is retried with backoff first.

The per-catalog count conflated a refusal with a damaged protected span, which is measurably wrong: of 228 entries left pending in core/sw only 36 carried a span at all. Outcome now carries values, refusals, damage and the last refusal message per call rather than on the client, where twenty lanes mixed their numbers into each other. That split is what surfaced the 502 the server had been answering with. The throwaway container also ran with check=True and capture_output=True, so docker error text died inside the swallowed exception and the log showed an argv list and rc 125.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 09:54:10 +02:00
kevinveenbirkenbachandClaude Opus 5 a0a24e2c0e feat(i18n): deploy libretranslate into the i18n runner before translating
server() now resolves a translation endpoint in three steps: use the running
service, rebuild when the host has a GPU but no accelerated deploy left its
stamp, otherwise deploy. A plain run therefore costs nothing, and the rebuild
fires once, on the move from a CPU-built container to an accelerated one.

deploy() brings the runner up, waits for State.Running and deploys the role
inside it. The exec steps address `docker compose exec i18n` directly:
cli.administration.deploy.development exec hardcodes the `infinito` service
(compose.py:261), so every deploy through it silently ran in the main
container, where mailu already holds port 25 and postfix cannot start.
check=True stops a failing step instead of carrying on against the wrong
target.

The deploy router writes its pid to build/deploy.pid, which survives the exec
into the subscripts, and deploying() refuses a translation while that process
lives. The running deploy is left untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 01:53:19 +02:00
kevinveenbirkenbachandClaude Opus 5 3475b8ae6f feat(i18n): deploy libretranslate in an isolated compose runner
The i18n toolchain gets its own runner behind the `i18n` compose profile,
with its own inner docker daemon and docker state, so an i18n deploy shares
no ansible run and no container state with `infinito`. It reuses the image
that service builds (pull_policy: never), which starts it in under a second.

cache-consumer.yml holds the package-cache extra_hosts once and both runners
take them through `extends`; a YAML anchor cannot cross a compose file, an
extends can.

GPU reservation moves from a layered override file to INFINITO_GPU_COUNT on
both services. `gpus:` only accepts the literal `all`, so a variable there
invalidates the project on a CPU host, while a device reservation takes
`count: 0`. compose/gpu.override.yml and gpu_runtime_available() go with it.

test_compose_override_services_defined.py fails when an override extends a
service no compose file defines, which renders the whole project invalid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 01:37:37 +02:00