Commit Graph
78 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 944a228ffb fix(tests): import the mock submodule the deploy-container suite uses
test_container.py decorates with @unittest.mock.patch but imports only
unittest. A submodule is not bound as an attribute by importing its
package, so the name resolves only while some other module in the same
interpreter has already imported unittest.mock.

That makes the defect invisible in a full run, where another suite
imports it first, and fatal under a narrow INFINITO_TEST_PATTERN or on
an xdist worker that receives this file alone: collection dies with
AttributeError: module 'unittest' has no attribute 'mock'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-14 00:00:41 +02:00
kevinveenbirkenbach de3127683f Merge branch 'feature/designer' into feature/store-i18n 2026-10-13 12:56:06 +02:00
kevinveenbirkenbachandClaude Opus 5 7c13e96c96 Merge branch 'main' into feature/store-i18n
Twelve files conflicted; most resolutions keep both sides rather than pick
one.

cli env.py and compose.py: both branches extend the same spots. The tools
override is appended before write_override, because that call filters the
mirror override to the compose files it is handed. compose.py keeps this
branch's CACHE_ROLE_SHELL_DIR, since the cache scripts moved into a role and
main's literal scripts/docker/cache path is their old home, and takes main's
mirrored init image alongside it.

utils/docker/image/discovery.py: this branch's version_variants expansion
plus main's whole runner-image block, which runner_mirror imports.

sys-svc-compose network create: main's side wins on substance and this
branch's on structure. main carries eda6249d7b, the fix for a probe that
matched an id prefix and let a sibling shadow the exact name; this branch had
refactored the task into create/overlay.yml and create/bridge.yml but brought
back the docker_network_info probe that fix replaced. The subfiles stay and
now probe through the exact-name lookup. main's half also read
SWARM_CLUSTER_AVAILABLE, which this branch removed in favour of
IS_SWARM_MODE, as the merge base confirms, so the gates follow this branch.
The lookup's import follows this branch's get_entity_name to entity_name
rename.

web-app-pixelfed: the custom entrypoints stay removed, since the new image
brings its own and the variables behind them no longer exist, and main's
host_alias=true comes along.

web-app-suitecrm: main's session.yaml and its config/packages path, because
only that version wires framework.session and only that directory is read by
Symfony.

web-app-odoo: main's onionOptions import is required by the test.use below
it; this branch's test-seaweedfs registration stays.

Four defects surfaced that no conflict marker shows.

The auto-merge of web-app-suitecrm/vars/main.yml duplicated
SUITECRM_SESSION_CONFIG_HOST, which ansible-lint rejects. Removing it exposed
the second: SUITECRM_SESSION_CONFIG_DOCKER still named config/session.yaml
while the mount had moved to config/packages, so the freshness comparison in
tasks/01_manager_ops.yml would have tested a path nothing writes.

svc-db-elasticsearch carried environment: on two include_tasks, which Ansible
accepts on include_role only. ELASTIC_PW now sits on the five tasks that read
it, which is also the tighter scope for a credential.

The openclaw templates render NETWORK_TRUSTED_PROXY_CIDRS while their test
renderer passes only its explicit kwargs, so seven tests raised Undefined.
That predates the merge: the template line is on this branch and not on main,
and both sides share the same test file.

test_runner_mirror neutralises four environment variables under clear=False
and missed three, so the gpu and tools overrides leaked in from the container
and the assertion on the layered list broke. It now neutralises all seven.

make quality-high passes both phases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-13 10:57:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 f7987679dd fix(ci): read every page of the job listing before reporting failed roles
report_failures asked `gh api --paginate` for the run's jobs with the jq
filter `[.jobs[] | {name, conclusion}]`. gh applies that filter to each page
and prints one array per page, so a run with more jobs than one page holds
produced two JSON documents, and json.loads stopped with "Extra data: line 2
column 1". Main run 38218402254 (113 jobs) failed its "Report main role
failures" job that way, and no issue was opened for its failed roles.

The listing now emits one job object per line and parses the lines one by
one, the same way logs/download/github.py lists jobs.

Verified: the old parse of the real output of run 38218402254 raises the
same "Extra data: line 2 column 1 (char 8491)" as CI; list_jobs on that run
returns all 113 jobs, 16 of them failed, grouped into 12 failed roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:46:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e31b9557d2 fix(env): keep the host .env when the runner container runs make setup
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.

compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.

test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.

Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
kevinveenbirkenbachandClaude Opus 5 9204f053dd fix(ci): keep the star on a priority row the sweep budget missed
cells() awarded the priority glyph only in the branch that has a chunk, so a
priority row beyond the budget rendered as a bare disabled mark,
indistinguishable from an ordinary row the sweep did not reach. The two need
different answers: --offset indexes the regular line only, so it never reaches a
missed priority row, which needs a shorter priority line or a run of its own.
The status now carries both glyphs and the module docstring says what to do
about it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-12 18:04:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 25a1f08ab7 feat(design): leave the design queue by switching the design service off
The design queue skipped a hand-kept UI_LESS_ROLES list in
cli/meta/roles/design. The decision now lives in each role: a web-app- or
web-svc- role without a UI to design sets design.enabled: false in its
meta/services.yml, and design_disabled() reads that flag. web-svc-design,
the provider, stays out by its entity name.

A new lint, test_design_off_reason.py, requires a design entry in every
web-app- and web-svc- role and, where the entry is switched off, a
"# nocheck: design-off  Reason: <text>" on the design: line or the comment
lines directly above it. The rule is in the suppression catalog, and
requirement 037 decision 13 describes it.

Twelve roles switch design off with a reason checked against their code:
asset and simpleicons were already off; file, onlyoffice, bridgy-fed and
mig were on and no longer receive the shared CSS and design script; cdn,
collabora, coturn, legal, mirror and xmpp had no entry. The four that were
on drop their bond and their design variant overrides, which the
disabled-bond and variants-static-override lints reject on a literal false.
The queue lists 77 candidates; HEAD's list also counted bridgy-fed, mig,
collabora and onlyoffice. Cosmos diagrams and the integration matrix are
regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 17:09:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0327114c44 fix(inventory): drop disable= toggles once their key leaves the list
Provisioning merges into host_vars and never deletes, so the false
enabled/shared toggles that apply_services_disabled writes for a key stayed
in the round inventory after a later run dropped that key from disable=.
That covered the provider's own entry too: with seaweedfs once disabled,
web-app-seaweedfs kept shared: false, the service registry offered no
seaweedfs provider, container_networks attached no consumer, and BookWyrm
ran USE_S3=true against http://seaweedfs:8333 without a route.

reconcile_services_disabled_from_env records the keys of each disable run
in the host_vars key _INFINITO_SERVICES_DISABLED. On the next run it deletes,
in every application block, the enabled/shared toggles that are still
exactly false for a recorded key the current disable no longer lists, drops
a block left empty, and rewrites or removes the record. It runs right after
ensure_host_vars_file, before credentials and the variant bake, so a variant
that wants a toggle off writes it again. With an unchanged or empty disable
it does nothing. An inventory written before this change has no record, so
its older toggles are only reverted after one run with the old list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 11:07:21 +02:00
kevinveenbirkenbach 1fb98b5b76 Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-10 10:59:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ba796d7f1 fix(ci): pull the images the runner uses itself from the GHCR mirror
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.

iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.

Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:41 +02:00
kevinveenbirkenbachandClaude Opus 5 d3610af31f fix(disable): turn off a service key that only a variant bakes
`disable=lam` aborted the local deploy of web-app-openbao with "`disable`
conflicts with the current inventory state", and web-app-moodle carries the same
construction in two of its variants.

The writer and the conflict check disagreed on scope. apply_services_disabled
walked `find_roles_with_service`, the roles whose meta/services.yml declares the
key, so neither role was in that set for `lam` and nothing was written.
find_services_disabled_conflicts walks every deployed app's host_vars instead,
found the `lam` block the variant had baked, and refused the run. Dropping
`lam` from the list is not an answer either: the round then deploys web-app-lam
although ldap is off.

The writer now also flips a key it finds in host_vars under an app whose role
does not declare it, and only the flags that are already in that block. It never
creates a key or a flag, which is what `_declared_toggles` exists to prevent:
compare_application_keys rejects a host_vars key the role contract does not
carry. Overwriting a value the variant already wrote introduces nothing, so that
rule is untouched; test_skips_app_whose_role_does_not_define_service still
passes and pins it.

The alternative the TODO proposed, declaring `lam` in meta/services.yml with the
`'web-app-lam' in group_names` flags, is the worse trade: a consumer entry is
resolved through the service registry, so it would bind web-app-lam and its
whole closure into every round that carries openbao or moodle, for a key both
roles read only to gate a Playwright spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-09 22:48:06 +02:00
kevinveenbirkenbach ebad39422c Merge branch 'feature/designer' into feature/store-i18n 2026-10-06 00:56:17 +02:00
kevinveenbirkenbachandClaude Opus 5.5 86cefeff26 feat(web-svc-design): derive one corporate design from a single base color
web-svc-css becomes web-svc-design and owns the configuration that lived in group_vars/all/13_design.yml. Consumers switch their css service key to design, and sys-front-inj-css becomes sys-front-inj-design. javascript stays an independent key; a role's optional design.js.j2 is injected together with the design and CSP-hashed like the functional script.

An in-repo OKLCH lookup replaces colorscheme_generator. From design.colors.base alone it derives surfaces, text, borders, the primary and status fills with their on-colors, hover and active states and a dedicated dark palette; a unit test proves WCAG AA, and AAA for body text, across six base colors in both modes. Semantic --design-* tokens replace --color-NN-SS in every role stylesheet. The shared element styles sit in the cascade layer infinito-design, so an app's own design system always wins and takes the palette through a variable mapping in its role stylesheet or through its own theming mechanism.

design.logo and design.title resolve globally from web-svc-design or per role from its design entry, and false or 0 disables the replacement. The design_assets action renders the logo lockup per slot as SVG, PNG and ICO, and the legacy title keys move into the design entries.

Three roles carry the design end to end. The dashboard takes logo, favicon and title from the generator and replaces the hard-coded port-ui colors. Gitea maps its theme variables onto the tokens and mounts the generated logo and a branded start page. Nextcloud receives primary color, logo, name and slogan through occ theming and ships no stylesheet.

make design-sync re-renders the static design files of a deployed app, make design-gallery captures before and after screenshots in light, dark, desktop and mobile, and make design-queue orders the roles that are due for a design pass. The i8-designer skill and docs/agents/action/design.md describe that loop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 06:04:14 +02:00
kevinveenbirkenbachandClaude Opus 5 e2948332fd fix(merge): repair what the main merge left broken
The merge brought in main's new roles and carried this branch's cross-cutting
renames and lints past them, so files that never conflicted were left
inconsistent. make quality-high is green again: 4 test targets, 15 lint targets.

a05071fe78 renamed get_entity_name to entity_name. Three files main added still
used the old name and merged cleanly because this branch never touched them:
plugins/lookup/email.py raised ImportError on import, which also took down
sys-ctl-hlth-msmtp's mail_recipient lookup; web-app-stalwart/tasks/00_core.yml
applied `| get_entity_name`, a filter that does not exist (the registration in
plugins/filter/get/entity_name.py is `entity_name`), so a compose deploy of
stalwart would have failed while rendering sys-stk-front-proxy's vars; and
test_database.py patched a name the module no longer binds.

web-app-stalwart and web-app-mailu spell DEPLOYMENT_MODE out in 15 places. The
named booleans in group_vars/all/18_swarm.yml are the SPOT and the bare form is
the idiom: 177 existing uses, and web-app-listmonk/tasks/utils/mcp.yml:42,49
drives both as mutually exclusive `when:` conditions, which a truthy string
would break.

svc-net-tor-smtp pinned aiosmtpd and PySocks inside its Dockerfile, where
Dependabot cannot see them. The pins move to files/requirements.txt; since
sys-svc-compose stages only the Dockerfile, 00_core.yml now stages the
requirements file into the build context too.

stalwart's webui_version had no upstream declared. Its three inline
`environment:` blocks stay, with a nocheck each: the role sets no
docker_compose_env, so one templates/env.j2 would also be the env_file of the
sibling containers, and clamav would receive the recovery admin credential and
the OIDC client secret.

The 13 bindings in stalwart's playwright env.js defaulted to "" and were
consumed unconditionally. Every key is emitted unconditionally by
playwright.env.j2 from a lookup that fails rather than yields empty, and
OIDC_ISSUER_URL always concatenates the realm path, so requireDotenvValue is
the correct binding. gitlab's adminUsername gets the toBeTruthy guard its two
siblings already had.

web-svc-api and svc-cache-package collided with stalwart on local.http 8094 and
8095 and move to 8098 and 8099, which exhausts the 8001-8099 band.
svc-ai-agent-broker collided with stalwart on 192.168.223.0/24 and moves to
192.168.81.0/24; the suggester only subdivides blocks already in use at the
requested prefix, so the move was validated with --block.

web-app-weblate calls lookup('email') without emitting
lookup('container_extra_hosts'), so on an onion deploy its SMTP client drops
every message without an error.

test_main.py read the primary domain from a literal while main moved
INFINITO_DOMAIN to main.infinito.test. It now takes the value from
default_domain_primary(), in the shape e2beab9f1c uses on
feature/web-app-wazuh/add-role, so that branch merges here without a conflict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 03:21:09 +02:00
kevinveenbirkenbachandClaude Opus 5 f2fedfc677 Merge branch 'main' into feature/store-i18n
Fourteen real conflicts plus the 66 generated READMEs.

utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.

plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.

roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.

group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.

docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.

inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.

roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.

The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 01:57:17 +02:00
kevinveenbirkenbachandClaude Opus 5 0d2fadce0c feat(cache): serve every checkout's own cache declarations
A checkout that declared a new cache host rendered the artefacts and reached nothing: the Nexus proxy repository, the frontend's leaf certificate and the nginx upstream map all live in the running stack, which only the owning checkout set up. cache-apply installs all three and reloads the frontend, and compose up runs it for a checkout that merely joins the network, so the manual call is left for a declaration changed between two bring-ups.

The map source is no longer selectable. INFINITO_CACHE_CONF_SOURCE existed to let a worktree read the primary checkout's map instead of its own, but the frontend mounts one map and the install replaces it, so the checkout that applied last wins either way: the knob chose which file to point at, never who prevailed. Dropping it also retires the git subprocess that resolved the owning checkout on every env build.

Two checkouts sharing a stack therefore overwrite each other's map, which the documentation now states instead of implying a selector could prevent it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-03 21:40:34 +02:00
kevinveenbirkenbachandClaude Opus 5 d88fe1d83a test(lint): let the legacy-path guard see the trees that still named the old layout
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.

So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.

`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.

Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.

`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.

Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.

The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 21:27:40 +02:00
kevinveenbirkenbachandClaude Opus 5 058e76e58c fix(test): read the two planner call sites through the shared cache
`test_no_uncached_file_reads` requires every read under tests/, utils/, cli/,
plugins/ and library/ to go through `utils.cache.files.read_text`, which keeps
one decoded read per file for the whole pytest session. The test added in
3b85e4e1e1 called `Path.read_text` directly and so read init.py and
deploy/cli.py past that cache.

The rule allows `# nocheck: cache-read` only for a read outside the project
tree, which is how bfdf3dee79 handled a tempdir. These are real source files,
so the reader is the answer and the marker would have been the convenient
wrong one; e5430ad4ca is the matching precedent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 02:34:39 +02:00
kevinveenbirkenbachandClaude Opus 5 3b85e4e1e1 fix(deploy): let init pin the guide host the way deploy already does
adb53392bd taught the deploy planner to pin the instructions host to its
smallest variant and to leave a pinned app out of the round total. It taught
only the deploy side. `init` writes the inventory and `deploy` reads it back,
and the directory carries a round suffix only when the plan holds more than
one round, so the two sides named different directories:

    >>> Inventory initialized at /home/runner/inventories/github-0
    === matrix-deploy: round 1/1 inv=/home/runner/inventories/github
    [ERROR] Some application_ids are invalid for this inventory
    [DEBUG] inventory file does not exist

Nine jobs of run 36596159856 died there, before ansible-playbook started. The
trigger is a subject with a single variant: init counted the host's two
variants and suffixed, deploy pinned the host out and did not. svc-ai-s1
declares two variants of its own and was the only instructions row to survive,
which is the shape of the bug read backwards.

`variant_select` opens by stating the invariant this broke - "keeps init and
deploy walking the exact same selection (SPOT)" - so the fix is to honour it
rather than to special-case the suffix.

The accompanying test asserts both entry points forward `pinned_variants`.
The five tests that shipped with adb53392bd all exercised the pin helper on
its own, which is why nothing caught a caller that never asked it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 01:40:55 +02:00
kevinveenbirkenbachandClaude Opus 5 adb53392bd fix(ci): deploy the instructions host in the shape its README documents
A row marked for the instructions replay gets `web-app-docs` appended to its
apps, and the planner hands every primary app the round's index. The host
therefore took the subject's index, which for a passenger is an arbitrary
shape. Measured in run 36558500327: the same appended role deployed
dashboard, matomo, prometheus and css in 141 tasks on a round-0 row and in
none at all on a round-1 row, purely because of whose round it was riding.

The README's Production block describes the bare role, so the host now takes
its smallest variant in every round. The index comes from `smallest_variant`
rather than a literal, so it follows `meta/variants.yml` instead of drifting
from it.

The row names the host in an environment variable. `--apps app#variant` would
have read better at the call site but needs a new matrix column in both
`axes.py` and `cli/meta/ci/plan.py`, while the deploy CLI already reads
`disable` from the environment and the four shell layers between the workflow
and the CLI pass it through untouched.

A pinned app no longer counts toward the round total: a passenger should not
lengthen the subject's sweep. That removes an unreachable round rather than
coverage, because `deployed_rounds` plans one app at a time and a role with
one variant therefore only ever yields `variant: 0` rows, which the workflow
pins on every deploy.

The host is left alone when it is also the subject. `web-app-docs` carries the
replay on its own rows too, where the apps list reads `web-app-docs,
web-app-docs` and the round index is the entire point of the sweep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 18:00:45 +02:00
kevinveenbirkenbachandClaude Opus 5 16c853545a fix(deploy): stop the dedicated deploy from git-cleaning its own checkout
`MODE_CLEANUP` says what to tidy on the deployed system: prune unused Docker
resources, clear Mastodon caches, switch an LDAP user list from `present` to
`exact`. Eight roles read it that way. The dedicated runner read the same flag
and ran `make clean` in the control repository instead, which is
`git clean -fdX` over every ignored path there.

`inventories/production/` is one of those paths. The published host install
guide provisions into it and then deploys against `$INVENTORY/.password`, so
the deploy deleted the vault password it was about to use:

    Removing inventories/production/
    [ERROR] The vault password file .../inventories/production/.password was
    not found

Reproduced in CI on two unrelated rows, arm64/zfs/debian and
amd64/btrfs/centos, so it is neither architecture nor filesystem specific.

The coupling was a name collision, not a decision. 7e5990aa16 generated the
MODE_* flags from `01_modes.yml` and removed the legacy skip flags, and the
repository clean - which had had a flag of its own - was attached to the
generated `MODE_CLEANUP` because one now existed by that name. No other deploy
path does this; `container`, `development` and `swarm` never call `make` at
all.

`make clean` keeps its own entry points, `make setup-clean`, `make clean-sudo`
and `infinito --git-clean`, where a caller asks for it deliberately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 17:24:05 +02:00
kevinveenbirkenbachandClaude Opus 5 62a8f1ce2a feat(ci): retrigger a failed role whole and an unfinished one by variant
--roles-only collapsed the entire failed set, so a job that never reached a verdict came back as a bare role name and the rotation re-picked axes the run had already proved green, dropping the one combination it never got to. Nothing distinguished a verdict from an interruption.

collapse_failures() takes the failed set and the hard-failure subset: a role that failed outright collapses to its name, everything else keeps the exact role#variant@mode+tor it was given. A role with a hard failure absorbs its own unfinished rows, since the collapsed name already covers every variant of it.

With --strict the two arguments are the same list, so i8ciall, i8ciallon and i8cifo are byte-identical to before; a test asserts that agreement rather than leaving it to inspection. The new combination is exposed as i8civar, which is i8ciallon without --strict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 13:33:41 +02:00
kevinveenbirkenbachandClaude Opus 5 c782208f99 feat(tools): rename the i18n lane to tools and give the cache paths one spot
The lane is no longer LibreTranslate-only, so `i18n` is the wrong name for
the runner, its compose override, its inventory and its env keys. Renames
to `tools` throughout and points the lane at
`svc-ai-libretranslate-engine`, which is the role that actually owns the
`libretranslate-engine` service key.

The six cache host paths were each written out in full. They now derive
from `DIR_CACHE`, one plain string in `group_vars/all/05_paths.yml`, via a
`cache_paths` handler; the generated values are byte-identical. The handler
is registered directly after `cache_stack` rather than late, because
`registry_cache_max_size` reads `INFINITO_CACHE_REGISTRY_HOST_PATH` and
returns early when it is empty, which silently dropped
`INFINITO_CACHE_REGISTRY_MAX_SIZE` from the generated `.env` and broke
every `docker compose` invocation that layers `cache.override.yml`.

`INFINITO_TOOLS_MODELS_HOST_PATH` derives from the same base, so the model
bind the lane mounts follows the cache layout instead of repeating it.

The inventory pins `DOCKER_BIND_HOST=0.0.0.0`: the group_vars default keys
off `DOCKER_IN_CONTAINER`, which evaluates false for the inner deploy and
would leave the lane's services on the runner's loopback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 23:50:26 +02:00
kevinveenbirkenbachandClaude Opus 5 a37039f0cb fix(web-app-docs): name the replayed role instead of taking the first one
The instructions test brings up a machine per role, so it can only ever
replay one guide. Which one it picked was a side effect of ordering: the
lookup returned the first invokable peer the round listed, and on a CI row
that is the role under test only because the row's apps happen to read
<role>,web-app-docs.

It now reads APP_ID, which the deploy workflow already sets to the row's app
and which the swarm routine already documents as the app under test. A round
that carries no APP_ID, or one whose app publishes no Production block, draws
an invokable peer at random instead, so a local round covers a different
guide each time rather than always the first one the inventory lists.

Ansible runs inside the infinito container rather than on the runner, and
that exec passes an explicit allowlist, so the name arrives only where it is
forwarded. Two seams carry it: _FORWARDED_ENV_KEYS, which covers CI and the
local initialize and reinstall verbs, and the update verb's own docker exec,
which assembles its arguments without that CLI.

The pin is tolerant rather than strict. APP_ID is set on every deploy row and
not only on documented ones, so refusing an app that publishes no guide would
let a generic variable abort a deploy.

Measured: the draw is observed over 200 rounds rather than assumed, and the
forwarding is negative-controlled, removing the key and watching the new test
fail on that key alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 09:16:29 +02:00
kevinveenbirkenbachandClaude Opus 5 666312dfbd refactor(i18n): translate against the dedicated LibreTranslate alone
server() preferred the deployed service and fell back to a throwaway
container when it did not answer. That container ran the upstream image
directly, with LT_UPDATE_MODELS on and neither the package-cache CA nor
the argos-net.com hijack, so it refetched gigabytes from upstream on
every run while the deployed service already carries those models in its
image, installed through the cache at build time. The fallback is gone.

It was reached by accident rather than by failure. deploy() ended in
await_service(), which polled for 300 seconds; the first start of the
service installs its models and takes far longer, so the wait expired,
deploy() reported nothing running, and the caller spawned the throwaway
beside the service it had just deployed successfully. await_service and
SERVICE_TIMEOUT_SECONDS are removed, leaving the single readiness wait
the client already performs with the budget those models need.

Step 0 of the deploy also stopped forcing a recreate. It rebuilt a
healthy lane on every invocation and took whatever model install was in
flight with it; compose recreates on a changed configuration by itself.

The wait no longer looks like a hang: it reports how many of the wanted
languages the server offers, which ones are still missing, how long it
has waited against its budget, and what the lane has pulled. i18n-tune
waits the same way and gets the same report.

service_url() and lane_container() replace the two places that each
rebuilt one of those strings, and deploy() reads the environment once.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 10:56:42 +02:00
kevinveenbirkenbachandClaude Opus 5 44473f7de1 refactor(i18n): move the dev files into their conventional homes
The top-level i18n/ directory held a compose override and a development
inventory and nothing else, so neither sat with its siblings. The override
joins compose/, whose entries are already named <name>.override.yml, and
the inventory joins inventories/development/, which is what it is. The
directory is gone.

Both moves are pure renames; only the four sites naming the old paths
change, and the -f argument in compose_file_args is the one that decides
which override every docker compose call in the i18n lane layers on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 10:37:18 +02:00
kevinveenbirkenbachandClaude Opus 5 50091194d5 fix(deploy): keep a non-invokable prerequisite out of the round's app list
Every desktop job of run 36246124624 aborted before ansible started:

    [ERROR] Some application_ids are invalid for this inventory:
      - user-workstation: not present in inventory

The include list the planner returns becomes two things at once: the
inventory provisioner's --include, and the deploy ids validate_application_ids
checks against the resulting inventory. The inventory is built from
get_all_invokable_apps, and utils/roles/stage.py loops a stage only over
invokable category paths, so a non-invokable id in that list is one no group
can hold and no stage can dispatch. `user-workstation` landed there because
the desktop-base service edge pulls it into the closure.

Measured over all 179 invokable roles, old resolution against new:

    rounds losing a member: 33
    distinct dropped roles: ['dev-java', 'user-workstation']

`user-workstation` in all 33 dsk rounds, `dev-java` in dsk-bluray-player,
which carries it as a meta dependency and would have aborted the same way
on main the day a round picked that role.

Neither stops running. A meta dependency resolves at role-include time,
independent of the inventory. `user-workstation` reaches its host through
the service loader, whose preload closure is computed from the filtered
whitelist and still contains it, and through dsk-base's own include_role:

    dsk-firefox: include=['dsk-base', 'dsk-firefox']
                 user-workstation in preload closure=True
                 workstation service required=True

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 23:06:58 +02:00
kevinveenbirkenbachandClaude Opus 5 c39f3ec9ee feat(web-app-docs): replay the published instructions in its CLI test
The replay of a role's "### Production" block moves out of the CI guide
scripts and into the documentation role itself. The block is read off the
rendered page over HTTP, so what runs is what a reader copies, and the role
that publishes the page is also the role that proves it.

scripts/github/guide/{one,compose_deploy,host_deploy}.sh and
cli/build/docs/readme/production.py are gone with the workflow step that
called them. The instructions marker now only adds web-app-docs to the row's
apps; test-e2e-cli discovers the role through its new templates/test.env.j2
and runs files/test/test.sh.

The block runs in a throwaway machine, never on the stack host. compose.yml
starts one service built from files/test/Dockerfile, whose base is
INFINITO_PARENT_IMAGE out of the mounted checkout's .env, so the machine is
the stack host's own distribution and the replay exercises the installation
instead of a prebuilt image. files/test/init.sh adds an init when the base
carries none and keeps the full -Syu that ef0f03ea98 established for arch.
scripts/tests/workspace/base/01_install.sh then prepares the machine before
the block runs.

Nothing is skipped any more. The previous host-install branch exited 0 and
passed silently over 65 of the 178 invokable roles; a block without a
"docker run" is now replayed against systemd in that same machine, with the
git clone dropped in favour of the mounted tree under test.

The target role comes from the new guide_role lookup, which reads
list_invokable_app_ids() rather than matching role-name prefixes, and falls
back to the asking role when the round deploys nothing else. deployment.py
grows running_apps() so both lookups share one definition of what the round
deploys. required_by_coverage.role_is_invokable drops its own walk over
categories.yml and delegates to the same SPOT; measured equal across all 280
roles.

Carried in the same index, because the refactor pass ran over this change
set: library.py splits into sites.py and build_queue.py mixins, builder.py
hands translated_languages to catalogs.py and folds three repeated log-tail
and state writes, and placeholders.py becomes a facade over spans.py,
repairs.py and damage.py. No test changed for those splits, and the two
ref-resolution dicts builder.py carried are replaced by _wanted_ref, which
also stops hashing the deployed tree on every tag build.

Verified: 7132 unit and lint tests pass over the whole tree, the guide_role
lookup renders through a real Templar, the base image resolves from .env,
and compose config validates the service. The replay itself has never been
executed; that needs a deploy of web-app-docs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 04:39:48 +02:00
kevinveenbirkenbachandClaude Opus 5 aceb5295ef feat(docs): let the documentation build generate what make docs generated
Cosmos, Quick Setup and Credits follow from meta/services.yml, categories.yml
and galaxy_info.author, yet 279 role READMEs carried a copy that went stale
whenever the metadata moved and nobody re-ran make docs. The documentation
build now runs the generators itself, against its own scratch checkout, and
the committed READMEs keep only the prose a person writes.

utils.cache.PROJECT_ROOT derives from the module's location, so putting the
checkout first on PYTHONPATH makes the generator operate on the scratch tree.
i18n.py and builder.py add it; the one remaining coupling, a relative_to over
PROJECT_ROOT in a log line, is gone.

CI extracted the "### Production" block from the committed README with awk.
Stripping the section made guide_deployable() return "" for every role, which
selects no deploy row and turns nothing red: a fail-open regression. It now
asks _is_role_invokable, which matches the old condition exactly (178 roles
either way, both difference sets empty), and cli/build/docs/readme/production.py
renders the block for the two guide scripts. All 178 renders are byte-identical
to what HEAD served, and both scripts' extraction pipelines produce identical
output for a web app, a service and a desktop role.

test_readme.py inverts: the three sections are now a hard failure in a
committed README. readme_md.md documents what each is derived from.

Removed: the docs, cosmos, readme-generate, readme-index and readme-check
targets, the five aliases pointing at them, and docs from make quality.

Verified by running generate_commands() in series against a copy of the
checkout: every command exits 0, no README is left without Cosmos or Credits,
and exactly the 178 invokable roles get a Quick Setup. The ansible_roles step
needs pandoc, which only the docs image carries, so that one link of the chain
is unexercised here; it is unchanged by this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 01:09:22 +02:00
kevinveenbirkenbach f2cee9bb34 Merge branch 'feature/svc-ai-litellm/remote-models-config' into feature/store-i18n
# Conflicts:
#	cli/build/docs/readme/generate.py
#	docs/requirements/035-mcp-proxy-expansion.md
#	roles/svc-ai-lmstudio/meta/services.yml
2026-09-25 16:45:16 +02:00
kevinveenbirkenbachandClaude Opus 5 ac54e2c487 test(compose): pin the GPU count the file list now depends on
Both modules pin every environment variable compose_file_args() reads. Adding Profile.gpu_reservable() in 4d09286d8b added INFINITO_GPU_COUNT as a reader without pinning it, so the expectation silently followed the ambient environment: green on a shell that has no such variable, red in the test container where BASH_ENV injects .env and this host reads 'all'. That was eight of the unit failures in the gate.

Pin it to 0 in the three affected fixtures and add the two cases the gate never had: no GPU means no override, a non-zero count adds it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-25 14:53:50 +02:00
kevinveenbirkenbachandClaude Opus 5 53c120aaac refactor(dsk): rename the desktop tier and give its agents a category
Every desk- role becomes dsk-, and the three coding agents move to dsk-gnt- under a category of their own. The category carries a title, hanzi and icon but no invokable flag: an invokable child under an invokable parent is rejected outright, which is also why web itself is not invokable.

Three things a search and replace would have broken without a sound. get_entity_name now strips dsk-gnt-, so the primary entity keys agnt-claude, agnt-cursor and agnt-pi no longer resolved and all three roles dropped out of the CI plan. Eighteen run_once guards carried the old name against a convention its own test enforces. DESKTOP_ROLE_PREFIX in the addon spec lint is a literal, so the desktop exemption it grants would have matched nothing and the lint would have moved to the wrong rule in silence.

docs/requirements/035 is not a historical document but a register a lint holds to every role with an application_id, so its list carries the new ids. In 026 only the links move; the prose stays as a statement about the tree at the time it was written.

Measured against the plan with the rename normalised, the deployed role#variant set is unchanged. A stale id in an inventory is caught before any deploy starts: the pre-flight validator reports Invalid group 'desk-firefox' (not in application_ids).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 22:15:41 +02:00
kevinveenbirkenbachandClaude Opus 5 d6c4646b1e fix(deploy-verify): let required_by waive a role the runtime gates off
tasks/stages/03_destructor.yml includes test-e2e-cli only `when: RUNTIME in [dev, act, github]`, so a production host skips it by design. The coverage verifier demanded it anyway and failed every partial deploy there with "required role(s) did not execute". It stayed hidden because a full deploy passes no --id, which leaves allowed_applications empty and the guard unarmed.

required_by takes a `runtimes` list beside the existing compose/swarm blocks, read before the mode narrowing replaces rb. An unknown or empty runtime waives nothing, so the guard keeps its teeth wherever the runtime cannot be established; only a known runtime the list omits drops the requirement. This is 48753de136 one axis further: a role that a deploy correctly skips must not be demanded of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 18:21:49 +02:00
kevinveenbirkenbachandClaude Opus 5 25dff2a780 fix(i18n): stop the translator from losing entries the server only refused
Four defects in the client surfaced in one run. LibreTranslate serves Chinese as zh-Hans and zh-Hant, never as bare zh, so the readiness wait blocked on a target that was never coming and the run stood still for an hour without printing a line. A blanket except turned every transient refusal into a permanently discarded entry, and the fallback path answered a failed batch by firing twenty single-text requests at the same overloaded server; the same catalogs that lost 130 to 295 entries came back with 0 discarded once the batch is retried with backoff first.

The per-catalog count conflated a refusal with a damaged protected span, which is measurably wrong: of 228 entries left pending in core/sw only 36 carried a span at all. Outcome now carries values, refusals, damage and the last refusal message per call rather than on the client, where twenty lanes mixed their numbers into each other. That split is what surfaced the 502 the server had been answering with. The throwaway container also ran with check=True and capture_output=True, so docker error text died inside the swallowed exception and the log showed an argv list and rc 125.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 09:54:10 +02:00
kevinveenbirkenbachandClaude Opus 5 a0a24e2c0e feat(i18n): deploy libretranslate into the i18n runner before translating
server() now resolves a translation endpoint in three steps: use the running
service, rebuild when the host has a GPU but no accelerated deploy left its
stamp, otherwise deploy. A plain run therefore costs nothing, and the rebuild
fires once, on the move from a CPU-built container to an accelerated one.

deploy() brings the runner up, waits for State.Running and deploys the role
inside it. The exec steps address `docker compose exec i18n` directly:
cli.administration.deploy.development exec hardcodes the `infinito` service
(compose.py:261), so every deploy through it silently ran in the main
container, where mailu already holds port 25 and postfix cannot start.
check=True stops a failing step instead of carrying on against the wrong
target.

The deploy router writes its pid to build/deploy.pid, which survives the exec
into the subscripts, and deploying() refuses a translation while that process
lives. The running deploy is left untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 01:53:19 +02:00
kevinveenbirkenbachandClaude Opus 5 3475b8ae6f feat(i18n): deploy libretranslate in an isolated compose runner
The i18n toolchain gets its own runner behind the `i18n` compose profile,
with its own inner docker daemon and docker state, so an i18n deploy shares
no ansible run and no container state with `infinito`. It reuses the image
that service builds (pull_policy: never), which starts it in under a second.

cache-consumer.yml holds the package-cache extra_hosts once and both runners
take them through `extends`; a YAML anchor cannot cross a compose file, an
extends can.

GPU reservation moves from a layered override file to INFINITO_GPU_COUNT on
both services. `gpus:` only accepts the literal `all`, so a variable there
invalidates the project on a CPU host, while a device reservation takes
`count: 0`. compose/gpu.override.yml and gpu_runtime_available() go with it.

test_compose_override_services_defined.py fails when an override extends a
service no compose file defines, which renders the whole project invalid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 01:37:37 +02:00
kevinveenbirkenbachandClaude Opus 5 8924a06878 feat(ci-trigger): state the selection and its axes, and refuse one that cannot deploy
The trigger could only derive a selection: --failed from a source run's verdicts, --apps as a whitelist, and every axis carried over from that run. A verification run has no source run whose axes it wants to inherit; it wants the combination that answers an open question. --priority states the tokens, and five overrides state the axes.

--priority composes with --apps but collides with --failed, which builds its own line, and it skips the resume-offset computation: that only means something when a source run says where the regular line stopped.

Every dispatch is now validated first. call-orchestrator.yml already runs the same check as its first job and every other job needs it, so a bad token never reaches a chunk. What it reaches is the branch's concurrency group, which entry-manual-steer.yml declares with cancel-in-progress: a doomed dispatch cancels the run currently in flight and then fails its own validation. That cancellation is what this refuses, not the deploy minutes.

The whitelist is passed as empty when it is the all-sentinel, which is not a role name and would be reported as an unknown one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 11:56:24 +02:00
kevinveenbirkenbachandClaude Opus 5 bd7e0ae7fd feat(ci): measure what each deploy axis costs in runner time
The cheapest combination that still answers a question was picked by assertion, because nothing recorded the cost. Deploy job titles already carry the whole row and the API gives every job a start and an end, so the cost per axis value is read back off the runs that already happened.

Measured over six runs of this branch: compose 47m against swarm 83m, clearnet 52m against tor 79m. Choosing the axis is therefore worth minutes per row, not seconds.

Medians, not means, so one queue wait cannot outvote a hundred normal jobs, and a value below the sample minimum is dropped rather than ranked. fastest() answers None there, which means the caller leaves that axis to the rotation instead of guessing it.

The job labels are parsed with the same primitives the retrigger uses, so a title format change breaks both together rather than leaving this one silently measuring nothing. A filesystem row states the kind the matrix assigned, not the kind the deploy ran on, because a fallback is not recorded in the title.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 11:50:29 +02:00
kevinveenbirkenbachandClaude Opus 5 4c98624e07 feat(i18n): gettext catalogs with LibreTranslate machine translation
Core and the documentation get gettext catalogs for all 184 ISO 639-1
languages listed in meta/languages.yml, 47 of which LibreTranslate can fill.

`make i18n-extract` merges the current sources into locale/<code>/
LC_MESSAGES/{core,docs}.po: role descriptions, category and menu titles and
the logout strings for the core domain, the Sphinx gettext output for the
docs domain. A changed source keeps its previous translation as a fuzzy
entry, and catalogs are written deterministically so an unchanged merge
leaves the files untouched.

`make i18n-translate domain=<core|docs> [languages=<codes>]` fills only the
empty and fuzzy entries with a short-lived LibreTranslate container (the
image web-svc-libretranslate pins) and removes it afterwards. It starts the
container, and loads models, only for languages that still have pending
entries. Code spans, placeholders, URLs, tags and dotted names such as
Infinito.Nexus are masked for the request and a translation that damages
one is discarded. An unreachable server stops the run instead of
discarding every remaining entry, so a rerun continues where it stopped;
machine translations carry the translator comment `libretranslate`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 18:22:03 +02:00
kevinveenbirkenbachandClaude Opus 5 c7fb6e60c5 fix(ci): narrow a row's architectures by what its variant deploys
row_architectures intersected the run's pool with the declarations of
every role in the complexity closure. That closure follows every
conditional provider flag through the base graph, so tor -> web-svc-mirror
-> prometheus -> openwebui -> litellm -> lmstudio put the amd64-only
lmstudio into the closure of 93 of 128 roles, including the minimal
variants that never deploy it. A single amd64 pin would have withdrawn
arm64 from nearly the whole matrix.

The matrix now replaces a row's services with the include set that
plan_dev_inventory_matrix, the planner the deploy step itself uses,
resolves for that variant. Against CI run 35674334334 it matches every
data point: lmstudio for zammad#2, moodle#2 and xwiki#2, which pulled it
on arm64, and not for docs#0 or legal#0, which did not. With the pins of
the next commit, 126 of 128 roles keep arm64 rows instead of 34.

Without any architecture declaration the emitted matrix is byte-identical
to before under a fixed discovery seed. The planner adds about 37 s per
matrix build, and a role whose plan errors now stops the matrix instead of
only its own deploy job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 16:01:09 +02:00
Alejandro Roman 9df3287adf Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-22 12:16:44 +04:00
kevinveenbirkenbach eea14a0cfb Merge branch 'main' into feature/store-i18n 2026-09-21 23:35:55 +02:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbach f0598470a7 Merge branch 'feature/ai-agent-employees' into feature/store-i18n 2026-09-19 07:03:43 +02:00
kevinveenbirkenbachandClaude Opus 5 2a7455bd90 feat(ci): spread the deploy matrix over amd64 and arm64
Both architectures were claimed to work and only one was ever exercised. The
deploy matrix now carries an architecture the way it carries mode, tor, distro
and filesystem, so every run proves both.

A job cannot ask for an architecture the way it asks for a distro, because the
architecture is the machine rather than something the job installs on it. The
matrix therefore carries a runner label and `runs-on` reads it, and an
architecture without a label in RUNNERS is a build error rather than a row that
lands on the wrong hardware and still reports green. assert_architecture.sh
checks the runner it actually got, because a label that stops resolving to
arm64 would otherwise go unnoticed forever.

Assignment is rotation, not chance: two architectures against five distros and
fifteen distro/filesystem positions are coprime enough that every combination
comes round. A role narrows the pool through `architectures` in its
meta/services.yml; without one it takes whatever the rotation hands it. An
empty intersection aborts instead of silently removing the role from the run.

Images are built per architecture and pushed under an architecture-suffixed
tag, then merged into the plain tag by manifest_all.sh; pushing the plain tag
from both builds would let the last one win and send half the rows to the wrong
platform. cleanup.sh now protects the `ci-<sha>-<architecture>` children of a
protected tag, because the plain tag is an index over them and deleting a child
leaves it unpullable on that platform.

label.py is split out of axes.py, which the 500-line limit no longer fits.
Writing a job title and reading one back belong together: a retrigger that
recovers an axis wrongly replays a different deploy than the one that failed,
and a drifting artifact name makes the failure reporter download nothing. The
leading `.*` in LABEL_RE is greedy on purpose so it anchors on the last mode
glyph; a reusable-workflow caller path carries a glyph of its own, and
test_selection.py guards the case.

Verified: 313 unit tests and 48 subtests across utils/github/variant,
utils/roles and cli/meta/ci. Locally, a deploy pinned to arm64 reaches ok=1444
and its image builds log aarch64-linux-gnu paths.

Not verified here: actionlint over the seven changed workflows, and a full
`make test`. The matrix doubles the row count, so runtime and runner
availability change for every CI run from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 07:02:44 +02:00
kevinveenbirkenbachandClaude Opus 5 673de05a60 fix(inventory): write only the service toggles a role's contract declares
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:

    Missing default for web-svc-cdn: services.dashboard.shared
    Missing default for web-svc-mirror: services.dashboard.shared
    [ERROR] Inventory validation failed. Aborting deployment.

_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.

Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.

Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.

Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.

Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 06:00:56 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 b356954d2e style(inventory): wrap the keycloak users fixture the way autoformat does
make quality-high pass 75 rewrote it: a concatenation whose right operand is a
multi-line literal gets parenthesised, with the operator leading its own line.

The string itself is unchanged, so
test_narrowing_the_credentials_still_rotates_every_user keeps the declaration
that separates web-app-keycloak from web-app-a and still fails when the user
half of the rotation is narrowed along with the credentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:50:27 +02:00
kevinveenbirkenbachandClaude Opus 5 46110f4311 perf(swarm-test): rotate the round's credentials, not every block in the file
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.

matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.

The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.

The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.

The CLI aborts when --include matches no block instead of rotating nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:32:02 +02:00